← All Neural Networks & Deep Learning modules

Module 1 — From a Neuron to a Network

The building block · hands-on · about 25 minutes.

A deep neural network may appear architecturally complex, but it is constructed from a single elementary unit replicated millions of times: the artificial neuron. Understanding this unit is equivalent to understanding the building block from which every component of modern deep learning — image classifiers, language models, the entire field — is composed. The neuron was introduced implicitly in Course 3 as the linear regression model \( w x + b \) with the addition of one further operation.

The computation performed by a single neuron

A neuron receives a vector of inputs, assigns each input a weight, computes their weighted sum, adds a bias term, and passes the result through an activation function. This constitutes the complete computation:

\[ \text{output} \;=\; \sigma\!\big(w_1 x_1 + w_2 x_2 + b\big) \]

Weights \( w_1, w_2 \) — the magnitude and sign by which each input contributes to the output. A large positive weight indicates that increases in the input correspond to increases in the output.
Bias \( b \) — a learned offset that shifts the neuron's output independently of the inputs.
Activation \( \sigma \) — here the sigmoid function introduced in Course 3, which maps any real input to the open interval \( (0, 1) \), interpretable as a probability-like score.

A single neuron as a linear classifier

The weighted sum \( w_1 x_1 + w_2 x_2 + b \) defines a hyperplane (here, a line) in the input space. The neuron outputs values near 1 on one side of this line and values near 0 on the other. In the activity below, two binary inputs each take the value 0 or 1 — defining the four corners of the unit square. Adjust the weights and bias to position the decision boundary so that the neuron implements a specified logic gate.

This activity needs JavaScript. The lesson below still covers everything.

The necessity of composition: from neuron to network

A single neuron can only produce a linearly separable decision boundary. This is a substantial capability, but it is insufficient for any problem requiring a non-linear boundary — the canonical example being the XOR function (the four input corners on which "exactly one input is on" should be classified as positive). The resolution is to compose neurons: feed multiple neurons the same inputs, then feed their outputs into a subsequent neuron. This composition — a neural network — is the architecture trained throughout the remainder of the course. Module 3 constructs the first such network.

Convince yourself of the limitation first: select the XOR gate in the activity above and adjust the weights and bias freely. Three of the four corners can be satisfied, but never all four — no single straight cut separates XOR’s two "yes" corners from its two "no" corners.

The equivalent neuron in Keras

from tensorflow.keras.layers import Dense

# one neuron: 2 inputs, sigmoid activation — exactly the formula above
layer = Dense(1, activation='sigmoid', input_shape=(2,))
# its weights are [w1, w2] and its bias is b — the parameters you just dragged

A Dense(1) layer constitutes a single neuron. A production network is a stack of Dense layers, each containing many neurons.

AI anchor — the elementary unit of every deep model The neuron studied here is the single repeated unit within every neural network in existence, from minimal three-neuron models to large language models comprising hundreds of billions of weights. The structure of the unit does not change with scale: it is always a weighted sum, a bias, and an activation function. What changes is the number of units and their connectivity. A complete understanding of this single unit is, materially, an understanding of the foundational building block of modern AI.

Check your understanding

Answer a short set of questions on the artificial neuron.

This activity needs JavaScript.

Why this matters next The activation function was applied to map the weighted sum into the interval \( (0, 1) \). Module 2 examines this operation in detail: the principal activation functions and the formal result that a network without non-linear activations cannot represent any function more complex than a linear one.

Summary: an artificial neuron computes a weighted sum of its inputs, adds a bias, and passes the result through an activation function. A single neuron defines a linear decision boundary; composing neurons into a network enables the representation of arbitrarily complex non-linear boundaries.

Next: Activation Functions →