← All Neural Networks & Deep Learning modules

Module 3 — The Forward Pass

Putting neurons together · hands-on · about 25 minutes.

Module 1 left us with a cliffhanger: one neuron can’t solve XOR ("output yes when exactly one input is on"), because that needs two cuts and a neuron makes one. The fix was to stack neurons. Here is that stack — a real two-layer network — and the act of pushing data through it from input to answer is called the forward pass.

Input → hidden → output

A network is layers of neurons wired in series. The middle layer is called a hidden layer — "hidden" only because its values are computed internally; they are neither the raw input nor the final output. Each hidden neuron looks at the inputs and reports its own activation; the output neuron then looks at those and makes the final call. Data flows one way: forward.

Layer 1 (hidden) — two neurons each draw their own line through the inputs. One learns roughly "is at least one input on?", the other "is it not the case that both are on?".
Layer 2 (output) — one neuron ANDs those two answers together. "At least one on, and not both" is exactly XOR.

Drag the probe point below. Watch each neuron’s activation light up as the values flow left to right, and see the output neuron land on the right answer — a decision no single straight line could make.

This activity needs JavaScript. The lesson below still covers everything.

That’s "inference"

Running the forward pass on a trained network — feeding in an input and reading out the answer — is what people mean by inference. Every time an app classifies a photo or a chatbot predicts the next word, it is doing a forward pass through a (much bigger) network exactly like this one. The only thing this toy is missing is how the weights got good — and that is the next three modules: gradient descent, backprop, and depth.

A forward pass in Keras — read only, nothing to install

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

model = Sequential([
    Dense(2, activation='tanh', input_shape=(2,)),  # hidden layer: 2 neurons
    Dense(1, activation='sigmoid')                  # output neuron
])
y = model.predict(X)   # the forward pass — input flows in, answer comes out

This is the exact 2-2-1 shape on the canvas above. .predict() runs the forward pass; the weights inside are what training will set.

AI anchor — every prediction is a forward pass The forward pass is the "use the model" step of all of AI. A vision model labelling a tumour, a recommender ranking your feed, an LLM choosing its next token — each is one forward pass through a deep network, the same input-to-output flow you’re dragging here, just with millions or billions of neurons instead of three. Master this little flow and the giants are only bigger versions of it.

Check your understanding

A few questions about the forward pass. You will get a score.

This activity needs JavaScript.

Why this matters next This network already had good weights baked in. Module 4 answers the real question: how does a network find those weights on its own? You will turn a learning-rate dial and watch a live loss curve dive — or explode — as gradient descent trains a net from scratch.

One-sentence summary: a neural network stacks neurons into layers, and the forward pass feeds an input through each layer in turn — the hidden layer builds intermediate features the output neuron combines, which is how a network draws boundaries (like XOR) that no single neuron can.

Next: Gradient Descent in Practice →