Neural Network Tutor: Replay, Gradients, And Ablation

A neuron computes:

z = w1*x1 + w2*x2 + ... + wn*xn + b
a = activation(z)

Vector form:

z = w dot x + b
a = f(z)

Meaning:

The weights define a direction in input space. The dot product measures how aligned the input is with that direction.

For two input features:

z = w1*x1 + w2*x2 + b

The boundary is:

w1*x1 + w2*x2 + b = 0

That is a line in 2D. In higher dimensions, it is a hyperplane.

one neuron
-> one learned boundary
-> one detector

The bias moves the boundary:

b larger  -> boundary shifts one way
b smaller -> boundary shifts the other way

The activation decides what the neuron emits after computing the boundary score.

Backpropagation is chain rule applied backward through the computation graph.

For the toy neuron:

loss = 0.5 * (prediction - target)^2
prediction = z
z = x1*w1 + x2*w2 + b

Gradient for w1:

dLoss/dw1
= dLoss/dprediction * dprediction/dz * dz/dw1

Substitute values:

dLoss/dprediction = prediction - target = -0.60
dprediction/dz = 1.0
dz/dw1 = x1 = 2.0

So:

dLoss/dw1 = -0.60 * 1.0 * 2.0
dLoss/dw1 = -1.20

For w2:

dLoss/dw2 = -0.60 * 1.0 * (-1.0)
dLoss/dw2 = 0.60

For bias:

dLoss/db = -0.60 * 1.0 * 1.0
dLoss/db = -0.60

With learning rate 0.10:

new_weight = old_weight - learning_rate * gradient

The sign is the central lesson:

negative gradient:
  gradient descent subtracts a negative
  parameter increases

positive gradient:
  gradient descent subtracts a positive
  parameter decreases

Why did w1 increase but w2 decrease?

the output was too low
x1 was positive, so increasing w1 increases output
x2 was negative, so decreasing w2 increases output

This is the kind of explanation a tutor should make visible.

If a network has no nonlinear activation, stacked layers collapse into one linear layer.

y = W3(W2(W1x))
y = (W3 * W2 * W1)x

The product of linear transformations is another linear transformation.

So this:

input -> dense -> dense -> dense -> output

without nonlinear activations behaves like:

input -> one dense linear transform -> output

With nonlinear activations:

y = W3 * relu(W2 * relu(W1x + b1) + b2) + b3

Now different inputs activate different paths. The model becomes piecewise linear.

Intuition:

layer 1 gates the input space
layer 2 combines those gated regions
layer 3 combines combinations

Ablation:

Model A: 3 dense layers with ReLU
Model B: same 3 dense layers, activation removed

Expected behavior:

This is one of the most important experiments in a neural network tutor.

Width means more neurons in a layer.

narrow layer:
  8 neurons

medium layer:
  64 neurons

wide layer:
  512 neurons

More width gives the model more parallel detectors.

one neuron:
  one boundary

many neurons:
  many boundaries
  many gated features
  many ways to separate the data

Mathematical effect:

layer output lives in H-dimensional feature space

If H is small, the layer compresses the input into few signals. If H is large, the layer can preserve and expand many directions.

Tradeoff:

Parameter cost grows linearly with width for one layer:

D -> H
parameters = D*H + H

But if two neighboring layers are both wide, cost can grow sharply:

H1 -> H2
parameters = H1*H2 + H2

Example:

512 -> 512 dense layer
= 512*512 + 512
= 262,656 parameters

4096 -> 4096 dense layer
= 4096*4096 + 4096
= 16,781,312 parameters

Width is not free. The tutor should always show parameter count next to architecture diagrams.

Shape:

input -> small hidden layer -> output

Example:

20 -> 8 -> 1

Good for:

simple tabular problems
small datasets
linear-ish relationships
fast educational demos

Bad for:

complex nonlinear boundaries
high-dimensional input
many independent patterns
image or language tasks

Failure symptom:

training loss high
validation loss high
model underfits

Ablation:

increase width from 8 to 64
keep depth fixed
compare training loss

If both training and validation improve, the narrow model lacked capacity.

Shape:

input -> narrow layer -> narrow layer -> narrow layer -> output

Example:

20 -> 16 -> 16 -> 16 -> 1

Good for:

compositional structure
parameter-efficient experiments
forcing intermediate abstractions

Bad for:

tasks needing many independent features at once
optimization without normalization/residual help
datasets where early compression destroys information

Deep narrow networks can be elegant, but a too-narrow early layer becomes a bottleneck.

Example:

input has 100 independent useful features
first hidden layer has 4 neurons

The model must compress 100 useful signals into 4 numbers. If the target depends on many independent details, that compression may be impossible.

Ablation:

20 -> 4 -> 4 -> 4 -> output
20 -> 16 -> 16 -> 16 -> output
20 -> 64 -> 64 -> 64 -> output

Inspect:

training loss
validation loss
activation saturation
gradient norms
examples whose predictions change

If widening the early layer fixes both training and validation, the narrow layer was discarding useful signal.

A bottleneck is a narrow layer between wider layers.

128 -> 16 -> 128

It forces compression.

Mathematical intuition for a linear bottleneck:

D -> K -> D
rank <= K

If K is small, the network cannot preserve all independent directions through that bottleneck.

Good bottleneck case:

input has 100 features
only 10 are useful
many features are noise

Architecture:

100 -> 16 -> 64 -> output

The bottleneck can force the model to ignore noise and keep useful structure.

Bad bottleneck case:

input contains many independent useful facts
small differences matter

Architecture:

512 -> 8 -> 512

The bottleneck may destroy important distinctions.

Ablation:

hidden width 128
hidden width 32
hidden width 8

Watch:

which classes collapse together
which examples become confused
whether training loss itself gets stuck

If training loss is bad, the bottleneck is too restrictive. If training loss is good but validation improves, the bottleneck may be regularizing.

Not every architecture should be fully dense.

Sparse/local connection:

each neuron sees only part of previous layer

Convolution:

small filter reused across image positions

Residual connection:

output = layer(input) + input

Why residual helps:

gradient has a shortcut path backward
signal has a shortcut path forward
deep network can learn modifications instead of everything from scratch

A tutor does not need to implement every architecture first. But it should explain that “dense” is one connectivity choice, not the definition of neural networks.

Task:

x1 XOR x2

Truth table:

One linear neuron fails because XOR is not linearly separable.

no single straight line separates the 1s from the 0s

A hidden layer solves it:

2 inputs -> 2 hidden nonlinear neurons -> output

Intuition:

hidden neuron A detects one region
hidden neuron B detects another region
output combines the regions

Ablation:

Lesson:

width plus nonlinearity creates multiple regions.

Task:

recognize cats in images

A raw dense shallow model sees:

pixel_1, pixel_2, pixel_3, ..., pixel_150528

It must learn:

all pixels -> cat/not cat

That is inefficient because image structure is hierarchical and local.

Deep visual model intuition:

early layers:
  edges, colors, simple textures

middle layers:
  corners, fur textures, eye-like patterns

later layers:
  ears, face shapes, object parts

output:
  class decision

Ablation:

Lesson:

Depth helps when useful features are built from smaller reusable features.

Task:

classify examples where 15% of labels are wrong

A high-capacity model may eventually memorize mislabeled examples.

Training pattern:

early training:
  model learns broad real pattern

late training:
  model starts fitting noisy exceptions

Symptoms:

training accuracy keeps rising
validation accuracy peaks then falls
high-confidence wrong predictions increase

Ablation:

train narrow model
train wide model
train wide model with early stopping
train wide model with weight decay

Lesson:

more capacity can fit truth first and noise later.

The tutor should show example-level prediction changes so the learner can see which examples were memorized.

Task:

classify objects where small details matter

Architecture:

512 -> 8 -> 512 -> output

If the important signal requires many independent directions, 8 hidden values cannot carry enough information.

Symptoms:

training loss remains high
classes with subtle differences collapse
hidden activations look similar across examples

Ablation:

bottleneck width 8
bottleneck width 32
bottleneck width 128

If training improves as bottleneck width increases, the small bottleneck was limiting capacity.

Training usually averages gradients across a mini-batch.

batch_gradient =
  average(example_1_gradient,
          example_2_gradient,
          ...
          example_n_gradient)

This matters for architecture inspection because different examples can push the same neuron in opposite directions.

Example:

example A gradient for w1: -1.20
example B gradient for w1:  0.40
example C gradient for w1: -0.10

batch average: -0.30

The final update is smaller than example A alone.

Tutor display:

selected weight: w1
example contributions:
  A: increase strongly
  B: decrease mildly
  C: increase mildly
batch update:
  increase slightly

This helps learners understand why one example may look “ignored” in a batch update.

Ablation means remove or change one thing and measure the effect.

Useful ablations:

Each ablation should report:

parameter count
training loss
validation loss
gradient norm per layer
activation sparsity
examples changed most
confidence distribution
runtime cost

The important part is controlled comparison.

Bad experiment:

changed depth, width, optimizer, learning rate, and dataset split

Good experiment:

same data
same seed
same optimizer
same training steps
only width changed

Without controlled ablation, architecture claims are guesses.

A replay record should store:

architecture id
dataset split id
training step
batch ids
layer shapes
parameter counts
forward activations
pre-activation z values
loss
gradients
optimizer state where relevant
before parameters
after parameters
validation metrics
ablation id if active

For large networks, store summaries by default and detailed traces only for selected examples/layers:

always store:
  loss
  accuracy
  gradient norms
  activation sparsity
  parameter counts

store on demand:
  full activation vectors
  per-weight gradients
  per-example contribution

This keeps the tutor responsive while still allowing deep inspection.

UI rule:

training and replay generation run off the UI thread
large diagrams render lazily
only selected detail panes compute heavy traces
generated replay artifacts are visible and deletable

Educational tools still need production discipline.

Used:

step-by-step replay
single-neuron inspection
forward-pass arithmetic
chain-rule traces
weight update tables
activation behavior views
width/depth architecture comparisons
parameter-count visibility
gradient norm inspection
activation sparsity inspection
architecture ablation experiments
train-vs-validation comparisons
example-level prediction deltas

Removed or avoided:

teaching neural networks only as equations
showing only final accuracy
hiding gradients behind charts
generic network diagrams with no numbers
claiming depth is always better
claiming width is always better
ignoring parameter count
changing multiple experiment variables at once

Deferred:

large-model training
GPU-heavy training runtime
automatic architecture search
full tracing for every weight in large networks
over-general visual framework for every model family

The core lesson is that architecture is behavior. Width gives parallel detectors. Depth gives composition. Dense connectivity gives flexible interaction at parameter cost. Bottlenecks force compression. Activations make depth meaningful. Optimizers decide how parameters move. Ablation proves which of those choices actually mattered.