Ramps make shapes
One neuron draws a line that is flat on one side and sloped on the other. Bob now needs both heating and cooling, with a quiet zone between them. One corner cannot do that.
AFTER THIS LESSON By the end, you can explain why one ramp cannot cover both heating and cooling, and use a few ramps to build a shape with several corners.
Multiple ramps
The first neuron made a heater ramp: strong when the room is cold, flat once it is warm. Bob needs something different on a hot day. A second neuron can make a cooler ramp that wakes up above 24°C. Between 16°C and 24°C, both can rest. One straight ramp with one corner cannot be high on the cold side, level in the middle, and active again on the hot side. Two separate ramps give us two corners to work with.
To keep the graph in one dimension, we call heating positive and cooling negative. A negative answer here means “cool,” not a physical heater running at negative power. Each ramp still has its own zero gate; only afterward do we add their contributions. That order matters because each specialist must decide independently when to switch on.
One temperature, two specialists
heat = max(0, 16 − 30) = 0restcool = −max(0, 30 − 24) = −6cool0 + (−6) = −6cooling answerSeparate neurons
At 30°C, the heater’s arithmetic before its gate is 16 − 30 = −14, while the cooler’s is 30 − 24 = 6. If we poured those unfinished numbers into one box and gated only the sum, −14 + 6 would become zero. We would miss a hot room that needs cooling. Gate them separately and we get heater 0 and cooler 6; the final answer marks that cooler contribution as −6. The two gates preserve two different decisions.
Slope changes
A ramp contributes nothing before its corner and a straight slope after it. Add one ramp and the total line starts climbing. Add a second with a negative contribution and the climb can flatten or turn downward. Add another and it can flatten again. That is what the “One step” and “A bump” controls show. The individual dashed lines are the pieces; the green line is their sum.
Even the wave example uses only straight segments. Five ramps can follow a smooth curve roughly, and more corners could follow it more closely. Nothing in this lab is learning yet: we placed every corner and chose every contribution. Training will eventually move those numbers from examples instead of from our hands.
Try the ramps
Pick a shape, then move across it. Watch individual contributions add into one answer.
White dashed line: the smooth wave. Green line: five straight pieces.
Predict before you reveal: at 20°C, what do the heat and cool ramps contribute?
Both contribute zero. The room is above the heater’s 16°C corner and below the cooler’s 24°C corner, so the combined answer is zero: neither action is needed.
WHAT CHANGED? Adding a new ramp changes the total line’s slope at that ramp’s corner.
Key concepts
- Ramp
- A flat-then-sloped or sloped-then-flat response made by a neuron’s zero gate. Its corner is where that gate changes state.
- Hidden layer
- Neurons that make intermediate answers before the network’s final output. Here separate heater and cooler ramps are intermediate answers.
- Output weight
- An adjustable multiplier on an intermediate answer. A negative output weight turns the cooler’s positive activity into a negative, cooling answer.
- Piecewise-linear shape
- A line made from straight sections with corners between them. Adding ramps changes the slope at those corners.
The compact version, once you have seen it
answer = ramp₁ × output weight₁ + ramp₂ × output weight₂ + … + output bias
The first ramps are the hidden layer. The last neuron combines them. A negative output weight turns a contribution downward.