📈Activation Function
Also known as Activation, ReLU, Sigmoid Function
A rule inside each neuron. It decides what number gets passed on.
Think of it like
Think of it like a gate. It only opens when the push is hard enough.
Example
A neuron in Jack's model adds up to -3.2. ReLU turns that into 0, so the neuron stays quiet. Another neuron adds up to 4.1, and ReLU passes 4.1 on.
Why it matters
It is what lets a neural network learn curves and twists in data. Pick the wrong one, and training can stall.
Definition
Common mistake
Key takeaways
- 1After the sum. Used on each neuron's total.
- 2Adds bends. Lets the network learn curves.
- 3ReLU is common. Below zero becomes zero.