# Activation Function

> A rule inside each neuron. It decides what number gets passed on.

**Field:** Artificial Intelligence  
**Category:** Machine Learning  
**Also known as:** Activation, ReLU, Sigmoid Function

## Think of it like

Think of it like a gate. It only opens when the push is hard enough.

## Definition

An activation function decides what a neuron sends on after it adds up its inputs. A common one, called ReLU, keeps any total above zero and turns the rest into zero. Another, called sigmoid, squeezes the total into a number from 0 to 1. These bends let a network learn curves, not just straight lines. Without them, a deep network would act like one straight line.

## Example

A neuron in Jack's model adds up to -3.2. ReLU turns that into 0, so the neuron stays quiet. Another neuron adds up to 4.1, and ReLU passes 4.1 on.

## Why it matters

It is what lets a neural network learn curves and twists in data. Pick the wrong one, and training can stall.

## Common mistake

Leaving it out to keep things simple. Layers stacked with no activation function add up to one plain straight line.

## Three principles

- **After the sum.** Used on each neuron's total.
- **Adds bends.** Lets the network learn curves.
- **ReLU is common.** Below zero becomes zero.

## Related terms

- [Artificial Neuron](https://learn.benibix.com/term/ai/artificial-neuron)
- [Neural Network](https://learn.benibix.com/term/ai/neural-network)
- [Logistic Regression](https://learn.benibix.com/term/ai/logistic-regression)
- [Deep Learning](https://learn.benibix.com/term/ai/deep-learning)
- [Softmax Function](https://learn.benibix.com/term/ai/softmax)

## Taught in

- [Neural Network Basics](https://learn.benibix.com/learn/ai-machine-learning/neural-network-basics): Machine Learning, 13/18

![Two small graphs. ReLU is flat at zero for negative numbers and rises in a straight line after zero. Sigmoid is an S-shaped curve that runs from 0 to 1.](https://learn.benibix.com/diagrams/nn-activation.svg)

---

Source: https://learn.benibix.com/term/ai/activation-function
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
