Skip to content
BENIBIX Learn

Neural Network Basics

Level 5 · Neural networks

How a neural network is built from neurons in layers, joined by weights. What activation functions and softmax do, and what an MLP is.

Course contentsLesson 13 of 18

After this lesson you can

  • Follow a number through one neuron
  • Name the layers of a network
  • Explain what an activation function adds

13 mincovers 7 glossary entries

Worth reading first: Classification vs Regression

How a neural network is built

A neural network is a model built from many small units, called neurons, joined in layers. Each neuron takes in numbers, multiplies each by a weight, adds them up and passes the result on. Data goes in at one end and an answer comes out the other. The 2012 ImageNet winner had 60 million weights.

The first layer is the input layer, which takes in the data. The last is the output layer, which gives the answer. The ones between are hidden layers. Early hidden layers find simple things, like edges in a photo. Later ones join those into eyes, wheels and faces.

A weight is a number on the link between two neurons. A big weight makes an input count a lot. A weight near zero makes it count very little, and a negative one pushes the other way. Training is mostly the work of finding good weights. In most networks, the weights are the parameters.

After adding up, each neuron passes its total through an activation function. A common one, ReLU, keeps any total above zero and turns the rest into zero. This small bend is what lets a network learn curves. Without it, any stack of layers would act like one straight line.

At the end, a network that picks a group gives one raw score per group. The softmax function turns those scores into chances that add up to 100%. Scores of 4 for cat, 2 for dog and 1 for fox become about 84%, 11% and 4%. A big lead in score becomes a big lead in chance.

The plainest kind is the multilayer perceptron, or MLP. Every neuron in one layer links to every neuron in the next, and data flows one way, with no loops. It works well on rows of numbers. Bigger designs still have MLP parts inside them.

The name suggests a copy of the brain. It is only loosely inspired by brain cells, and it works very differently. A neuron here is a sum and a bend, nothing more. The power comes from millions of them and from training, not from anything like thought.

Weigh each input, add them up, then bend the total. Training changes the weights.

Worked example

Lucy's house price network

Lucy feeds 12 numbers about each house into an MLP with two hidden layers of 64 neurons. It gives back one number, the guessed price.

Between her 12 inputs and the first hidden layer alone there are 768 weights. Training sets every one of them, and she sets none.

The words for this lesson7

Check yourself

What does an activation function do?

Finished this lesson?

Marking it complete also marks its 7 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.