Skip to content
BENIBIX Learn

Classification vs Regression

Level 4 · Classic models

The two main jobs of supervised learning. Classification picks a group and regression guesses a number. Then the simplest model for each.

Course contentsLesson 8 of 18

After this lesson you can

  • Tell classification from regression
  • Explain how linear regression fits a line
  • Say what logistic regression gives back

12 mincovers 4 glossary entries

Worth reading first: Supervised and Unsupervised ML

Classification or regression: picking a group or guessing a number

Classification puts each example into one of a few fixed groups, like spam or not spam. Regression guesses a number on a scale, like a price or a time. Both are supervised learning. The test is the answer: if it is a name from a list, it is classification. If it is an amount, it is regression.

The word regression is old. Francis Galton used it in 1886, to describe how very tall parents tend to have children closer to average height. It has nothing to do with going backwards. Today it just means guessing a number.

Linear regression is the simplest way to guess a number. It draws the straight line that best fits the data. Each feature gets a weight that pushes the answer up or down. The way to find that best line, called least squares, was first published in 1805, and it is still in daily use.

Its strength is that you can read it. A line that says each extra square metre adds 15 euros of rent is easy to explain and to check. Its weakness is the straight line. When the real link curves, a line misses, and the model underfits.

Logistic regression is for classification, despite its name. It adds up the features with weights, just as linear regression does. Then it bends the total into an S-shaped curve that stays between 0 and 1, so the result reads as a chance. A cut-off, often 0.5, turns that chance into a yes or a no.

The chance is often more useful than the yes or no. A bank might approve a loan above 0.8, send it to a person between 0.5 and 0.8, and refuse it below 0.5. Moving the cut-off trades one kind of mistake for the other. That is a choice about cost, not about the model.

People often pick the task by the data rather than by the question. 'Will this customer leave?' is classification, even if the data is all numbers. 'How many days until they leave?' is regression. Get the question right first. The choice of model follows.

Which one to pick

  1. 1

    Pick classification

    When the answer is a name from a list, like spam, cat or approved.

  2. 2

    Pick regression

    When the answer is an amount, like a price, a time or a count.

A straight line gives a number. An S-curve gives a chance, and the cut-off turns it into yes or no.

Worked example

Jack's two rent models

Jack fits a line to 300 flats in his town. It says the rent is 400 euros, plus 15 euros for each square metre.

For a 60 square metre flat, that gives 1,300 euros. His second model, a logistic one, gives the same flat a 0.9 chance of being let within a week.

The words for this lesson4

Check yourself

What job does logistic regression do?

Finished this lesson?

Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.