Feature Engineering
Level 2 · Getting the data ready
How raw data becomes inputs a model can use. Make new features, one-hot encode the words, and put the numbers on one scale.
Course contentsLesson 4 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Make a useful feature from raw data
- One-hot encode a column of words
- Explain why number columns need scaling
12 mincovers 3 glossary entries
Worth reading first: Machine Learning Basics
How raw data becomes something a model can use
Feature engineering is making better inputs from raw data, so a model learns more easily. A date can become 'day of the week'. Price and size can become 'price per square metre'. On table data, a good new feature often helps more than a fancier model. It puts the clue the model needs into one column.
Most models only read numbers, so words have to be turned into numbers first. Calling London 1, Paris 2 and Tokyo 3 looks fine. But it tells the model that Tokyo is three times London. That order is false, and the model will use it anyway.
One-hot encoding fixes this. Each city gets its own column. A London house has a 1 in the London column and a 0 in the others. Now no city is bigger than another. The cost is width: a column with 500 different values becomes 500 columns.
Feature scaling puts number columns on a similar range. Age runs from 0 to 100, while income can reach 200,000. Some models judge rows by how far apart they are, and then the big column drowns out the small one. Scaling both to run from 0 to 1 gives each a fair say.
There are two common ways. Min-max scaling squeezes each column to between 0 and 1. Standardization shifts each column so its average is 0 and most values sit close to it. Tree models need neither. Models that measure distance, or that learn in small steps, usually do.
Scaling has a trap of its own. Work out the ranges from the training set only, then apply them to the test set. If the test rows help set the ranges, a little of the test data has leaked into training, and the score will flatter the model.
Deep learning finds many features by itself, which is why photos and sound need far less of this work. On tables of numbers and words, though, this is still where most of the gains come from. People often reach for a bigger model first, when a better column would do more.
Worked example
Lucy adds a weekend column
Lucy's shop data has the time of each order, down to the second. Her sales model guesses busy days poorly.
She adds one column: weekend or not. That single feature cuts her model's daily miss from 40 orders to 15.
The words for this lesson3
Check yourself
Why not code London, Paris and Tokyo as 1, 2 and 3?
Finished this lesson?
Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.