Data Labeling and Augmentation
Level 2 · Getting the data ready
Where labels come from, and what wrong ones cost. Why a rare class fools a model, and how changing the examples you have makes more of them.
Course contentsLesson 5 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Plan how examples get their labels
- Spot class imbalance in a set of data
- Make more examples with data augmentation
12 mincovers 3 glossary entries
Worth reading first: Feature Engineering
Getting enough of the right examples
Data labeling is the work of adding the right answer to each example. A person looks at a photo, a sentence or a sound clip and tags it: cat, spam, angry. It is slow and costs money, so many people share the work. The 14 million photos in ImageNet were tagged this way, by people paid online.
Wrong labels teach wrong lessons, and the model learns them just as well as right ones. Two people often tag the same item in different ways. Good teams write clear rules for each tag, have two people label the hard cases, and settle any clash before training.
Class imbalance is when one answer is far rarer than the rest. In a well-known set of 284,807 card payments, only 492 were fraud, under 2 in 1,000. A model that always says 'not fraud' is right 99.8% of the time. It is also useless, because it never finds a single fraud.
So an imbalanced set needs care. You can show the rare rows more often, show the common rows less often, or make a miss on the rare class cost more in training. You also judge the model by how many rare cases it catches, not by how often it is right overall.
Data augmentation makes more training examples by changing the ones you have. A photo can be flipped, turned, cropped or made darker. The label stays the same, because a flipped cat is still a cat. The model sees more variety, and copes better with messy real photos.
Augmentation has limits. Each change must keep the label true. Flipping a photo of a cat is fine, but turning a photo of the number 6 upside down can make a 9. Text is harder still, since swapping one word can change what a sentence means.
People often blame the model when the trouble is in the labels or the balance. Before trying a new model, count each class and read a few hundred labels by hand. An hour spent there often fixes more than a week of tuning.
Worked example
Jack labels X-rays
Jack's clinic has 600 X-rays marked broken or not broken. Only 60 show a break, and two doctors check each of those.
He flips and turns each break photo 4 ways, which gives 300 break photos to learn from. His model now finds 41 of 50 new breaks, up from 22.
The words for this lesson3
Check yourself
Why can a fraud model be right 99.8% of the time and still be useless?
Finished this lesson?
Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.