Skip to content
BENIBIX Learn

Data Labeling and Augmentation

Level 2 · Getting the data ready

Where labels come from, and what wrong ones cost. Why a rare class fools a model, and how changing the examples you have makes more of them.

Course contentsLesson 5 of 18

After this lesson you can

  • Plan how examples get their labels
  • Spot class imbalance in a set of data
  • Make more examples with data augmentation

12 mincovers 3 glossary entries

Worth reading first: Feature Engineering

Getting enough of the right examples

Data labeling is the work of adding the right answer to each example. A person looks at a photo, a sentence or a sound clip and tags it: cat, spam, angry. It is slow and costs money, so many people share the work. The 14 million photos in ImageNet were tagged this way, by people paid online.

Wrong labels teach wrong lessons, and the model learns them just as well as right ones. Two people often tag the same item in different ways. Good teams write clear rules for each tag, have two people label the hard cases, and settle any clash before training.

Class imbalance is when one answer is far rarer than the rest. In a well-known set of 284,807 card payments, only 492 were fraud, under 2 in 1,000. A model that always says 'not fraud' is right 99.8% of the time. It is also useless, because it never finds a single fraud.

So an imbalanced set needs care. You can show the rare rows more often, show the common rows less often, or make a miss on the rare class cost more in training. You also judge the model by how many rare cases it catches, not by how often it is right overall.

Data augmentation makes more training examples by changing the ones you have. A photo can be flipped, turned, cropped or made darker. The label stays the same, because a flipped cat is still a cat. The model sees more variety, and copes better with messy real photos.

Augmentation has limits. Each change must keep the label true. Flipping a photo of a cat is fine, but turning a photo of the number 6 upside down can make a 9. Text is harder still, since swapping one word can change what a sentence means.

People often blame the model when the trouble is in the labels or the balance. Before trying a new model, count each class and read a few hundred labels by hand. An hour spent there often fixes more than a week of tuning.

A rare class hides behind a high score. Changed copies of a photo give the model more to learn from.

Worked example

Jack labels X-rays

Jack's clinic has 600 X-rays marked broken or not broken. Only 60 show a break, and two doctors check each of those.

He flips and turns each break photo 4 ways, which gives 300 break photos to learn from. His model now finds 41 of 50 new breaks, up from 22.

The words for this lesson3

Check yourself

Why can a fraud model be right 99.8% of the time and still be useless?

Finished this lesson?

Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.