Clustering and PCA
Level 4 · Classic models
Three jobs with no answer key. Group like rows with k-means, squeeze many columns into a few with PCA, and spot the odd case out.
Course contentsLesson 11 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Explain how k-means finds groups
- Say what PCA keeps and what it loses
- Describe how anomaly detection spots odd cases
13 mincovers 5 glossary entries
Worth reading first: Supervised and Unsupervised ML
Finding groups, shrinking columns and spotting the odd one out
Clustering puts similar examples into groups without being told the groups first. k-means is the classic method. You choose k, the number of groups. It drops k centre points, puts each row with its nearest centre, moves each centre to the middle of its group and repeats until nothing moves. It dates from 1957.
The catch is that you have to pick k, and the method will always find k groups, real or not. Try a few values and look at what each group holds. A cluster only means something once a person names it, like 'weekend bargain hunters' or 'calm piano songs'.
Dimensionality reduction squeezes many columns down to a few. It keeps as much of the meaning as it can. A survey with 50 questions per person might shrink to 2 numbers each. Then the people can be drawn as dots on a flat chart, and groups that hid in 50 columns show up by eye.
Principal component analysis, or PCA, is the oldest way to do it, from 1901. It finds the direction in which the data spreads out most, and makes that the first new column. The second new column is the next biggest spread, at a right angle to the first. Keeping just a few keeps most of the story.
The cost is meaning. A new PCA column is a mix of all the old ones, so 'column 1' no longer means age or income. Some detail is always lost. PCA also only finds straight-line directions, so it can flatten patterns that curve.
Anomaly detection finds the rare cases that do not fit the usual pattern. Fraud, a failing machine and a break-in are all rare. So there are few bad examples to learn from. The model learns what normal looks like from lots of daily data instead. Then it flags anything far from it.
These tasks share one trap: there is no answer key. A supervised model can be scored against labels. A cluster, a squeezed chart or a flagged payment has to be checked by a person who knows the field. Treat what they find as a lead to look into, not a fact.
Worked example
Emma groups her customers
Emma runs k-means with k set to 4 on a year of her customers' spending. It finds four groups, from rare small spenders to weekly big spenders.
The same data flags 12 orders that sit far from any group. Ten turn out to be one firm buying in bulk, and two are stolen cards.
The words for this lesson5
Check yourself
What must you choose before you run k-means?
Finished this lesson?
Marking it complete also marks its 5 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.