Machine Learning Foundations / Core / 02.08
Unsupervised learning
No answer key, no teacher. Unsupervised learning looks at raw data and finds the groups, links and odd ones out that are hiding in it.
14 min readFree
Learning with no answer key
In the last lesson, every example came with an answer. A photo was tagged 'cat'. A house came with its price. The computer studied those pairs and learned to guess the answer for new ones. That is supervised learning.
Now imagine a different job. Someone hands you a box of 500 loose buttons. Nobody tells you what to do. You tip them onto the table, and you start making piles: big and small, shiny and dull, two holes and four. Nobody gave you the piles. You found them by looking.
That is unsupervised learning. Wikipedia describes it as a framework where 'algorithms learn patterns exclusively from unlabeled data'. IBM says these algorithms 'discover hidden patterns or data groupings without the need for human intervention'. There is no teacher and no answer column. The data is all you have.
Why would anyone want this? Wikipedia notes that such data is typically 'harvested cheaply' from the wild, with no one tagging it. If a computer can find structure in it alone, you learn something without labelling a thing.
Key words
- Label
- The answer attached to an example, such as 'spam' or '$250,000'. Unsupervised data has none.
- Cluster
- A group of examples that look alike, like one pile of buttons.
- Centroid
- The centre of a cluster: the average position of everything in it.
- Anomaly
- An example that does not fit in. The odd one out.
- Ground truth
- The known right answer. Unsupervised learning does not have one to check against.
The four main jobs
IBM says unsupervised models are used for 'three main tasks': clustering, association and dimensionality reduction. Wikipedia's list of problems also names anomaly detection. So there are four jobs to know. Here they are side by side.
Clustering
Sort things into groups of look-alikes. Like piling the buttons.
Association rules
Find items that show up together. Like noticing that people who buy bread often buy milk.
Anomaly detection
Spot the one that does not belong. Like the single red button in a pile of blue ones.
The fourth job is dimensionality reduction. You met it two lessons ago, so we will not teach it again. In one line: it squeezes many columns into a few while keeping what matters. IBM describes it as reducing 'the number of data inputs to a manageable size while also preserving the integrity of the dataset as much as possible'. No labels are needed for it either, which is why it counts as unsupervised. The next sections cover the other three.
Clustering: making the piles
Google's Machine Learning Crash Course puts it plainly: 'Clustering is an unsupervised machine learning technique designed to group unlabeled examples based on their similarity to each other.' Its example is a health study. Patients report how often they get symptoms and how bad they are. Clustering can group patients with similar responses, with nobody saying in advance which group is which.
After clustering, each group gets a number, called a cluster ID. Google points out a handy side effect: you can replace many columns with one cluster ID, which cuts the storage and work needed. Instead of storing a long profile for every customer, you store 'customer is in group 3'.
There are many ways to cluster. The simplest one to learn is k-means. In the next section we will build it by hand.
The k-means idea
Picture shop owners choosing where to open two coffee carts in a park. They want every visitor to have a cart close by. A good way is to start with two guesses, send each visitor to the nearer cart, then move each cart to the middle of its own visitors. Repeat until the carts stop moving. That is k-means.
The letter k is just the number of groups you ask for. The scikit-learn guide describes the loop in three steps: pick starting centres, 'assigns each sample to its nearest centroid', then 'creates new centroids by taking the mean value of all of the samples assigned to each previous centroid'. It repeats 'until the centroids do not move significantly'.
- 1
Choose k and start centres
Say you want two groups. Pick two starting points. Even a poor guess is fine.
- 2
Assign
Measure the distance from every point to each centre. Each point joins the group of the nearest centre.
- 3
Update
Move each centre to the average position of the points in its group.
- 4
Repeat
Go back to step 2. Stop when the centres no longer move.
K-means by hand
Here are eight made-up points on a flat map, each as an (x, y) pair. We ask for k = 2. We start with a deliberately poor guess: the first two points are the two starting centres. After each round the code prints the size and centre of each group. Only Python's built-in maths is used.
import math
points = [(1, 1), (2, 1), (1, 3), (3, 2), (6, 5), (8, 7), (9, 8), (8, 9)]
centres = [points[0], points[1]] # a poor start on purpose
for round_no in range(1, 4):
groups = [[], []]
for p in points:
d = [math.dist(p, c) for c in centres]
groups[d.index(min(d))].append(p)
centres = [
(sum(x for x, _ in g) / len(g), sum(y for _, y in g) / len(g))
for g in groups
]
print("round", round_no)
for i, g in enumerate(groups):
c = centres[i]
print(" group", i, "size", len(g), "centre", (round(c[0], 2), round(c[1], 2)))Output
round 1 group 0 size 2 centre (1.0, 2.0) group 1 size 6 centre (6.0, 5.33) round 2 group 0 size 4 centre (1.75, 1.75) group 1 size 4 centre (7.75, 7.25) round 3 group 0 size 4 centre (1.75, 1.75) group 1 size 4 centre (7.75, 7.25)
Read the output like a story. In round 1 the start was bad. The two centres sat almost on top of each other, so one group grabbed six points and the other only two. The centres moved. In round 2 the split was clean: four points in the lower-left, four in the upper-right. In round 3 nothing changed. The centres stayed put, so we are done.
Google's course lists a nice property of k-means: it 'always converges', meaning the loop does settle. It also names weak spots. The result is sensitive to where the centres start, and outliers can drag a centre off. Our poor start still worked on this tiny map, but on messy data a bad start can matter more.
Association rules: bought together
Wikipedia calls association rule learning 'a rule-based machine learning method for discovering interesting relations between variables in large databases'. The classic setting is a shop. Each shopping basket is called a transaction. You look for items that keep turning up in the same basket. IBM says shops use these rules to build cross-selling strategies and recommendation engines.
A rule has two sides: 'if this, then that'. For example, bread leads to milk. Two numbers tell you how much to trust a rule.
Key words
- Support
- How common the items are. The share of all baskets that contain them.
- Confidence
- How often the rule holds. Of the baskets that have the 'if' item, the share that also have the 'then' item.
- Itemset
- A set of items that appear in a basket together.
Wikipedia gives the support of a rule as the number of transactions containing both items divided by the total number of transactions. Confidence is the number of transactions containing both divided by the number containing the first item. In code it is just counting.
baskets = [
{"bread", "milk"},
{"bread", "butter", "milk"},
{"bread", "butter"},
{"milk", "eggs"},
{"bread", "milk", "eggs"},
]
total = len(baskets)
with_bread = sum(1 for b in baskets if "bread" in b)
with_both = sum(1 for b in baskets if {"bread", "milk"} <= b)
print("baskets:", total)
print("support(bread and milk):", with_both / total)
print("confidence(bread -> milk):", with_both / with_bread)Output
baskets: 5 support(bread and milk): 0.6 confidence(bread -> milk): 0.75
Three of the five baskets hold both bread and milk, so support is 3 out of 5, which is 0.6. Four baskets hold bread, and three of those also hold milk, so confidence is 3 out of 4, which is 0.75. In words: 'Three in four bread buyers also bought milk.' Nobody told the computer to look for bread and milk. It counted.
IBM names Apriori as the most widely used algorithm. Wikipedia explains that it starts with frequent single items and extends them to larger sets 'as long as those item sets appear sufficiently often'. The counting idea is the same as ours.
Anomaly detection: the odd one out
Wikipedia defines anomaly detection as 'the identification of observations, events, or patterns that depart substantially from normal behavior expected in a particular setting'. Think of a night guard who knows the building well. A door open at 3 a.m. gets noticed. Nobody wrote a rule for that exact door. The guard just knows what normal looks like.
Wikipedia notes that unsupervised anomaly detection assumes the data is unlabelled and is 'by far the most commonly used'. That makes sense. The system learns what normal looks like, then flags what does not fit.
An odd result is not always a mistake. Wikipedia says an anomaly 'is not necessarily a data error'. It may be a measurement slip, an unusual but valid observation, or an event of interest 'such as fraud, equipment failure, or a security intrusion'. So a flag means 'look at this', not 'this is wrong'.
How do you know it worked?
With supervised learning you can score the model: how many answers did it get right? Here there is no answer key. Google's clustering course says so: clustering 'doesn't produce or include a ground "truth" against which you can verify the output'. So it advises you to check the result against your expectations at the cluster level and the example level. In plain words, a person looks. Do the groups make sense? Do the points inside one group really look alike?
IBM lists this as a challenge too: people must step in to validate the output, and there can be a lack of transparency into the basis on which data was clustered. The computer says 'these belong together', but it may not say why.
A number can help. The scikit-learn guide says that when 'the ground truth labels are not known, evaluation must be performed using the model itself'. One such measure is the silhouette coefficient. For each point it compares a, the average distance to the others in its own group, with b, the average distance to the points in the next nearest group. The score is (b - a) divided by the larger of the two. It runs from -1 for incorrect clustering to +1 for highly dense clustering, and scores near zero mean overlapping clusters.
Here it is for the two groups k-means found above. The code works out the score for each of the eight points and prints the average.
import math
group_a = [(1, 1), (2, 1), (1, 3), (3, 2)]
group_b = [(6, 5), (8, 7), (9, 8), (8, 9)]
def mean_dist(p, others):
others = [q for q in others if q != p]
return sum(math.dist(p, q) for q in others) / len(others)
def silhouette(p, own, other):
a = mean_dist(p, own)
b = mean_dist(p, other)
return (b - a) / max(a, b)
scores = [silhouette(p, group_a, group_b) for p in group_a]
scores += [silhouette(p, group_b, group_a) for p in group_b]
print("mean silhouette:", round(sum(scores) / len(scores), 2))Output
mean silhouette: 0.7
A score of 0.7 is close to +1, so the two groups are fairly tight and well apart. It matches what the map looks like. Still, a good score is a hint, not proof. The scikit-learn guide warns that the coefficient is 'generally higher for convex clusters than other concepts of clusters', so it can favour round blobs over other shapes.
Quick check
You cluster customers and get a silhouette score near 0. What does that suggest?
Where it is used
These ideas appear in many everyday products. Each use below comes from the sources.
Customer groups
Google's course lists market segmentation as a use of cluster analysis. IBM mentions customer personas, which help firms understand the common traits and buying habits of their customers.
Topic grouping
IBM says Google News uses unsupervised learning to categorize articles on the same story from different outlets. K-means is also named for document clustering and image compression.
Fraud and fault signals
Wikipedia says anomaly detection is 'vital in fintech for fraud prevention'. IBM adds that flagged odd points can raise awareness of faulty equipment, human error or security breaches.
Google's course also lists medical imaging and social network analysis, and notes that clustering can help fill in missing data by looking at other examples in the same cluster.
Pitfalls to watch for
Unsupervised results always look neat on a chart. That is the danger. Four traps catch beginners again and again.
Trap one: you choose the number of groups. The scikit-learn guide says plainly that k-means 'requires the number of clusters to be specified'. Google's course says choosing k stays a manual process that strongly shapes the result. IBM explains the effect: a larger k gives 'smaller groupings with more granularity', and a smaller k gives larger, coarser groups. Ask for 2 and you get 2, even if the data really has 5. Wikipedia lists the elbow method, and then warns that the 'elbow' is 'not well-defined and this is known to be unreliable'. The silhouette score from above is another guide.
Trap two: scale matters. K-means works with distances. If one column is salary in dollars and another is age in years, salary will swamp age, because its numbers are far bigger. Google's workflow says that before clustering you must 'normalize, scale, and transform feature data'. This is exactly what the feature scaling lesson was for. Do it first.
Trap three: the groups may not mean anything. The computer finds groups, whether or not they are useful. IBM notes a 'higher risk of inaccurate results'. Treat each group as a question to a human, not an answer. Open a few examples in every group and ask: would a real person call these the same kind of thing?
Trap four: the shape and the strays. The scikit-learn guide says k-means assumes clusters are 'convex and isotropic', and 'responds poorly to elongated clusters, or manifolds with irregular shapes'. Google adds that outliers can distort the result, and suggests removing or clipping them first. In very high dimensions distances get inflated, so the guide suggests running PCA before k-means. That links back to the dimensionality reduction lesson.
Tip
Before you trust a clustering, run it twice with different starting centres, and look at some real examples in each group. If the groups change a lot or no human can name them, be careful.
Quick check
Your data has a salary column in dollars and an age column in years. What should you do before k-means?
What to take away
Unsupervised learning finds structure in data with no labels. Its four main jobs are clustering (make piles), dimensionality reduction (fewer columns), association rules (items that go together, measured by support and confidence) and anomaly detection (the odd one out). K-means shows how clustering works: assign each point to the nearest centre, move the centres to the average, repeat until they settle.
There is no answer key, so people judge the results by looking, with help from scores like the silhouette. Choose k with care, scale your columns first, and never assume a group means something until a human agrees.
Next: semi-supervised learning.
Sources
- IBM: What is unsupervised learning?
- Google ML Crash Course, Clustering: What is clustering?
- Google ML Crash Course, Clustering: Clustering algorithms
- Google ML Crash Course, Clustering: Clustering workflow
- Google ML Crash Course, Clustering: Advantages and disadvantages of k-means
- scikit-learn user guide: Clustering (k-means, silhouette coefficient)
- Wikipedia: Unsupervised learning
- Wikipedia: Association rule learning
- Wikipedia: Anomaly detection
- Wikipedia: K-means clustering
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.