Machine Learning Foundations / Advanced / 03.06
F1-Score: one number for two goals
Precision and recall pull in opposite directions. F1 blends them so that one weak side cannot hide. Learn how it works, how to tilt it, and when to skip it.
13 min readFree
Two scores, one decision
The last lesson ended with recall: out of everything that really was positive, how much did the model find? Before that came precision: out of everything the model flagged, how much was really positive. Each one alone can be fooled. A model that flags everything has perfect recall and poor precision. A model that flags one sure case and nothing else has perfect precision and poor recall.
So you need both. But when you compare ten models in a table, two numbers per model gets messy. Which is better: precision 0.9 with recall 0.5, or precision 0.7 with recall 0.7? You want one number you can sort by. That number is the F1 score.
This lesson goes deeper than the one-time formula in the Model evaluation lesson. You will see why F1 uses a strange kind of average, how to tilt it toward the side you care about, how to use it when there are more than two classes, and what it cannot see.
Key words
- F1 score
- One number that blends precision and recall. It is their harmonic mean.
- Harmonic mean
- A kind of average that is pulled toward the smaller number.
- F-beta
- A version of F1 with a dial, beta, that says how much more recall matters than precision.
- Macro average
- Score each class on its own, then take the plain average of those scores.
- Micro average
- Add up the hits and misses of all classes first, then score once.
- Weighted average
- Like macro, but bigger classes count for more.
- True negative
- The model said no, and the truth was no. F1 never looks at these.
The harmonic mean, in plain words
Start with a trip. You drive to a town 60 km away at 60 km/h. You drive back the same road at 20 km/h. What was your average speed? Most people say 40, the plain average of 60 and 20. That is wrong. The trip took one hour out and three hours back: 120 km in 4 hours, which is 30 km/h. Wikipedia's page on the harmonic mean gives this very kind of example: when each leg covers the same distance, the average speed is the harmonic mean, 30 km/h, not the arithmetic mean, 40 km/h.
Why 30? Because you spent far more time at the slow speed. The slow leg dragged the whole trip down. That is the harmonic mean in one picture: a kind of average where the small number gets more say than the big one.
Wikipedia defines it as the reciprocal of the arithmetic mean of the reciprocals. In words: flip each number upside down (4 becomes one quarter), take the plain average of those, then flip the answer back. For two numbers there is a shortcut, and it is exactly the F1 formula.
Say it out loud: multiply precision and recall, double that, and divide by their sum. Google's Machine Learning Crash Course describes the result in three plain facts. When both scores are a perfect 1.0, F1 is 1.0. When the two are close, F1 is close to them. When they are far apart, F1 lands near the worse one.
Why not just the plain average?
pairs = [(0.9, 0.9), (0.9, 0.5), (0.9, 0.1), (1.0, 0.0)]
for precision, recall in pairs:
average = (precision + recall) / 2
f1 = 2 * precision * recall / (precision + recall)
print(precision, recall, "average", round(average, 2), "F1", round(f1, 2))Output
0.9 0.9 average 0.9 F1 0.9 0.9 0.5 average 0.7 F1 0.64 0.9 0.1 average 0.5 F1 0.18 1.0 0.0 average 0.5 F1 0.0
Read the rows from top to bottom. With 0.9 and 0.9 the two methods agree. Make recall 0.5 and the plain average slips to 0.7, but F1 drops to 0.64. Make recall 0.1 and the plain average still says 0.5, while F1 says 0.18. The last row is the extreme case. Precision is perfect, recall is zero, the plain average is 0.5, and F1 is 0.0. Wikipedia says the same: the lowest possible F-score is 0, when precision or recall is zero.
Tip
The harmonic mean never lets one great score buy back a terrible one. That is the whole reason F1 uses it.
Working out F1 by hand
You can get F1 from precision and recall, or straight from the counts of hits and misses. Both routes give the same answer. Here is the walk-through with counts, using a model that flagged 75 emails as spam.
- 1
Count the true positives
Spam the model flagged that really was spam. Call this TP. Say 45.
- 2
Count the false positives
Real emails the model flagged by mistake. Call this FP. Say 30.
- 3
Count the false negatives
Spam the model missed. Call this FN. Say 5.
- 4
Get precision and recall
Precision is TP divided by TP plus FP. Recall is TP divided by TP plus FN.
- 5
Blend them
Put both into the F1 formula. The code in the next section does steps 4 and 5.
F-beta: tilting the scale
F1 treats precision and recall as equally important. Often they are not. A cancer screening wants to miss nobody, so recall matters more. A spam filter that hides your job offer is a worse day than one stray junk email, so precision may matter more. F-beta adds a dial.
Wikipedia says beta is chosen so that recall counts as beta squared times as important as precision. Scikit-learn writes the same formula and says that with beta equal to 1, F-beta and F1 are the same, and recall and precision are equally important.
In words: the same shape as F1, but beta squared multiplies the precision in the bottom. A bigger beta pushes the score toward recall. A smaller beta pushes it toward precision. Wikipedia names the two common choices: beta of 2, which weighs recall higher than precision, and beta of one half, which weighs recall lower.
F2 (beta = 2)
Recall-heavy. Use it when missing a real case costs more than a false alarm, like screening for an illness or finding fraud for a human to check.
F1 (beta = 1)
Balanced. Use it when both mistakes hurt about the same, or you have no reason to prefer one.
F0.5 (beta = 0.5)
Precision-heavy. Use it when a false alarm costs more than a miss, like a filter that deletes mail on its own.
Now one model, three scores. It flagged 75 emails: 45 were spam, 30 were not, and it missed 5 spam emails. So precision is 0.6 and recall is 0.9, a model that finds nearly everything but cries wolf a lot.
def f_beta(precision, recall, beta):
b2 = beta ** 2
return (1 + b2) * precision * recall / (b2 * precision + recall)
tp, fp, fn = 45, 30, 5
precision = tp / (tp + fp)
recall = tp / (tp + fn)
print("precision", round(precision, 2), "recall", round(recall, 2))
for beta in (0.5, 1, 2):
print("F" + str(beta), round(f_beta(precision, recall, beta), 3))Output
precision 0.6 recall 0.9 F0.5 0.643 F1 0.72 F2 0.818
Look at how the score moves. F0.5 is 0.643, because the weak side is precision and F0.5 listens to precision. F1 sits in the middle at 0.72. F2 is 0.818, because the strong side is recall and F2 listens to recall. Same model, three honest answers to three different questions.
Tip
Pick beta before you look at the results, from the cost of each mistake. If you try all three and keep the nicest number, you are flattering your model, not judging it.
F1 with more than two classes
F1 is built for a yes-or-no problem: spam or not spam. What if there are three or more answers, like cat, dog and bird? Scikit-learn's answer: the data is treated as a collection of binary problems, one for each class. For the cat class, cat is the yes and everything else is the no. Then you do the same for dog, and for bird. That gives one F1 per class. The question is how to boil three scores into one.
The three common ways are macro, micro and weighted.
Macro
Take the plain average of the per-class F1 scores. Every class gets equal weight. Scikit-learn says this can highlight infrequent classes that still matter, but it may over-emphasize the usually low score on a rare class.
Micro
Add up the hits and misses of every class, then work out one F1 from the totals. Every example counts the same, so big classes dominate.
Weighted
Take the average of the per-class scores, but weight each class by how many real examples it has. Scikit-learn says this accounts for class imbalance.
Here is a made-up model for three classes with an uneven mix: 70 cats, 20 dogs and 10 birds. A seeded random draw makes the model right about 90% of the time on cats, 70% on dogs and 40% on birds. The seed is fixed, so you get the same run every time.
import random
random.seed(7)
labels = ["cat", "dog", "bird"]
truth = ["cat"] * 70 + ["dog"] * 20 + ["bird"] * 10
# A made-up model: right 90% of the time on cats, 70% on dogs, 40% on birds.
chance_right = {"cat": 0.9, "dog": 0.7, "bird": 0.4}
pred = []
for t in truth:
if random.random() < chance_right[t]:
pred.append(t)
else:
pred.append(random.choice([x for x in labels if x != t]))
def f1_from(tp, fp, fn):
return 2 * tp / (2 * tp + fp + fn) if tp else 0.0
counts = {}
for label in labels:
tp = sum(1 for t, p in zip(truth, pred) if t == label and p == label)
fp = sum(1 for t, p in zip(truth, pred) if t != label and p == label)
fn = sum(1 for t, p in zip(truth, pred) if t == label and p != label)
counts[label] = (tp, fp, fn)
print(label, "tp", tp, "fp", fp, "fn", fn, "F1", round(f1_from(tp, fp, fn), 3))
per_class = [f1_from(*counts[l]) for l in labels]
sizes = [truth.count(l) for l in labels]
macro = sum(per_class) / len(labels)
weighted = sum(f * n for f, n in zip(per_class, sizes)) / len(truth)
micro = f1_from(sum(c[0] for c in counts.values()),
sum(c[1] for c in counts.values()),
sum(c[2] for c in counts.values()))
print("macro F1", round(macro, 3))
print("weighted F1", round(weighted, 3))
print("micro F1", round(micro, 3))
print("accuracy", round(sum(t == p for t, p in zip(truth, pred)) / len(truth), 3))Output
cat tp 65 fp 4 fn 5 F1 0.935 dog tp 15 fp 3 fn 5 F1 0.789 bird tp 6 fp 7 fn 4 F1 0.522 macro F1 0.749 weighted F1 0.865 micro F1 0.86 accuracy 0.86
Cats are easy: F1 of 0.935. Dogs are fine at 0.789. Birds are poor at 0.522. Now watch the three averages. Macro F1 is 0.749. It listens to the bird class as much as the cat class, so the weak bird score pulls it down. Weighted F1 is 0.865, because cats are 70 of the 100 examples and they carry most of the weight. Micro F1 is 0.86.
That last number is not a coincidence. Scikit-learn notes that when every label is included in a multiclass problem, micro-averaging gives precision, recall and F that are all identical to accuracy. Check the last line of the output: accuracy is also 0.86. So in this kind of problem, micro F1 tells you nothing new beyond accuracy.
Tip
If a rare class matters to you, such as the birds here, read macro F1 or the per-class scores. A high weighted or micro number can hide a class the model barely understands.
What F1 cannot see
Google's course gives F1 in a second form, straight from the counts: two times TP, over two times TP plus FP plus FN. There is no TN in it. F1 never looks at true negatives, the cases where the model said no and was right. Wikipedia reports David Powers making this point: F1 ignores the true negatives and so can mislead on unbalanced classes.
Is that a bug? Sometimes it is a feature. If you search for a rare thing, like a fraud among a million payments, the huge pile of correct no answers would drown the score. Accuracy would look wonderful just because most things are not fraud. F1 only judges how well you handled the cases that matter. Other times it is a flaw, when the correct no answers matter too.
Here are two models with exactly the same hits and misses: 8 true positives, 4 false positives, 2 false negatives. They differ only in how many true negatives there are, 10 for one and 10,000 for the other.
# Two models with the same hits and misses. Only the number of true negatives differs.
tp, fp, fn = 8, 4, 2
for tn in (10, 10000):
accuracy = (tp + tn) / (tp + tn + fp + fn)
f1 = 2 * tp / (2 * tp + fp + fn)
print("true negatives", tn, "accuracy", round(accuracy, 3), "F1", round(f1, 3))Output
true negatives 10 accuracy 0.75 F1 0.727 true negatives 10000 accuracy 0.999 F1 0.727
Accuracy jumps from 0.75 to 0.999, just because of the pile of true negatives. F1 does not move: 0.727 both times. If the true negatives are what you care about, F1 is the wrong tool.
When to use F1, and when to skip it
Use it when
The positive class is rare or matters most, the classes are lopsided, and false alarms and misses cost about the same. It is also handy as one number to sort a table of models. Google's course calls it preferable to accuracy for class-imbalanced data.
Tilt it when
One mistake clearly costs more. Use F2 or F0.5 instead of F1. The Wikipedia page quotes criticism from David Hand that F1 gives equal importance to precision and recall when real mistakes have different costs.
Skip it when
True negatives matter, or you need to see precision and recall separately to make a decision.
One more point. Google adds that all these metrics are computed at a single fixed threshold and change when the threshold changes. F1 is a snapshot at one cut-off. To judge a model across every cut-off at once, you need a different tool.
Tip
A single number can hide the story. When you report F1, report precision and recall next to it. A reader who sees 0.72 alone cannot tell whether the model cries wolf or sleeps on the job.
Quick check
Quick check
A model has precision 1.0 and recall 0.0. What are its plain average and its F1?
Quick check
A hospital screening model must miss as few sick patients as possible, and false alarms only mean a second test. Which score fits best?
What to take away
One number from two
F1 is the harmonic mean of precision and recall: 2 times precision times recall, over their sum. It lands near the weaker of the two.
Tilt and average
F-beta moves the focus: beta 2 for recall, beta 0.5 for precision. With many classes, choose macro, micro or weighted on purpose.
Know the blind spot
F1 ignores true negatives and works at one threshold. Report precision and recall beside it.
Next: ROC-AUC, judging a model across every threshold.
Sources
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.