Confusion matrix, precision and recall — without the maths headache

A spam filter with 99% accuracy sounds excellent — until you learn that only 1% of email is spam. A model that marks everything "not spam" also scores 99%, and catches nothing.

Accuracy answers "how often is it right?" That is rarely the question you care about. The real questions are: when it raises an alarm, is it real? and how much does it miss?

Precision and recall answer exactly those two, and they both come from one small table.

The confusion matrix

Four numbers. Every classification metric is built from them.

Two of them are mistakes, and they are not the same kind of mistake:

• False Positive — a false alarm. Good email in the spam folder.
• False Negative — a miss. Spam in the inbox.

Which one hurts more depends entirely on your problem. That single question decides which metric you optimise.

Get it in Python

from sklearn.metrics import confusion_matrix, classification_report

predictions = model.predict(X_test)

print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions, digits=3))

Output:

[[850  30]
 [ 20 100]]

              precision    recall  f1-score   support
           0      0.977     0.966     0.971       880
           1      0.769     0.833     0.800       120

Read the matrix as: 850 correctly called negative, 30 false alarms, 20 misses, 100 correctly caught.

The support column matters. 880 versus 120 means the classes are imbalanced — so accuracy alone would flatter the model badly.

Precision — can I trust an alarm?

Of everything the model flagged, how much was really positive?

Precision = TP / (TP + FP) = 100 / (100 + 30) = 0.77

When this model says "spam", it is right 77% of the time. The other 23% are real emails sent to the spam folder.

Optimise precision when false alarms are expensive. Spam filters, fraud blocks that freeze a customer's card, automated content removal. A false accusation costs more than a miss.

Recall — how much do I catch?

Of everything that really was positive, how much did the model find?

Recall = TP / (TP + FN) = 100 / (100 + 20) = 0.83

This model catches 83% of the spam. 17% still lands in the inbox.

Optimise recall when misses are expensive. Cancer screening, security threats, equipment failure. Missing a real case is far worse than an extra check.

The memory hook: precision = trust the alarm, recall = catch them all.

F1 — one number when both matter

F1 = 2 × (precision × recall) / (precision + recall) = 0.80

F1 scoreThe harmonic mean of precision and recall — punishes a bad score in either is the metric to report when you cannot say which mistake is worse. It uses the harmonic mean, so 0.99 precision with 0.02 recall gives F1 ≈ 0.04, not the comfortable 0.5 a plain average would suggest. Being brilliant at one and useless at the other is not a good model.

The trade-off is a dial, not a fact

Precision and recall move in opposite directions, and you control the dial. Every classifier produces a probability; the 0.5 cutoff is just a default:

probabilities = model.predict_proba(X_test)[:, 1]

strict = (probabilities > 0.8).astype(int)   # higher precision, lower recall
loose = (probabilities > 0.3).astype(int)    # higher recall, lower precision

Raise the threshold and the model only speaks when confident: fewer false alarms, more misses. Lower it and it flags everything suspicious: catches more, cries wolf more.

See the whole curve at once instead of guessing:

from sklearn.metrics import precision_recall_curve
import matplotlib.pyplot as plt

precision, recall, thresholds = precision_recall_curve(y_test, probabilities)

plt.plot(thresholds, precision[:-1], label="precision")
plt.plot(thresholds, recall[:-1], label="recall")
plt.xlabel("threshold")
plt.legend()
plt.show()

Where the two lines cross is a balanced choice. Where you should sit depends on your costs, not on the chart.

Picking a metric on purpose

For very imbalanced problems prefer average precision over ROC-AUC:

from sklearn.metrics import average_precision_score

print(average_precision_score(y_test, probabilities))

ROC-AUC can look flattering when negatives vastly outnumber positives, because a huge true-negative count keeps the false-positive rate low no matter what.

More than two classes

The same table grows, and classification_report still works. Only the averaging changes:

from sklearn.metrics import f1_score

print(f1_score(y_test, predictions, average="macro"))     # every class counts equally
print(f1_score(y_test, predictions, average="weighted"))  # weighted by class size

Use macro when small classes matter as much as large ones. Use weighted when you want an overall figure that respects class sizes. Saying "F1 = 0.82" without saying which average is not a complete statement.

A worked decision

You are building a model to flag machine failures a day in advance.

• A false alarm costs one engineer inspection: about ₹2,000
• A miss costs an unplanned production stop: about ₹4,00,000

A miss is 200 times more expensive. Optimise recall, accept plenty of false alarms, and set the threshold low:

flagged = (probabilities > 0.15).astype(int)

That is what "choose your metric" really means: put a number on each mistake and let the numbers pick.

FAQ

What is the difference between precision and recall?

Precision is the share of flagged items that were genuinely positive — trust in an alarm. Recall is the share of genuinely positive items that were flagged — how much you catch. Raising one usually lowers the other.

How do I read a confusion matrix?

Rows are the truth, columns are the prediction. The diagonal holds correct answers. Off-diagonal cells are the two error types: false positives (false alarms) and false negatives (misses).

When should I use F1 score instead of accuracy?

Use F1 when classes are imbalanced or when both error types matter. Accuracy is misleading when one class dominates, since always predicting that class already scores high.

How do I improve recall without destroying precision?

Lower the decision threshold on predict_proba and inspect the precision-recall curve. Choose the point where recall is high enough for your cost of a miss while precision remains acceptable.

Should I use ROC-AUC or PR-AUC for imbalanced data?

Use PR-AUC, obtained via average_precision_score. ROC-AUC looks optimistic on heavily imbalanced data because the large number of true negatives keeps the false-positive rate low.

Next reading on Sythra Articles

• Train your first ML model in 20 lines
• How to detect overfitting in your own model
• Train/test split and the accuracy lie