Track your calibration across many predictions

Score your confidence levels against outcomes over dozens of predictions to find your systematic biases.

Why it works

A single decision tells you almost nothing about your decision-making quality — outcomes are too noisy. Calibration — the match between stated confidence and actual accuracy — only becomes visible across many predictions. If you say "80% confident" across 20 decisions and are right only 50% of the time, you are systematically overconfident at that level, and the journal gives you the data to see it.

How to do it

1. When logging each decision, assign a probability to your expected outcome (e.g. 70%).
2. After outcomes are resolved, record whether the outcome matched the prediction.
3. After 20–30 entries, group predictions by confidence band (50–60%, 70–80%, 90%+) and calculate your accuracy rate in each band.
4. If your accuracy lags your confidence consistently, revise your confidence levels downward in that domain.

Evidence

Calibration training — making probabilistic predictions and scoring them against outcomes — is one of the few interventions with evidence of genuinely improving forecasting accuracy. The superforecasting research showed that explicit tracking and feedback narrows overconfidence. (observational)

Superforecasting evidence comes from geopolitical prediction tournaments; transfer to personal and professional decisions is plausible but the effect sizes in ordinary life decisions are not directly established. Calibration improves with practice, but overconfidence is persistent even in experienced practitioners.

Sources

• Tetlock & Gardner (2015), Superforecasting: The Art and Science of Prediction

Common mistake

Logging decisions without assigning probabilities, which makes calibration tracking impossible and leaves you with only a narrative record instead of a statistical one.