Practice probability calibration

Regularly make probabilistic predictions and track your accuracy across many of them.

Why it works

Calibration is the match between your stated confidence and your actual accuracy: if you say "80% confident" across 100 predictions, you should be right roughly 80 times. Hindsight bias interferes with calibration by making you believe you were more confident about outcomes you got right than you actually were. Training calibration directly — making many small forecasts and scoring them — builds accurate self-knowledge about uncertainty, which is the foundation of better decisions under uncertainty.

How to do it

1. Pick a domain with checkable outcomes: sports results, project deadlines, colleague decisions.
2. Make a probabilistic prediction and write it down with a confidence level.
3. After the outcome, score whether the outcome fell within your stated probability.
4. Over dozens of predictions, calculate your calibration score and notice where you are systematically over- or under-confident.

Evidence

Calibration training is one of the few debiasing interventions with real evidence of improving forecasting accuracy. Research on superforecasters found that people trained in probabilistic thinking outperform experts without such training. (observational)

Superforecasting findings come from specific geopolitical forecasting contexts; how much the skill transfers to personal and professional decisions is plausible but not directly studied.

Sources

• Tetlock & Gardner (2015), Superforecasting: The Art and Science of Prediction

Common mistake

Making predictions only on things you expect to get right, rather than sampling broadly, which produces fake calibration from a non-random selection of easy cases.