Superforecasters Calibration Training: 10 Evidence-Based Coaching Practices

Describe almost anything you are working through and IX Coach finds the practices whose real-world fit is closest. For Superforecasters Calibration Training, these are the strongest matches in the current practice library.

Practices that may help

1. Calibration Training
   Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable.
2. Score your own past predictions to calibrate your outside-view use
   Keep a forecast log and score it — you cannot improve calibration without feedback on where you were over- or under-confident.
   The Outside View
3. Treat forecasting accuracy as a skill that improves with practice
   Take prediction errors as performance feedback, not as proof that forecasting is futile.
   Superforecasting
4. Superforecasting
   Philip Tetlock’s forecasting tournament research found that a subset of ordinary people — "superforecasters" — consistently outperform domain experts and intelligence analysts at probabilistic prediction. They share measurable cognitive and behavioral habits: they think in probabilities, update frequently on evidence, and actively seek disconfirming information. These habits are learnable.
5. Practice probability calibration
   Regularly make probabilistic predictions and track your accuracy across many of them.
   Hindsight Bias: Why Everything Seems Obvious in Retrospect
6. Practice probabilistic calibration by tracking your predictions
   Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
   Base-Rate Neglect: Why We Ignore the Odds
7. Reference Class Forecasting
   Reference class forecasting, developed by Daniel Kahneman and Amos Tversky and formalized by Bent Flyvbjerg, improves forecast accuracy by anchoring on the statistical distribution of outcomes for similar past projects rather than on the details of the current one. The method reliably corrects the optimism bias that inflates cost, time, and benefit estimates in planning — the evidence base here is real and specific.
8. Maintain a scored prediction log
   Record predictions with explicit probabilities and score them when they resolve.
   Calibration Training
9. Build calibration reps with low-stakes trivia and almanac questions
   Use factual trivia questions as a practice ground for calibration — outcomes resolve immediately and the stakes are zero.
   Calibration Training
10. Update your forecast incrementally as new evidence arrives
    Treat your forecast as a probability that should shift with each new piece of evidence, not a commitment that survives contradiction.
    Reference Class Forecasting

Does this sound like the set of challenges you might be facing?

• I make calls about how things will go all the time, but I never write them down or check them after→
• When a prediction of mine blows up, I either spiral into thinking the whole thing is pointless or I just shrug and forget it→
• I want to actually know whether my gut is any good — when I feel sure about something, am I right that often, or am I just confident?→
• I’m sure about things constantly and I have no idea if that confidence is earned→
• I make calls all the time but I never write them down or check them later, so my confidence is just a feeling that floats free→

Describe your situation in your own words to search the complete practice library.