Coaching practices for Superforecasting Calibration
Describe almost anything you are working through and IX Coach finds the practices whose real-world fit is closest. For Superforecasting Calibration, these are the strongest matches in the current practice library.
Does this sound like the set of challenges you might be facing?
- I make calls about how things will go all the time, but I never write them down or check them after
- When a prediction of mine blows up, I either spiral into thinking the whole thing is pointless or I just shrug and forget it
- I want to actually know whether my gut is any good — when I feel sure about something, am I right that often, or am I just confident?
- I’m sure about things constantly and I have no idea if that confidence is earned
- When I say I’m eighty percent sure I genuinely don’t know if that means anything
Practices that may help
- Score your own past predictions to calibrate your outside-view use
Keep a forecast log and score it — you cannot improve calibration without feedback on where you were over- or under-confident.
The Outside View - Calibration Training
Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable. - Treat forecasting accuracy as a skill that improves with practice
Take prediction errors as performance feedback, not as proof that forecasting is futile.
Superforecasting - Superforecasting
Philip Tetlock’s forecasting tournament research found that a subset of ordinary people — "superforecasters" — consistently outperform domain experts and intelligence analysts at probabilistic prediction. They share measurable cognitive and behavioral habits: they think in probabilities, update frequently on evidence, and actively seek disconfirming information. These habits are learnable. - Practice probability calibration
Regularly make probabilistic predictions and track your accuracy across many of them.
Hindsight Bias: Why Everything Seems Obvious in Retrospect - Practice probabilistic calibration by tracking your predictions
Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
Base-Rate Neglect: Why We Ignore the Odds - Practice calibration by tracking your confidence predictions
Your "80% confident" beliefs should come true about 80% of the time — check that they do.
Bayesian Thinking: How to Update Beliefs Rationally - Maintain a scored prediction log
Record predictions with explicit probabilities and score them when they resolve.
Calibration Training - Update beliefs frequently and in small increments
When new evidence arrives, adjust your probability estimate — even if the change is small.
Superforecasting - Update your forecast incrementally as new evidence arrives
Treat your forecast as a probability that should shift with each new piece of evidence, not a commitment that survives contradiction.
Reference Class Forecasting
Related concerns
- Confidence Prediction Tracking
Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
Practice probabilistic calibration by tracking your predictions
- Forecasting Calibration
Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable.
- Prediction Accuracy Tracking
Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
- Calibration Control Classification
Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable.
- Calibration Tracking Decisions
Keep a forecast log and score it — you cannot improve calibration without feedback on where you were over- or under-confident.
Score your own past predictions to calibrate your outside-view use
- Calibration Training When Starting Out
Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable.
Describe your situation in your own words to search the complete practice library.