Coaching practices for Reference Class Confidence

Describe almost anything you are working through and IX Coach finds the practices whose real-world fit is closest. For Reference Class Confidence, these are the strongest matches in the current practice library.

Does this sound like the set of challenges you might be facing?

• My confidence comes straight from how good a story I can tell myself about this particular case, and the more vivid the story the surer I get
• I make calls all the time but I never write them down or check them later, so my confidence is just a feeling that floats free
• When I’d swear I know roughly where a number lands, the real answer is constantly outside the range I’d have bet on
• I’m sure about things constantly and I have no idea if that confidence is earned
• I slap a confident-sounding percentage on things I genuinely have no basis to estimate, just to seem rational

Practices that may help

1. Anchor your confidence level on base rates, not gut feel
   Start your confidence estimate from the historical hit rate for this type of prediction, not from how confident you feel.
   Calibration Training
2. Reference Class Forecasting
   Reference class forecasting, developed by Daniel Kahneman and Amos Tversky and formalized by Bent Flyvbjerg, improves forecast accuracy by anchoring on the statistical distribution of outcomes for similar past projects rather than on the details of the current one. The method reliably corrects the optimism bias that inflates cost, time, and benefit estimates in planning — the evidence base here is real and specific.
3. Maintain a scored prediction log
   Record predictions with explicit probabilities and score them when they resolve.
   Calibration Training
4. Practice confidence interval estimation
   Estimate ranges for factual quantities and check how often the true value falls within your range.
   Calibration Training
5. Practice probabilistic calibration by tracking your predictions
   Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
   Base-Rate Neglect: Why We Ignore the Odds
6. Distinguish uncertainty (quantifiable) from ignorance (unquantifiable)
   Know when you can assign a probability and when the situation is so novel that a number would be fabricated.
   Calibration Training
7. Practice calibration by tracking your confidence predictions
   Your "80% confident" beliefs should come true about 80% of the time — check that they do.
   Bayesian Thinking: How to Update Beliefs Rationally
8. Regression-to-mean awareness in JOLs
   Discount your highest-confidence JOLs most — they are the most likely to be inflated.
   Judgments of Learning: Why You Misjudge What You’ve Learned
9. Calibration Training
   Calibration training teaches you to match your stated confidence levels to your actual accuracy: if you say "70% confident," roughly 70% of those claims should turn out to be correct. Tetlock’s forecasting research and probability training studies show calibration is trainable with practice and feedback — the evidence here is specific and replicable.
10. Identify the right reference class for your situation
    Find a well-defined set of past situations that are structurally similar to yours and collect their outcome data.
    Reference Class Forecasting

Describe your situation in your own words to search the complete practice library.