Anchor your confidence level on base rates, not gut feel

Start your confidence estimate from the historical hit rate for this type of prediction, not from how confident you feel.

Why it works

Gut-feel confidence is contaminated by how fluently the answer came to mind, how motivated you are for the outcome, and how recently a similar event happened. None of these is a reliable proxy for accuracy. Base rates from a defined reference class provide a hard external anchor that separates the confidence estimate from the emotional and cognitive noise that gut feel mixes in.

How to do it

1. Before expressing a confidence level, ask: "For predictions like this one, what fraction have I gotten right historically?"
2. If you lack personal data, use domain base rates (e.g., new product success rates, diagnosis hit rates).
3. Set your initial confidence at the base rate, then adjust for specific features of this case.
4. Keep adjustments modest — the research on anchoring shows you will not adjust far enough anyway.

Evidence

Base-rate neglect is well documented: people weight case-specific information over statistical base rates, leading to overconfident individual assessments. Forcing base-rate anchoring before case-specific reasoning reduces this effect. (observational)

Base-rate anchoring improves accuracy on average but can be too conservative for genuinely unusual cases where the individual is unusually well-positioned relative to the reference class.

Sources

• Kahneman & Tversky (1973), "On the psychology of prediction," Psychological Review
• Kahneman & Lovallo (1993), on planning fallacy and inside-view bias

Common mistake

Using the base rate as one input among many rather than as the anchor, so it gets diluted by the compelling narrative of the specific case.