Track the accuracy of your domain predictions

Keep a record of what you predicted in your area of claimed expertise and score it.

Why it works

Experts frequently over-attribute good outcomes to their expertise and blame bad outcomes on circumstances — a self-serving attribution pattern that sustains overconfidence. Tracking actual predictions and scoring them objectively breaks this by creating a record that can’t be rewritten by attribution. In domains where you genuinely have skill, the record will confirm it; where confidence is inflated, the record will show it.

How to do it

1. In your area of work or expertise, start logging specific predictions: a market outcome, a project timeline, how a person will respond.
2. Assign a confidence level to each prediction before knowing the result.
3. Score each prediction as correct or not once the result is known.
4. After 20+ predictions, calculate your accuracy rate by confidence level — 80%-confident predictions should be right roughly 80% of the time if you are calibrated.

Evidence

Tracking prediction accuracy is the method used in formal forecasting research to identify who has genuine expertise versus confidence built on noise. It is the empirical test that separates the two. (observational)

Tetlock’s work is on geopolitical prediction; transfer to personal and professional domains is plausible but not directly tested at the same scale.

Sources

• Tetlock (2005), Expert Political Judgment: How Good Is It? How Can We Know?

Common mistake

Selecting only predictions you feel confident about, which produces a biased sample. The point is to track all predictions in the domain, not just the ones you expect to win.