Keep a decision journal to score your EV estimates

Log your probability estimates and payoff predictions, then compare them to what happened.

Why it works

EV thinking is only as good as the quality of the probability and outcome estimates feeding it. Without feedback, estimates drift toward whatever is emotionally convenient. A decision journal creates accountability: you record your estimates before the outcome, then revisit after. This feedback loop is the same mechanism that makes calibration practice effective — seeing systematic errors allows deliberate correction.

How to do it

1. Before each significant decision, write: the scenarios, the probabilities, the estimated values, and the EV result.
2. Note which option you chose and whether it matched the highest EV.
3. After the outcome resolves, record what actually happened.
4. Quarterly, review your estimates for systematic biases: optimism, recency, specific domain overconfidence.

Evidence

Decision journaling is a widely endorsed practice in forecasting and decision quality communities. Its feedback mechanism is identical to calibration practice, which has observational evidence for improving predictive accuracy in expert forecasters. (mechanistic)

The journal practice relies on honest retrospective evaluation; post-outcome motivated reasoning can contaminate even written records unless the pre-outcome note is detailed.

Sources

• Tetlock & Gardner (2015), Superforecasting, Crown Publishers

Common mistake

Recording only the decisions that went well, or rewriting the original prediction after the outcome is known — both eliminate the feedback the journal was designed to provide.