Distinguish uncertainty (quantifiable) from ignorance (unquantifiable)

Know when you can assign a probability and when the situation is so novel that a number would be fabricated.

Why it works

Knight’s distinction between risk (quantifiable uncertainty) and uncertainty (unquantifiable) matters practically: assigning a 60% probability to an event requires a reference class or a model. When neither exists, the number is a feeling dressed as a statistic. Distinguishing the two prevents false precision from masking genuine ignorance, and prompts the honest response: "I don’t have enough information to put a calibrated number on this."

How to do it

1. Before assigning a probability, ask: "What reference class or model is this based on?"
2. If you cannot name one, label the estimate as "gut feel / unquantified" rather than a calibrated probability.
3. In group settings, explicitly flag when a discussion is in the "unquantifiable" regime to prevent false-precision decisions.
4. For genuinely novel situations, focus on scenarios and contingencies rather than point probabilities.

Evidence

Knight (1921) distinguished risk from uncertainty; Ellsberg (1961) demonstrated that people prefer quantifiable risk over unquantifiable uncertainty even when expected values are equal — the "ambiguity aversion" that makes people uncomfortable saying "I don’t know." The distinction is conceptual and foundational; direct evidence for this specific practice is mechanistic. (mechanistic)

In practice, almost all real situations fall somewhere between pure quantifiable risk and complete unknowability. The categorization is a matter of degree and judgment.

Sources

• Knight (1921), Risk, Uncertainty and Profit
• Ellsberg (1961), "Risk, ambiguity, and the Savage axioms," Quarterly Journal of Economics

Common mistake

Assigning a probability to every question as a performance of rationality — a confident 50% for a genuinely unknowable question is worse calibration than simply saying "I don’t know."