How does "Actively seek disconfirming evidence" work?

Confirmation bias causes people to weight confirming evidence more than disconfirming evidence, producing overconfidence. Superforecasters counteract this by making disconfirmation search an explicit habit: they ask "what would prove me wrong?" before seeking supporting evidence. This does not eliminate the bias but changes the evidence set the estimate is drawn from.

How does "Decompose complex questions into sub-questions" work?

Forecasting a complex outcome directly invites an availability-based gut estimate, which is sensitive to framing and salient recent events. Decomposing into sub-questions forces the analysis to address each component independently, catches missed dependencies, and makes the estimate auditable. Aggregating well-calibrated sub-estimates produces more accurate total estimates than directly estimating the whole.

Superforecasting

Q: How does "Think and communicate in explicit probabilities" work?

Words like "likely" are systematically ambiguous — one person’s "likely" is another’s 55% and another’s 85%. Numerical probabilities force specificity, make predictions scorable, and create a feedback loop. Without numbers, there is no signal for whether you are calibrated. The discipline of naming a number also forces genuine uncertainty accounting — you cannot say "basically certain" when you have to commit to 95%.

Q: How does "Update beliefs frequently and in small increments" work?

Superforecasters update more often and in smaller increments than average forecasters. The mechanism is Bayesian: each new piece of evidence has a diagnostic weight, and updating in proportion to that weight produces calibration. The two failure modes are under-updating (anchoring on the original estimate) and over-updating (overweighting recent salient events). Small, frequent updates resist both by keeping the estimate close to current evidence.

Q: How does "Treat forecasting accuracy as a skill that improves with practice" work?

Superforecasters treat their scoring results as diagnostics, not verdicts. When they are miscalibrated in a domain, they investigate why — was the reference class wrong, was the evidence weighting off, was there a structural dependency they missed? This feedback orientation is the same mechanism that distinguishes deliberate practice from mere experience. Without it, repetition produces confidence without accuracy.

Q: How does "Use structured team forecasting to aggregate diverse views" work?

Group discussion tends to amplify the views of the most confident or senior member and suppress minority information. Aggregating independent estimates before discussion preserves information that discussion would wash out. The wisdom of crowds effect depends on independence; the mechanism breaks down the moment estimates are made in the presence of others’ views.

How do superforecasters make more accurate predictions than experts?

Philip Tetlock’s forecasting tournament research found that a subset of ordinary people — "superforecasters" — consistently outperform domain experts and intelligence analysts at probabilistic prediction. They share measurable cognitive and behavioral habits: they think in probabilities, update frequently on evidence, and actively seek disconfirming information. These habits are learnable.

In the Good Judgment Project, Tetlock and colleagues ran the largest forecasting tournament in history and found something unexpected: a small group of non-expert volunteers outperformed CIA analysts with access to classified intelligence. The superforecasters were not smarter overall — they deployed a specific set of cognitive practices that most people can adopt. Below are those practices, with the mechanism behind each and an honest account of the evidence.

Practices

Think and communicate in explicit probabilities

Replace vague language ("probably," "likely") with numerical probabilities.

Update beliefs frequently and in small increments

When new evidence arrives, adjust your probability estimate — even if the change is small.

Actively seek disconfirming evidence

Deliberately look for evidence that your current belief is wrong.

Decompose complex questions into sub-questions

Break a hard forecasting question into smaller, estimable pieces and aggregate them.

Treat forecasting accuracy as a skill that improves with practice

Take prediction errors as performance feedback, not as proof that forecasting is futile.

Use structured team forecasting to aggregate diverse views

Combine independent estimates from multiple people before the group talks — aggregation beats any single expert.

Practice this with IX Coach

Reading about a practice changes nothing on its own. IX Coach turns these into a guided, adaptive routine — discerning where you are in real time and walking the practice with you, session after session.

Practice this with IX Coach

IX Coach: 7 days free, then $40/month (about $1.30/day).

Superforecasting

Practices

Think and communicate in explicit probabilities

Update beliefs frequently and in small increments

Actively seek disconfirming evidence

Decompose complex questions into sub-questions

Treat forecasting accuracy as a skill that improves with practice

Use structured team forecasting to aggregate diverse views

Practice this with IX Coach

Related