Structure the analysis before searching for evidence

Define what you’re looking for and what would count as evidence before starting your search.

Why it works

Confirmation bias operates heavily in search: the framing of a search query, the databases chosen, the first result clicked — all are shaped by the prior hypothesis. Specifying in advance what evidence would count for and against the hypothesis, and what sources would be authoritative, reduces the search process’s susceptibility to motivated filtering. It converts an open-ended exploration into a pre-registered test.

How to do it

1. Before researching any question, write: "I will consider my hypothesis confirmed if I find X; I will consider it disconfirmed if I find Y."
2. List the three or four sources that would be most credible — not the ones most likely to confirm.
3. Commit to reporting the result of the structured search, not just the findings you happened to encounter.
4. Do the search as designed, then evaluate.

Evidence

Pre-registration in science — committing to hypotheses and analysis plans before data collection — reliably reduces the rate of false positives, demonstrating that structure-before-search is an effective anti-bias intervention at the institutional level. (observational)

Pre-registration works in scientific studies with clear outcomes; informal everyday evidence search is harder to pre-register in the same rigorous way.

Sources

• Nosek et al. (2018), the preregistration revolution, PNAS
• Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606.

Common mistake

Specifying the hypothesis after starting to search, when the search results have already shaped what seems "important" — a subtle form of HARKing (hypothesizing after results are known).