Coaching practices for How to Evaluate Evidence Fairly
Describe almost anything you are working through and IX Coach finds the practices whose real-world fit is closest. For How to Evaluate Evidence Fairly, these are the strongest matches in the current practice library.
Does this sound like the set of challenges you might be facing?
- A flimsy study that confirms what I want I’ll wave straight through, but a solid one that contradicts me I’ll pick apart for hours
- I’m leaning hard on the evidence that lines up with what I want to be true and brushing off the rest
- I start "researching" a question but I notice I’m really just typing in searches that hand me the answer I already wanted
- I want to know whether the evidence actually changed my mind or just confirmed where I already landed
- Someone stated something with total confidence and I just nodded and built on it
Practices that may help
- Apply the same evidence standard regardless of whether you like the conclusion
Grade the evidence before you know which conclusion it supports.
Motivated Reasoning - Evaluate evidence by its likelihood ratio, not by how it makes you feel
Ask how much more likely this evidence would be if you’re right versus if you’re wrong.
Bayesian Thinking: How to Update Beliefs Rationally - Structure the analysis before searching for evidence
Define what you’re looking for and what would count as evidence before starting your search.
Confirmation Bias: Seeing What You Expect to See - State your prior probability before seeing the evidence
Before looking at any data, commit to a numerical estimate of how likely something is.
Bayesian Thinking: How to Update Beliefs Rationally - Probe for evidence behind claims
Ask "what evidence supports that?" before accepting any factual or evaluative claim.
Socratic Questioning - Examine the evidence for and against
Systematically list what supports your interpretation and what doesn’t — like a fair-minded lawyer, not a prosecutor.
Check the Facts, Made Practical - Apply the double-standard check: would I accept this reasoning if it pointed the other way?
Test whether you are holding your team’s arguments to a lower bar than the other side’s.
The Scout Mindset - Specify what would falsify your belief before you look at evidence
Write down what evidence would prove you wrong before you go looking.
Falsification Thinking - Update beliefs by degrees, not wholesale
Treat new information as evidence that shifts probabilities, not as proof that changes everything.
Base-Rate Neglect: Why We Ignore the Odds - Examine the evidence for and against the thought
Treat the thought as a hypothesis and find the actual evidence on both sides.
Cognitive Restructuring: The CBT Method for Changing Unhelpful Thoughts
Related concerns
- Evidence Quality Check
Grade the evidence before you know which conclusion it supports.
Apply the same evidence standard regardless of whether you like the conclusion
- How To Weigh Evidence
Ask how much more likely this evidence would be if you’re right versus if you’re wrong.
Evaluate evidence by its likelihood ratio, not by how it makes you feel
- Probe Evidence Critical Thinking
Ask "what evidence supports that?" before accepting any factual or evaluative claim.
Probe for evidence behind claims
- What Counts As Evidence
Ask how much more likely this evidence would be if you’re right versus if you’re wrong.
- Evidence Based Thinking
Before looking at any data, commit to a numerical estimate of how likely something is.
State your prior probability before seeing the evidence
- How To Evaluate Evidence Quality
Grade the evidence before you know which conclusion it supports.
Describe your situation in your own words to search the complete practice library.