Coaching practices for Structured Evidence Evaluation
Describe almost anything you are working through and IX Coach finds the practices whose real-world fit is closest. For Structured Evidence Evaluation, these are the strongest matches in the current practice library.
Does this sound like the set of challenges you might be facing?
- A flimsy study that confirms what I want I’ll wave straight through, but a solid one that contradicts me I’ll pick apart for hours
- I’m leaning hard on the evidence that lines up with what I want to be true and brushing off the rest
- I start "researching" a question but I notice I’m really just typing in searches that hand me the answer I already wanted
- The story hangs together so neatly
- Someone stated something with total confidence and I just nodded and built on it
Practices that may help
- Apply the same evidence standard regardless of whether you like the conclusion
Grade the evidence before you know which conclusion it supports.
Motivated Reasoning - Evaluate evidence by its likelihood ratio, not by how it makes you feel
Ask how much more likely this evidence would be if you’re right versus if you’re wrong.
Bayesian Thinking: How to Update Beliefs Rationally - Structure the analysis before searching for evidence
Define what you’re looking for and what would count as evidence before starting your search.
Confirmation Bias: Seeing What You Expect to See - Identify which features are actually diagnostic
Separate features that genuinely differentiate categories from ones that just complete the picture.
The Representativeness Heuristic — Judging by Resemblance - Probe for evidence behind claims
Ask "what evidence supports that?" before accepting any factual or evaluative claim.
Socratic Questioning - Build branches that are MECE at every level
Ensure each set of branches is mutually exclusive (no overlap) and collectively exhaustive (nothing important missing).
Issue Tree Analysis - State your prior probability before seeing the evidence
Before looking at any data, commit to a numerical estimate of how likely something is.
Bayesian Thinking: How to Update Beliefs Rationally - Group supporting ideas into mutually exclusive, collectively exhaustive buckets
Organize your three to five supporting points so they don’t overlap and together cover the argument completely (MECE).
The Minto Pyramid Principle - Practice elenchus — cross-examine your own beliefs
Pick one confident belief and interrogate it as Socrates would: What do you mean? How do you know? What would refute it?
The Socratic Method, Made Practical - Rate the strength of each logical link
Assign a confidence to each "supports" or "rebuts" link, not just to the conclusion.
Argument Mapping
Related concerns
- Evidence Quality Check
Grade the evidence before you know which conclusion it supports.
Apply the same evidence standard regardless of whether you like the conclusion
- How To Evaluate Evidence Quality
Grade the evidence before you know which conclusion it supports.
- What Counts As Evidence
Ask how much more likely this evidence would be if you’re right versus if you’re wrong.
Evaluate evidence by its likelihood ratio, not by how it makes you feel
- How To Evaluate Evidence Fairly
Grade the evidence before you know which conclusion it supports.
- Probe Evidence Critical Thinking
Ask "what evidence supports that?" before accepting any factual or evaluative claim.
Probe for evidence behind claims
- Asymmetric Evidence Standards
Grade the evidence before you know which conclusion it supports.
Describe your situation in your own words to search the complete practice library.