The Scientific Method — Reference Sheet
Key concepts from Week 1. Keep this handy throughout the module.
The cycle
Problem → Plan → Data → Analysis → Conclusion
↑ │
└───────────────────────────────────┘PPDAC (MacKay & Oldford; popularized by Spiegelhalter, The Art of Statistics) is the scientific method as practised with data. Science is not a body of facts — it is a process for reducing uncertainty.
| Stage | Ask yourself |
|---|---|
| Problem | What exactly is the question? Is it testable? |
| Plan | What would I measure? What result would prove me wrong? |
| Data | Where does it come from? How was it collected? |
| Analysis | What does it show? Is that a big number? |
| Conclusion | What does it mean, and for whom? What new question does it raise? |
The Plan is the pivot: skip it and you collect biased samples and confounded comparisons. The cycle never really ends: every conclusion raises the next problem.
Reasoning types
| Type | Direction | Example |
|---|---|---|
| Deductive | General → specific | “If biomass is carbon-neutral, we predict zero net CO₂ over a growth cycle.” |
| Inductive | Specific → general | “We observe that forests take 40+ years to regrow, so biomass may not be carbon-neutral on policy timescales.” |
Deduction tells you what should follow from a theory. Induction tells you what the evidence suggests. Good science uses both — but deduction is where testable predictions come from.
Testable hypotheses
A hypothesis is testable if you can specify what evidence would disprove it. If no observation could ever show it wrong, it isn’t a scientific hypothesis.
| Statement | Testable? | Why? |
|---|---|---|
| “The Earth is 6,000 years old.” | Yes | Radiometric dating, geological evidence, and cosmological observations can (and do) contradict this. |
| “Wind farms reduce bird populations.” | Yes | Compare bird counts near wind farms to control sites before and after construction. |
| “Renewable energy is the right thing to do.” | No | This is a value judgement, not a factual claim. No measurement can confirm or refute it. |
| “Burning biomass is carbon-neutral.” | Depends | As stated, it’s vague. Over what timescale? Including what costs? Once you define the terms precisely, it becomes testable. |
If your hypothesis isn’t testable, sharpen it until it is.
Falsifiability (Popper)
- We can never prove a theory is true — only fail to disprove it.
- A single counterexample can overturn a theory, but a thousand confirmations can’t prove it forever.
- Confirmation bias makes us seek evidence for what we already believe. The scientific method forces us to look for evidence against it.
Every statistical test is the same five steps (Week 6): measure δ, build a null world where δ is really zero (e.g. by shuffling labels), drop δ into it, count how often the null world is as extreme (the p-value), then decide against a standard set in advance. Andrew Heiss’s Null worlds lets you play with it interactively.
The Wason selection task — a reminder
You were shown four cards and a rule. Most people instinctively check the cards that could confirm the rule, not the ones that could falsify it.
This isn’t a personal failure — it’s a universal cognitive bias. The whole point of the scientific method is to protect us from it by making disconfirmation systematic.
Questions to carry forward
These will recur throughout the module:
| Question | First appears | What it checks |
|---|---|---|
| “Is that a big number?” | Week 2 | Context and comparison |
| “Compared to what?” | Week 2 | Control and baseline |
| “What assumptions are we making?” | Week 3 | Hidden premises |
| “How plausible was this before we tested?” | Week 6 | Prior probability / base rate |
| “What is the model not capturing?” | Week 8 | Model limitations |
You don’t need to understand all of these yet. By Week 10, they should be reflexes.