← Evaluate statistically based reports

8 Evaluating the overall claim, not just one flaw

The specification says questions "will require candidates to evaluate claims or conclusions" — the strongest answers weigh several factors together and reach an explicit verdict, rather than listing flaws in isolation.

Building a full evaluation

• Identify the SPECIFIC claim being made — quote or closely paraphrase it, so your evaluation clearly targets it.
• Work through the standard checklist: sampling method, response rate, study design, margin of error, and how the data is presented — but only report what is actually RELEVANT to this specific claim.
• Weigh the factors: is one flaw serious enough to undermine the claim entirely, or are several smaller issues each worth noting?
• Reach an explicit VERDICT: state whether the claim is justified, justified with a caveat, or not justified — and say why, referencing the specific evidence.

✕

Listing every possible flaw with equal weight

A strong evaluation identifies the MOST CONSEQUENTIAL problem for the specific claim being tested and explains its likely direction of effect, rather than mechanically listing every textbook flaw (sampling, response rate, wording, graphing) regardless of whether it actually matters for this claim.

Practise just this