Evaluating a statistically based report (internal)
Internal 2 credits Sampling error Non-sampling error Margin of error Purpose ClaimsAS 91266, 2 credits. Read a real published report — often a poll — and evaluate it. The standard is explicit: identify and comment on features and findings relevant to the report's PURPOSE, and Merit supports those comments with statistical evidence — a different shape from the "each component" ladder the rest of L2 statistics uses.
1 Reading the notation
A different shape from the rest of L2 statistics: identify → justify with evidence → assess overall quality against purpose.📖 Terms the schedule grades you on
4 terms- Evaluate (Achieved)
- Identify and comment on features/findings relevant to the report's purpose.
- With justification (Merit)
- Support comments with reference to statistical evidence and the statistical processes used.
- With statistical insight (Excellence)
- Integrate statistical and contextual information to assess QUALITY in terms of purpose.
- Named features (note 3)
- Population measures/variables; sampling methods; survey methods; sampling and non-sampling errors; sample size.
2 Sampling error vs non-sampling error
Only one of the two shrinks with a bigger sample. ✕Assuming a bigger sample fixes everything
A huge sample with a biased sampling frame, badly worded questions, or heavy non-response is still confidently wrong — size only addresses sampling error. Naming a specific NON-sampling error is usually the stronger evaluation point.3 The margin of error
What it covers, what it does not, and the rough n rule.| Ask | What to look for | What it undermines |
|---|---|---|
| What is the margin of error? | Quoted, and appropriate for n | Any claim of a small difference |
| Do two figures actually differ? | Whether their margins overlap | A headline claiming a lead or trend |
Two well-evidenced points beat seven asserted ones in a 2-credit standard. Pick the faults that actually threaten the report's main claim.
4 Purpose, method, and who is missing
Comments must be relevant to what the report is actually trying to establish.| Ask | What to look for | What it undermines |
|---|---|---|
| Who was sampled, and how? | Random, quota, self-selected, convenience | Generalisation to the population |
| Who is missing? | Coverage of the sampling frame | Representativeness |
| What was the response rate? | Non-response bias | Whether respondents resemble non-respondents |
| Who paid for it? | Selective emphasis | Which findings got reported |
| Does the claim match the data? | Causation asserted from a survey | The headline conclusion |
Practice
Recall it first, then use it — testing yourself builds the memory, where rereading the notes above only feels easier.
Recall and application shuffled together — you have to work out what each item wants before you can answer it, which is the part the exam actually tests. Both kinds feed the same spacing schedule, so anything you miss comes back sooner.
Say the answer out loud before flipping — that attempt is where the learning happens, even when you get it wrong.
Exam-style only. Instant feedback, shuffled each attempt, best score saved on this device.