Evaluate statistically based reports

Practice

Recall it first, then use it — testing yourself builds the memory, where rereading the notes above only feels easier.

Recall and application shuffled together — you have to work out what each item wants before you can answer it, which is the part the exam actually tests. Both kinds feed the same spacing schedule, so anything you miss comes back sooner.

Say the answer out loud before flipping — that attempt is where the learning happens, even when you get it wrong.

Exam-style only. Instant feedback, shuffled each attempt, best score saved on this device.

What actually comes up

Three real media/research reports every year, each with a resource booklet of graphs and figures (not fully reproduced here — only the question wording and the schedule's own worked figures). Every paper repeats the same shape: construct and interpret a confidence interval from real percentages, spot a specific design or sampling flaw, and argue whether a claim is actually justified by the evidence given.

Across 4 years

The pattern — what to prepare

The 2022 paper ↗ Marking schedule ↗

Question One — Children's eating habits (BBC Good Food Nation)

A survey of UK children and parents reported a margin of error of 3.2%, and claims that over half of UK children chop vegetables and were more interested in cooking during lockdown.

• Stated margin of error: 3.2%

• a(i) Show how the stated margin of error of 3.2% was calculated. Achieved
• a(ii) Explain why the margin of error is needed to interpret any inferential claims made from the survey. Achieved
• b Identify one of the survey percentages, and explain why it would not be appropriate to use the reported margin of error to construct an approximate 95% confidence interval using it. Merit
• c(i) Discuss what further information is required for the assumption to be valid that the surveyed sample is representative of all UK children at the time of the survey. Merit
• c(ii) Identify one further variable that could affect whether UK children can chop vegetables, and discuss its possible impact on the claim that over half of UK children chop vegetables. Excellence
• d Could the claim be made that a higher proportion of UK parents agreed than disagreed that their children were more interested in cooking during lockdown? Construct one confidence interval and interpret it. Excellence

Question Two — Memes as mood boosters

748 participants were split across treatment groups to test whether memes boost mood, rating memes for cuteness, humour, anxiety, and positive emotions on repeated similar scales.

• a Identify the explanatory and response variables for the study. Achieved
• b(i) Explain how the allocation of participants into the treatment groups should have been done. Achieved
• b(ii) Discuss, with specific reference to this experiment, why it is important to allocate participants this way. Merit
• c Explain why asking participants to rate each type of response multiple times using similar scales was important, using specific evidence from the report. Merit
• d Discuss TWO potential issues with extending the results of this study to all people in New Zealand. Excellence

Question Three — Companion Animals in New Zealand 2020

A nationally representative online survey (n = 3599, including 2322 pet owners) reported New Zealand's pet-ownership rate is second only to the US, using Census-based quotas for age, gender and region.

• Total sample: 3599 respondents, including 2322 pet owners

• a Suggest a suitable alternative headline for the report, identifying and describing the evidence that supports it. Achieved
• b Evaluate the claim that more than half of pet-less NZ households would like to have a companion animal, by calculating and interpreting a confidence interval. Merit
• c Explain what is meant by "nationally representative sample" in this context. Merit
• d Could a claim be made that the proportion of US households with a pet is higher than the proportion of NZ households with a pet? Construct one confidence interval and interpret it. Excellence

The 2025 paper ↗ Marking schedule ↗

Question One — Politeness to AI

A survey reported 44% of Americans (n = 2000) say they are polite when communicating with AI, with a stated margin of error of 2.2%. A comparable NZ survey found 624 of 1250 respondents were polite to AI.

• Americans: 44% polite, n = 2000, MoE 2.2%
• New Zealanders: 624/1250 ≈ 49.9% polite, n = 1250

• a(i) Show how the stated margin of error of 2.2% was calculated. Achieved
• b Assuming the sample is representative, construct and interpret a confidence interval for the proportion of Americans who are polite when communicating with AI. Merit
• c Can the claim "New Zealanders are more polite when communicating with AI than Americans" be made? Construct ONE confidence interval for the difference, and interpret it. Excellence

Question Two — Human-created vs AI-created art

A study randomly assigned "human-created" or "AI-created" labels to paintings (independent of their true origin) and asked participants to rate them, showing paintings in a random order.

• a(i) Explain why this study is an experimental study, and not an observational study. Achieved
• b(i) Explain why, for each participant, the labels "Human-created" and "AI-created" were randomly assigned to each painting. Merit
• c A potential issue is extending the results inappropriately. Discuss why this may be a problem for the claim "people are biased against AI-created art." Excellence

Question Three — The Planted Pot house-plant survey

An NZ online plant-pot retailer surveyed subscribers to its newsletter and social media about house-plant ownership; the target population was house-plant owners across NZ, Canada, the USA, the UK, and Australia.

• 25% of respondents own fewer than 10 house plants (below the 30–70% MoE range)
• Comparison: 43% spend <1 hr/week vs 38% spend 1–3 hrs/week on plant care

• b Describe ONE situation where the target population and the sampling frame for this survey may not match. Merit
• c Identify ONE survey percentage and explain why it would not be appropriate to use the reported margin of error to construct an approximate 95% confidence interval using it. Merit
• d Could the claim "a higher proportion spend less than one hour per week than one to three hours" be made? Construct ONE confidence interval and interpret it. Excellence

The 2024 paper ↗ Marking schedule ↗

Question One — UK body-part awareness

A survey funded by Pall Mall Medical (a private healthcare clinic), run via the OnePoll online panel, reported that 55% of UK adults could confidently state where their rectum is located.

• 55% confident locating the rectum, CI [52.8%, 57.2%]
• OnePoll panellists are paid 40c–
  per survey, claimable at ~$50 total

• b Construct and interpret a confidence interval for the proportion of UK men and women who can confidently state where their rectum is located. Achieved
• c One factor to consider when evaluating a report is who funded the study. Explain whether there is a potential issue with the source of funding here. Merit
• e Discuss whether the OnePoll sampling method is likely to generate a representative sample of all UK men and women — identifying the target population, the sampling frame, and at least one non-sampling error. Excellence

Question Two — Socks-over-shoes on icy footpaths

A study tested whether wearing socks over shoes reduces slipping on ice, measuring self-rated slipperiness, observer-rated slipperiness, and descent time; blinding participants or staff to the treatment was not possible.

• Previous falls: 57% of the sock group vs 73% of the control group
• Female participants: 50% of the sock group vs 33% of the control group

• b Identify the explanatory and response variables for this study. Achieved
• c(ii) Discuss TWO measures the researchers used to minimise the impact of not being able to blind participants, and how each may have helped. Merit
• d Discuss TWO of the baseline variables reported that could be confounding, and their potential effect on the findings. Excellence

Question Three — Overconfidence about landing a plane

A YouGov survey headline claimed "50% of men surveyed think they could land a passenger plane"; a separate University of Waikato experiment showed some participants a video before asking their confidence, concluding the video inflated it.

• 13% "very confident" they could land a plane relying only on air traffic control (below the 30–70% MoE range)

• b Identify one of the survey percentages in the YouGov study and explain why it would not be appropriate to use the rule-of-thumb margin of error to construct a confidence interval using it. Achieved
• e In a repeated version of the study, the researchers randomised the order in which two confidence questions were asked. Explain why. Merit
• c Discuss the main differences between the designs of the YouGov study and the University of Waikato study, including the study designs, the inferences each supports, and the assumptions needed. Excellence

The 2023 paper ↗ Marking schedule ↗

Question One — Daylight saving preferences

A survey of 1010 New Zealanders found 47% in favour of keeping daylight saving as it is. Respondents could select multiple reasons for their preference, so the reported reasons summed to more than 100%.

• 47% in favour of keeping daylight saving as is, n = 1010
• All-year supporters: 18% of sample (182), 65% gave the "daylight hours" reason
• Keep-as-is supporters: 47% of sample (475), 57% gave the same reason

• a Construct and interpret a confidence interval for the proportion of NZ adults in favour of keeping daylight saving as it is. Achieved
• b(ii) A student compares support for two reasons using a confidence interval for the difference between them. Discuss why this approach may or may not be appropriate. Merit
• c Can the claim "the proportion who want all-year daylight saving for this reason is larger than those who want to keep it as is" be made? Construct ONE confidence interval and interpret it. Excellence

Question Two — A hotel towel-reuse experiment

Hotel guests were randomly assigned to one of three towel-reuse sign conditions; the number of towels used per room per day was measured, with guests blinded to being studied and staff blinded to the hypothesis.

• a Discuss ONE reason why the number of towels used per room per day is a good choice for the response variable. Achieved
• b Discuss how random allocation minimises the impact of one variable other than the explanatory variable, its likely impact on the response variable, and how random allocation lessens the issue. Merit
• c(i) Describe the blinding used with hotel guests and with hotel staff in the context of this study. Merit

Question Three — Where are the audiences?

A media-consumption report asked respondents what platforms they used "yesterday", and used regional sample stratification as part of its sampling method.

• a Discuss why asking about behaviour "yesterday" (rather than in general) is important to reduce bias in survey responses. Merit
• b Describe what "regional sample stratification" means in this situation, and why its use may be important. Merit

Every claim-evaluation part rewards a stated verdict with the calculated interval as the reason — a correct confidence interval with no verdict, or a verdict with no interval, both fall short of the top band.

Past exams & resources

⏱ Timed opens the paper and starts a countdown matching the exam length. Official NZQA PDFs. If a link 404s, that year’s paper may sit under a slightly different code: browse the NZQA subject page.