← Analysing an area of computer science

Big data

Offered 2 of the last 4 years cached (2021, and confirmed by schedule in 2024) — a strong choice if you find data collection and bias arguments easier to write about than algorithms.

The foundations to define precisely

  • The 3 Vs: Volume (the sheer amount of data), Variety (structured, semi-structured and unstructured formats together), Velocity (the rate at which it arrives and must be processed). Veracity (trustworthiness) and Value (whether it is worth anything) are often added as a 4th and 5th.
  • Each V creates its own distinct technical problem: volume needs distributed storage across many machines, variety needs schema-flexible tools that do not assume one fixed structure, velocity needs stream processing rather than batch processing.
Worked example Naming who is missing from a dataset A company analyses customer behaviour using only data collected through its mobile app. Identify a problem with this data collection and explain its effect. IDENTIFY WHO IS ABSENT: customers without a smartphone, or who prefer not to install the app, are entirely missing from the dataset. NAME THE BIAS: this is sampling bias — the sample systematically differs from the true population of all customers, not just randomly smaller than it. EFFECT ON CONCLUSIONS: any pattern found (e.g. "most customers browse in the evening") may reflect app-users' habits specifically, not customers generally, and decisions based on it could underserve the excluded group. HOW IT COULD BE ADDRESSED: combine app data with another collection channel (in-store surveys, website analytics) that reaches the customers the app misses, and state the limitation explicitly when reporting conclusions. ✓ The strongest big-data answers always name WHO is absent from the dataset, not just that "the data might be biased" in the abstract. ✕

"More data" does not fix a biased collection method

A common weak answer suggests collecting MORE data as the fix for bias. If the collection method still only reaches the same group (e.g. still only app users), a bigger sample just repeats the same systematic gap more precisely — it does not close it. The fix is a different or additional collection method that reaches who was missing, not simply a larger version of the same one.

Where big data affects people in practice

  • Recommendation systems learn from past behaviour, so a biased collection method feeds into what future users are shown — the bias compounds rather than staying contained.
  • Distributed storage across many machines means a failure in one location should not lose data — redundancy is a deliberate design cost, not an accident of scale.
  • Privacy: combining several "harmless" datasets can re-identify individuals who were anonymous in any one of them alone.

Practise just this