Enormous scale — petabytes or more. Too large for traditional DBMS to store.
V
Velocity
Generated and needs to be processed at very high speed (near real-time). E.g. social media feeds, sensor data.
V
Variety
Structured (tables), semi-structured (JSON, XML), and unstructured (images, video, text).
Why Traditional DBMS Fails
Why Traditional Databases Can't Handle Big Data
Traditional RDBMS requires a fixed schema — Big Data often has no consistent structure
Cannot scale to petabytes on a single machine — too expensive
Processing is serial — too slow for high-velocity data streams
Solution: distributed systems — spread data and processing across many commodity machines (a cluster)
Key idea: move the computation to the data, not the data to the computation
Hadoop
Apache Hadoop
Hadoop is an open-source framework for distributed storage and processing of Big Data across clusters of computers. Two core components: HDFS and MapReduce.
HDFS (Hadoop Distributed File System) — stores data split into blocks across many nodes; each block replicated 3× for fault tolerance
MapReduce — programming model for processing large datasets in parallel
Scale horizontally by adding more commodity servers to the cluster
Fault tolerant — if one node fails, data is recovered from replicas
MapReduce
MapReduce
MapReduce processes data in two phases: Map (transform/filter in parallel) → Reduce (aggregate results).
MAP phase
Each node processes its local data chunk in parallel. Outputs key-value pairs.
E.g. word count: emit (word, 1) for each word found
→
REDUCE phase
Aggregates/combines key-value pairs with the same key.
E.g. word count: sum all 1s for each word → (word, total_count)
Traditional relational databases require a fixed schema. Big Data solutions often use schema-less (NoSQL) or graph databases for flexibility.
Graph database — stores data as nodes (entities) and edges (relationships). Example: social networks where relationships between users are complex and varied.
Key-value store — simple lookups (e.g. Redis, used for caching)
Document store — stores JSON/XML documents (e.g. MongoDB)
Real World Uses
Big Data Real-World Applications
Social media — Facebook processes billions of posts, likes, and events daily; recommends content
Healthcare — analysing genomic data, patient records, drug trial results at scale
Finance — fraud detection using real-time analysis of millions of transactions per second
Retail — Amazon recommendation engine uses purchase history from hundreds of millions of customers
Government — census data, traffic analysis, smart city sensor networks
AQA Exam Style
Practice Question
AQA 7517 — Paper 2 Style
(a) Describe the THREE characteristics of Big Data known as the 3 Vs. [3] (b) Explain why traditional relational database systems are not suitable for Big Data. [2] (c) Describe the Map phase and the Reduce phase of MapReduce processing. [4]
[9 marks]
3 marks
(a) Volume — the scale is so large (petabytes) that a single machine cannot store it [1]; Velocity — data is generated at very high speed and must be processed quickly or in real time [1]; Variety — data comes in different formats including structured, semi-structured, and unstructured [1]
2 marks
(b) Traditional RDBMS requires a fixed schema but Big Data is often unstructured [1]; it cannot scale to petabyte volumes on a single machine and processes data serially, too slowly for high-velocity data [1]
4 marks
(c) Map phase: data is divided across multiple nodes; each node processes its chunk in parallel [1] and outputs intermediate key-value pairs [1]; Reduce phase: key-value pairs with the same key are grouped [1] and aggregated to produce the final result [1]
Summary
Key Points to Remember
3 Vs of Big Data: Volume (scale), Velocity (speed), Variety (formats)
Traditional RDBMS: fixed schema, not scalable, serial processing — fails for Big Data
Hadoop — open-source; HDFS (distributed storage with replication) + MapReduce