Big Data and Recommender Systems – Filter Bubbles
Computer ScienceData, AI & Machine LearningAges 17–18
Use it with your class: students open it from a link or a QR code, with no account needed.
Next in Data, AI & Machine Learning
The top half shows the big data pipeline: raw records are collected, then cleaned (removing duplicates, missing fields and badly formatted entries) before analysis; students drag the record count and the cleaning level to see how the number of usable records and the result's reliability change. The bottom half runs a recommender system round by round: it keeps suggesting the topic you watch most, your interest in it grows, diversity measured by entropy keeps dropping and a filter bubble forms; raising the exploration rate breaks the bubble open.
Lesson: Big data and artificial intelligence: data collection, cleaning, analysis and recommender systems
What it shows
Big data work follows a pipeline: collect raw records, clean them, analyze them and act on the results. Raw data from social media, sensors, sales or learning platforms contains duplicates, missing fields and errors, and without cleaning, even a huge dataset gives unreliable results. The second part models a recommender system that keeps suggesting whatever a user watches most. Diversity is measured by normalized entropy, H = −Σ pᵢ·log pᵢ ÷ log(number of topics), which falls toward 0% as one topic dominates and a filter bubble forms. An exploration rate adds occasional unfamiliar suggestions.
How to use
Choose a Data source and drag the Records and Cleaning sliders, comparing the number of usable records and the estimated reliability in the pipeline. Then set Exploration to 0%, press Run Recommender or use Step to go round by round, and watch the diversity curve fall. Press Reset, raise the exploration rate and run again to compare. Clicking a topic bar makes the user watch that topic, showing how quickly the system follows.
Parameters you can change
- Data source Social media posts, Smart device sensors, Sales transactions, Online learning logs
- Number of raw records collected 1–10000 thousand records
- Data cleaning level 0–100 %
- Recommender exploration rate 0–50 %
- Number of recommendation rounds 5–50 rounds
Questions to explore
- Why can a very large dataset that has not been cleaned still produce inaccurate results?
- What does a falling entropy value tell you about the content the user sees?
- Roughly what exploration rate keeps a filter bubble from forming within 20 rounds?
