Sampling methods – bias and variability of estimates

What it shows

The population is a simulated school of 400 students in four grades and 20 classes; each student's weekly hours of exercise are hidden until you reveal them. Every sampling method gives an estimate of the population mean or proportion. Repeating a method many times shows its sampling distribution: the centre compared with the true value is the bias, and the spread is the variability. A larger sample reduces variability but not bias. Selection probabilities in the convenience, quota and voluntary methods, and response rates, are simple illustrative models.

How to use

On Compare methods, choose a Method, Sample size n and Bias, then press Take 1 sample or Take 100 samples; Compare all methods fills the table with 200 samples of each. Tick Show true values to see the bias. On Investigate, pick a Question, press Next, choose a method and press Take a sample, then press Next to display the data and reach a conclusion.

Parameters you can change

• Screen Compare methods, Statistical investigation
• Sampling method Census, Simple random sample, Systematic sample, Stratified sample, Cluster sample (whole classes), Quota sample, Convenience (opportunity) sample, Voluntary response sample
• Sample size n 5–200
• Extra source of bias None, Undercoverage: list misses Grade 9, Nonresponse
• Quantity to estimate Mean hours of exercise, Proportion exercising at least 5 h a week
• Show the true population values
• Statistical question (Investigate screen) Is the mean more than 5 h a week?, Do Grade 6 students exercise more than Grade 9?, Do more than half exercise at least 5 h a week?
• Population seed (same number, same students) 1–999

Questions to explore

1. Which methods give estimates centred on the true value, and which are biased?
2. What happens to the spread of the estimates when you quadruple the sample size?
3. Why can a convenience sample give strong but wrong evidence?