k-nearest neighbours (k-NN) classification: an idea from machine learning
Computer ScienceData, AI & Machine LearningAges 17–18
Use it with your class: students open it from a link or a QR code, with no account needed.
Next in Data, AI & Machine Learning
The training data is two classes of points on a plane defined by two features. Students drag a new point (the star) to any position: the computer finds its k nearest neighbours, counts each class's votes and labels the new point; the shaded background shows the model's decision boundary. Add training points by clicking, and change k to see the model become sharper or smoother.
Lesson: Introduction to machine learning: classification problems, training data, the k-nearest neighbours algorithm
What it shows
k-nearest neighbours is one of the simplest supervised machine-learning algorithms. Labelled training examples are plotted by two features, such as mass and sweetness for oranges and grapefruit. To classify a new point, the algorithm measures the straight-line (Euclidean) distance to every training point, takes the k closest and lets them vote; a tie goes to the single nearest neighbour. A small k gives a jagged decision boundary that follows noisy points, a larger k a smoother one. When the classes overlap, some misclassification is unavoidable.
How to use
Choose a Problem and start with k = 1 and Shade the decision region on. Drag the star into different areas and read the votes. Raise Number of neighbours k and watch the boundary smooth out, then raise Spread to make the classes overlap. Use Add a class A point or Add a class B point to place an outlier and see how it affects small and large k. Press New data to test whether your conclusions hold.
Parameters you can change
- Number of neighbours k 1–15
- Training points per class 5–40 points
- Spread (overlap between the two classes) 1–5
- Shade the decision region
- Problem Orange or grapefruit (mass, sweetness), Two flower species (petal length, width), Spam or normal email (number of links, length)
Questions to explore
- Why is an odd value of k usually chosen when there are two classes?
- How confident can you be about a new point placed where the two classes overlap?
- How does one mislabelled training point change the decision boundary for k = 1 and for k = 7?
