Decision tree classifier – sorting data with yes/no questions
Computer ScienceData, AI & Machine LearningAges 17–18
Loading…
Sign in to playTwo-class data on a plane defined by two features (weather and going outside, apple or lemon, passing a test). The computer builds a decision tree by choosing, at each node, the question "feature ≤ threshold?" that reduces Gini impurity or entropy the most, or students click to cut regions themselves and preview the impurity drop. The plane is split into rectangles and the tree is drawn below; change the maximum depth to compare accuracy on training and test data and see overfitting clearly.
Lesson: Machine learning: decision trees, Gini impurity and entropy, training and test data, overfitting
What it shows
A decision tree classifies an example by asking a chain of yes/no questions, each comparing one feature with a threshold, so the feature plane is split into rectangles. The data here come from a hidden rule plus randomly flipped labels (noise); 70% train the tree and 30% are kept for testing. At every node the computer greedily picks the cut that lowers the weighted Gini impurity or entropy the most. Deeper trees fit the training data better but can start memorising noise, so test accuracy stops improving or falls.
How to use
In Computer-built tree, move Maximum depth from 1 to 8 and watch training and test accuracy on the chart. Switch to Your own questions, choose Vertical cut or Horizontal cut, hover to preview the impurity drop and click a region to split it. Compare with Best question, undo with Merge, and press New data to repeat.
Parameters you can change
- Data set Go outside? (temperature, humidity), Apple or lemon? (mass, redness of skin), Pass the test? (hours of revision, practice score)
- Number of data points 40–200 points
- Noise (share of flipped labels) 0–30 %
- Maximum tree depth 1–8
- Splitting criterion Gini impurity, Entropy (information gain)
- Show test data
Questions to explore
- Why does training accuracy keep rising with depth while test accuracy does not?
- Why does the "Pass the test?" data set, with its diagonal boundary, need so many questions?
- What does the first question chosen at the root tell you about the data?
