Decision tree classifier — a list of thresholds
A classification tree picks a column and a threshold that most reduce impurity in the two resulting groups, then repeats on each group. Impurity is Gini or entropy; they rarely disagree on the winning split. The finished model is a nested list of rules a person can read: petal length above 2.45, then petal width above 1.75, and so on.
An unpruned tree keeps splitting until every leaf is pure, which on a CSV with duplicate-looking rows is memorization. Max depth, minimum samples per leaf, and minimum samples to split are the brakes. Cost-complexity pruning cuts the weakest branches after the fact and is the control that tracks a validation error instead of a guess about depth.
Trees ignore monotone transforms of a single column, so scaling does nothing. They do not ignore noise columns: a noise column can still win a split by chance on a small leaf.
Settings that change this fit
- max depth
- Longest chain of rules. Depth 3 is readable. Depth 20 on a few hundred rows is a lookup table.
- min samples leaf
- Smallest group that can sit at the end of a rule. Raising it stops the tree from isolating single rows.