Agglomerative clustering — a tree of merges
Agglomerative clustering starts with one cluster per row and merges the closest pair until the requested number of clusters remains. Ward's criterion merges the pair that increases total within-cluster variance the least, which pulls the result toward compact groups, similar in spirit to K-Means but with a full merge history you can cut at any height.
Ward's cut at 3 on scaled Iris scored silhouette 0.447 and adjusted Rand 0.615 against species. On the wine table the same linkage and k=3 scored adjusted Rand 0.790, and the silhouette path from k=2 to k=8 peaks at k=3 (0.277). That peak is the opposite of Iris, where the path preferred k=2. The tree is not a reason to ignore the cut you actually asked for.
Ward requires Euclidean distance. If you need a different distance, switch linkage to average or complete; the merge rule, and the clusters, will change.
Settings that change this fit
- n_clusters
- Where the tree is cut. The merges above that cut are discarded, so this is the only place the cluster count enters.
- linkage
- Ward minimizes variance. Complete linkage uses the farthest pair between clusters and tolerates less chaining. Average linkage sits between them.