Support vector machine — classify from the margin
A linear SVM looks for the hyperplane that separates the classes with the widest margin. Only the rows on the margin, the support vectors, define it. An RBF kernel does the same search in a space of local bumps, so the boundary in the original columns can bend around a class. The hinge loss ignores rows that are already on the correct side of the margin by more than that gap.
C trades margin width against training mistakes. A large C will twist the boundary to classify a single odd row correctly. Gamma, for the RBF kernel, is how fast similarity decays with distance: a large gamma memorizes neighborhoods. The usual failure on a small CSV is a perfect training score and a useless boundary.
SVM scores are not probabilities. If you need a probability, calibrate them on a held-out fold. Do not read the decision function as a percent.
Settings that change this fit
- C
- Cost of a margin violation. Large C fits the training rows more tightly and generalizes worse when labels are noisy.
- kernel and gamma
- Linear keeps a flat boundary. RBF bends it. Gamma sets the RBF width; scale it with the column ranges or standardize first.