Support vector regression — an epsilon tube around the target
SVR fits a function that ignores residuals smaller than epsilon. Only points outside that tube, and the points on its edge, become support vectors and determine the function. Inside the tube, a row can move a little without changing the fit. That is a different idea from least squares, which pays for every residual.
A linear kernel gives a weighted sum, like ridge, but with the tube instead of a squared penalty. An RBF kernel lets the function bend, with gamma controlling how local each bend is. C limits how hard the fit chases points outside the tube. A large C and a tiny epsilon will try to hit nearly every training target.
Scale both the columns and, if epsilon is left at a default, think about the target's units. An epsilon of 0.1 means something very different for a target in millimeters than for a target in dollars.
Settings that change this fit
- epsilon
- Half-width of the ignored tube, in target units. Wider tubes use fewer support vectors and smooth more.
- C
- Penalty on points outside the tube. Small C allows a wider miss and a flatter function.