[R] svmpath: fitting the entire SVM regularization path

Trevor Hastie hastie at stanford.edu
Tue Jul 6 20:24:28 CEST 2004

svmpath is a contributed package that fits the entire regularization path
for a two-class SVM model.

The SVM (with any kernel), has a regularization or cost parameter C, which
controls the amount of overlap
at the soft margin. When the SVM criterion is expressed in terms of a hinge
loss plus lambda x quadratic penalty, then lambda=1/C.
In many situations, the choice of C can be critical, and different regimes
for C are called for as the other kernel tuning parameters
are changed.

Most software packages come with a default value for C (typically very
large), and the user is left to explore different values of C.
It turns out that the lagrange multipliers which define the SVM solution for
any C are piecewise linear in C (and more usefully piecewise
linear and mostly piecewise constant in lambda) This means that we can
compute the entire sequence of solutions for all values of C exactly.
svmpath does this with essentially the same cost as fitting a single SVM
model with a specified value of C.

See the paper (joint work with Saharon Rosset, Ji Zhu and Rob Tibshirani)
for details.

This code has been tested on moderate sized problems, with up to 1000
observations. The current version is not industry
ready; occasionally it will run into situations where the steps are too
small, leading to machine zero situations. Usually increasing the
parameter eps from its default 1e-10 will avoid this.

Trevor Hastie

  Trevor Hastie                                  hastie at stanford.edu
  Professor, Department of Statistics, Stanford University
  Phone: (650) 725-2231 (Statistics)          Fax: (650) 725-8977
  (650) 498-5233 (Biostatistics)   Fax: (650) 725-6951
  URL: http://www-stat.stanford.edu/~hastie
  address: room 104, Department of Statistics, Sequoia Hall
           390 Serra Mall, Stanford University, CA 94305-4065

More information about the R-help mailing list