[R] svmpath: fitting the entire SVM regularization path
Trevor Hastie
hastie at stanford.edu
Tue Jul 6 20:24:28 CEST 2004
svmpath is a contributed package that fits the entire regularization path
for a two-class SVM model.
The SVM (with any kernel), has a regularization or cost parameter C, which
controls the amount of overlap
at the soft margin. When the SVM criterion is expressed in terms of a hinge
loss plus lambda x quadratic penalty, then lambda=1/C.
In many situations, the choice of C can be critical, and different regimes
for C are called for as the other kernel tuning parameters
are changed.
Most software packages come with a default value for C (typically very
large), and the user is left to explore different values of C.
It turns out that the lagrange multipliers which define the SVM solution for
any C are piecewise linear in C (and more usefully piecewise
linear and mostly piecewise constant in lambda) This means that we can
compute the entire sequence of solutions for all values of C exactly.
svmpath does this with essentially the same cost as fitting a single SVM
model with a specified value of C.
See the paper (joint work with Saharon Rosset, Ji Zhu and Rob Tibshirani)
http://www-stat.stanford.edu/~hastie/Papers/svmpath.pdf
for details.
This code has been tested on moderate sized problems, with up to 1000
observations. The current version is not industry
ready; occasionally it will run into situations where the steps are too
small, leading to machine zero situations. Usually increasing the
parameter eps from its default 1e-10 will avoid this.
Trevor Hastie
--------------------------------------------------------------------
Trevor Hastie hastie at stanford.edu
Professor, Department of Statistics, Stanford University
Phone: (650) 725-2231 (Statistics) Fax: (650) 725-8977
(650) 498-5233 (Biostatistics) Fax: (650) 725-6951
URL: http://www-stat.stanford.edu/~hastie
address: room 104, Department of Statistics, Sequoia Hall
390 Serra Mall, Stanford University, CA 94305-4065
More information about the R-help
mailing list