[R] tune an support vector machine

Mon Dec 9 10:57:56 CET 2013

   This is the solution fpr the problem.
   Its not the cleanest one i guess but it works just the way i wanted.
   If anybody has better performing or cleaner code, please send it to me.

   best regards
   Uwe

   type<-sample(c(-1,1) , 20, replace = TRUE )
   weight<-sample(c(20:50),20, replace=TRUE)
   height<-sample(c(100:200),20, replace=TRUE)
   width<-sample(c(30:50),20,replace=TRUE)
   volume<-sample(c(1000:5000),20,replace=TRUE)
   data<-cbind(type,weight,height,width,volume)
   train<-as.data.frame(data)
   train$type<-as.factor(train$type)
   library("e1071")
   features <- c("weight","height","width","volume")
   (formula<-as.formula(paste("type ~ ", paste(features, collapse= "+"))))
   svmtune=tune.svm(formula,         data=train,         kernel="radial",
   type="C-classification",cost=2^(-2:5), gamma=2^(-2:1),cross=10)
   summary(svmtune)

   namen<-expand.grid(c("weight",NA),
   c("height",NA),c("width",NA),c("volume",NA), stringsAsFactors=FALSE)
   l <- apply(head(namen, -1), 1, function(x)
                  reformulate(paste(na.omit(x), collapse = "+"), response =
   "type"))
   for (i in (1:(nrow(namen)-1)))
   {assign(paste("svmtune",i,sep=""),tune.svm(l[[i]],         data=train,
   kernel="radial",type="C-classification",                cost=2^(-2:1),
   gamma=2^(-2:1),cross=10))}
   for (i in (1:(nrow(namen)-1)))
   {Modell[i]<-eval(parse(text=paste("svmtune",i,"$best.performance",sep="")))}
   Modell2<-as.data.frame(Modell)
   bestesModell_Index<-which(Modell2==max(Modell2))[1]
   bestesModell_Gamma<-eval(parse(text=paste("svmtune",bestesModell_Index,"$bes
   t.parameters$gamma",sep="")))
   bestesModell_Cost<-eval(parse(text=paste("svmtune",bestesModell_Index,"$best
   .parameters$cost",sep="")))
   bestesModell_Formel<-l[[bestesModell_Index]]
   #bestesModell_Parameter
   bestesModell<-c(bestesModell_Index,bestesModell_Gamma,bestesModell_Cost,
   bestesModell_Formel)
   bestesModell_SVM<-svm(bestesModell_Formel, data=train, kernel="radial",
   gamma=bestesModell_Gamma, cost=bestesModell_Cost, type="C-classification")
   test<-data[1:10 ,]
   pred<-predict(bestesModell_SVM, test)
   test[, 1]
   pred
   table(pred, test[, 1])

   Gesendet: Samstag, 07. Dezember 2013 um 09:15 Uhr
   Von: "Uwe Bohne" <balu555 at gmx.de>
   An: "Wuming Gong" <gongx030 at umn.edu>
   Cc: "r-help mailinglist" <r-help at r-project.org>
   Betreff: Re: [R] tune an support vector machine
   Thank you very much,
   your proposal is one practical way to check for significant features.
   I tried to check for all combination in a loop, but unfortunately there is a
   problem with NA values.
   Maybe anybody has an idea.
   This is my expansion of the former code:
   namen<-expand.grid(c("weight",NA),
   c("height",NA),c("width",NA),c("volume",NA), stringsAsFactors=FALSE)
   namen2<-as.data.frame(namen)
   for(i in 1:nrow(namen2)){
   assign(paste("a", i, sep = ""), namen2[i,])
   }
   This generates vectors containing the features.
   If i pick one of them i can produce a formula that i can use for svm tuning.
   For example
   a7
   a7q<-t(as.data.frame(a7[!is.na(a7)]))
   a7q
   a7f<-as.formula(paste("type~",paste(a7q,collapse="+")))
   a7f
   and
   svmtune_a7=tune.svm(a7f, data=train, kernel="radial", cost=2^(-2:5),
   gamma=2^(-2:1),cross=10)
   works as desired.
   So my key idea was to tune SVM with every possibel "a...f" formula and
   choose the best one according to the best performance measure in the
   summary.
   Unfortunately I just have problems to make it in a loop.
   I tried
   for(iin1:nrow(namen2)){paste("a",i,"q",sep="")<-t(as.data.frame(paste("a",
   i,"[!is.na(a",i,")]", sep="")))}
   and produced error. Probably i didnt paste correctly.
   Any ideas?
   Thanks a lot!
   Uwe
   Gesendet: Samstag, 07. Dezember 2013 um 08:26 Uhr
   Von: "Wuming Gong" <gongx030 at umn.edu>
   An: "Uwe Bohne" <balu555 at gmx.de>
   Cc: "r-help mailinglist" <r-help at r-project.org>
   Betreff: Re: [R] tune an support vector machine
   Hi Uwe,
   It looks SVM in e1071 and Kernlab does not support feature selection, but
   you can take a look at package penalizedSVM
   ([1][1]http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf)
   .
   Or you can implement a SVM-RFE
   ([2][2]http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf)
   by
   the alpha values returned by svm() in e1071 or ksvm() in Kernlab.
   Wuming
   On Fri, Dec 6, 2013 at 7:06 AM, Uwe Bohne <[3]balu555 at gmx.de> wrote:
   Hej all,
   actually i try to tune a SVM in R and use the package "e1071" wich
   works
   pretty well.
   I do some gridsearch in the parameters and get the best possible
   parameters
   for classification.
   Here is my sample code
   type<-sample(c(-1,1) , 20, replace = TRUE )
   weight<-sample(c(20:50),20, replace=TRUE)
   height<-sample(c(100:200),20, replace=TRUE)
   width<-sample(c(30:50),20,replace=TRUE)
   volume<-sample(c(1000:5000),20,replace=TRUE)
   data<-cbind(type,weight,height,width,volume)
   train<-as.data.frame(data)
   library("e1071")
   features <- c("weight","height","width","volume")
   (formula<-as.formula(paste("type ~ ", paste(features, collapse= "+"))))
   svmtune=tune.svm(formula, data=train, kernel="radial", cost=2^(-2:5),
   gamma=2^(-2:1),cross=10)
   summary(svmtune)
   My question is if there is a way to tune the features.
   So in other words - what i wanna do is to try all possible combinations
   of
   features : for example use only (volume) or use (weight, height) or use
   (height,volume,width) and so on for the SVM and to get the best
   combination
   back.
   Best wishes
   Uwe
   ______________________________________________
   [4]R-help at r-project.org mailing list
   [5][3]https://stat.ethz.ch/mailman/listinfo/r-help
   PLEASE do read the posting guide
   [6][4]http://www.R-project.org/posting-guide.html
   and provide commented, minimal, self-contained, reproducible code.
   References
   1. [5]http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
   2. [6]http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf
   3. file://localhost/tmp/balu555@gmx.de
   4. file://localhost/tmp/R-help@r-project.org
   5. [7]https://stat.ethz.ch/mailman/listinfo/r-help
   6. [8]http://www.R-project.org/posting-guide.html
   ______________________________________________
   R-help at r-project.org mailing list
   [9]https://stat.ethz.ch/mailman/listinfo/r-help
   PLEASE do read the posting guide
   [10]http://www.R-project.org/posting-guide.html
   and provide commented, minimal, self-contained, reproducible code.

References

   1. http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
   2. http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf)by
   3. https://stat.ethz.ch/mailman/listinfo/r-help
   4. http://www.R-project.org/posting-guide.html
   5. http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
   6. http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf
   7. https://stat.ethz.ch/mailman/listinfo/r-help
   8. http://www.R-project.org/posting-guide.html
   9. https://stat.ethz.ch/mailman/listinfo/r-help
  10. http://www.R-project.org/posting-guide.html