[Rd] [R] How to improve the "OPTIM" results

[Spencer Graves]

<moved to R-devel> 

      What would people suggest someone do to improve the optimization 
capabilities in R?  A few ideas are mentioned below.  I just got 578 
hits from RSiteSearch("optimization", "fun"), and I wonder if there's a 
better way to get a reasonable overview of what's currently available in 
R -- and outside R?  My initial thought was to develop an enhanced 
version of 'optim' that would check for negative curvature (via 
indefinite Hessians), do auto-scaling (see 
http://finzi.psych.upenn.edu/R/Rhelp02a/archive/125518.html), and output 
an object of a class like 'mle' or a new class like 'optimMLE' for which 
appropriate methods could be written.  I have to often encountered 
problems with overparameterization, which could be fairly easily 
diagnosed via a singular or indefinite hessian.  It would help me, and I 
believe other users, if functions like 'optim' and 'nls' would include 
simple diagnostics like this -- rather than returning a cryptic error or 
warning message. 

      However, the comment from Hans Borchers (below) raises a related 
question:  What might be the best way to build a collaboration with 
existing optimization initiatives, possibly making R a platform of 
choice for making it easy for users to access and compare alternative 
optimization methods? 

      Thanks,
      Spencer         

Hans W Borchers wrote:
>> MORE GENERAL OPTIM ISSUES
>>
>>       I'm considering creating a package 'optimMLE' that would automate 
>> some of this and package it with common 'methods' that would assume that 
>> sum(fn(...)) was either a log(likelihood) or the negative of a 
>> log(likelihood), etc.  However, before I do, I need to make more 
>> progress on some of my other commitments, review RSiteSearch("optim", 
>> "fun") to make sure I'm not duplicating something that already exists, 
>> etc.  If anyone is interested in collaborating on this, please contact 
>> me off-line. 
>>
>>       Hope this helps. 
>>       Spencer
>>     
>
> Thanks for your tips on using `optim()'. I believe `optim' is a reasonable good
> implementation of numerical optimization techniques such as quasi-Newton BFGS or
> conjugate gradients. Maybe methods using modern line searches or trust regions
> can follow someday.
>
> Everybody applying `optim' should be aware that it is a *Local Optimization*
> (LO) approach. What you describe appears to be a step towards *Global
> Optimization* (GO) in R. And indeed more and more requests in to the R-help list
> are about `optim' as a tool for global optimization, not always being fully
> aware of the differences.
>
> I am wondering whether it would be more useful to provide one or two global
> optimization approaches to the R community. And as this is an active research
> area, there are many and with different advantages and drawbacks.
>
> I would like to propose IPOPT as one of he newer and more advanced methods for
> global optimization. This powerful software package is open source and available
> through the COIN-OR initiative and its Web pages:
>
>     http://www.coin-or.org/Ipopt/documentation/
>
>     ``Ipopt (Interior Point OPTimizer, pronounced I-P-Opt) is a software package 
>     for large-scale nonlinear optimization. Ipopt is written in C++ and is 
>     released as open source code under the Common Public License (CPL). It is
>     available from the COIN-OR initiative. The code has been written by Carl
>     Laird (Carnegie Mellon University) and Andreas Wachter (IBM's T.J. Watson 
>     Research Center), who is the COIN project leader for Ipopt.''
>
> PERHAPS the COIN project would agree for IPOPT to be integrated into the open
> source project R as a package. For testing it right now there is an AMPL-based
> interface to IPOPT at the NEOS solver:
>
>     http://neos.mcs.anl.gov/neos/solvers/nco:Ipopt/AMPL.html
>
> There may be other rewarding projects in the COIN-OR initiative, such as
> `SYMPHONY' for solving mixed-integer linear programs (MILP, stronger than `glpk'
> and `lp-solve'), or the BONMIN open source *non-linear* mixed integer
> programming (MINLP) solver. I could imagine R to be a good platform for
> integrating some of these algorithms.
>
> ------------
> Hans W. Borchers
> Control and Optimization Group
> ABB Corporate Research Germany
> hans-werner.borchers at de.abb.com
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>