[R] nonlinear least squares fitting Trust-Region"

Sun Sep 3 01:36:02 CEST 2006

I think the idea of parameter and intrinsic nonlinearity is due to Beale (JRSSB 1960).  Was he Doug Bates' thesis advisor?

Ravi.

----- Original Message -----
From: Spencer Graves <spencer.graves at pdf.com>
Date: Saturday, September 2, 2006 2:05 pm
Subject: Re: [R] nonlinear least squares fitting Trust-Region"
To: RAVI VARADHAN <rvaradhan at jhmi.edu>
Cc: Prof Brian Ripley <ripley at stats.ox.ac.uk>, Martin Ivanov <tramni at abv.bg>, r-help at stat.math.ethz.ch

>      May I also suggest Bates and Watts (1988) Nonlinear 
> Regression 
> Analysis and Its Applications (Wiley).  This book carefully 
> explains the 
> difference between "parameter effects" and "intrinsic" curvature 
> in 
> non-linear fitting. I don't know if this idea was original with 
> Bates or 
> Watts, but I believe that Bates' PhD dissertation made important, 
> original contributions to our understanding of it -- and it helped 
> get 
> him the faculty position in Statistics at the University of 
> Wisconsin, 
> where he still is.  Bates is also a leading contributor to R. 
> 
>      hope this helps. 
>      spencer graves
> 
> RAVI VARADHAN wrote:
> > As suggested by Prof. Ripley, you should read a good book in the 
> optimization area.  One that I would highly recommend is the book 
> by Dennis and Schnabel (1983) - Numerical methods for 
> unconstrained optimization, which does a great job of explaining 
> both "line-search" and "trust-region" approaches for achieving 
> globally-convergent versions of a fast numerical scheme such as 
> Gauss-Newton.
> >
> > Best,
> > Ravi.
> >
> > ----- Original Message -----
> > From: Prof Brian Ripley <ripley at stats.ox.ac.uk>
> > Date: Saturday, September 2, 2006 5:51 am
> > Subject: Re: [R] nonlinear least squares fitting Trust-Region"
> > To: Martin Ivanov <tramni at abv.bg>
> > Cc: r-help at stat.math.ethz.ch
> >
> >   
> >> I believe people (including me) did not reply because you 
> appeared 
> >> not to 
> >> have done your homework.  The help page for ?nls _does_ have a 
> >> reference 
> >> to the 'port' documentation, and RSiteSearch("trust region") is 
> >> informative and leads to an R package that does trust-region 
> >> optimization.  
> >> (So would looking in the R FAQ.)
> >>
> >> You say:
> >>
> >>     
> >>> Since I am not an expert in the field of optimization, I am 
> just 
> >>> conforming to what matlab documentation
> >>>       
> >> Please note that some of the R developers are really expert in 
> >> that area, 
> >> and their advice (in the R documentation) should be taken as 
> >> seriously as 
> >> that in some commercial package that is merely commenting about 
> >> the very 
> >> sparse choice it offers.  Or if R is not in your personal trust 
> >> region, 
> >> just use 'matlab'.
> >>
> >> Please
> >>
> >> 1) do not shout at your helpers: using all caps is regarded as 
> >> shouting.
> >> 2) study and follow the posting guide.  People are much more 
> >> likely to 
> >> help you if you demonstrate you have made efforts to help yourself.
> >>
> >> 3) read the literature.  The R FAQ leads to books that cover 
> >> fitting 
> >> non-linear models in S/R in considerable detail.
> >>
> >>
> >> On Sat, 2 Sep 2006, Martin Ivanov wrote:
> >>
> >>     
> >>> Dear Mr Graves,
> >>>       
> >>> Thank you very much for your response. Nobody else from this 
> >>>       
> >> mailing 
> >>     
> >>> list ventured to reply to me for the two weeks since I posted 
> my 
> >>> question. "nlminb" and "optim" are just optimization 
> procedures. 
> >>>       
> >> What I 
> >>     
> >>> need is not just optimization, but a nonlinear CURVE FITTING 
> >>>       
> >> procedure.
> >> Which is just optimization: usually by least squares (although 
> you 
> >> have 
> >> not actually specified that and there are better modern 
> >> statistical 
> >> ideas).
> >>
> >>     
> >>> If there is some way to perform nonlinear curve fitting with 
> the 
> >>> "Trust-Region" algorithm using any of these functions, I would 
> >>>       
> >> me much 
> >>     
> >>> obliged to you if you suggest to me how to achieve that. You 
> >>>       
> >> asked me 
> >>     
> >>> why I do not want Gauss-Newton. Since I am not an expert in 
> the 
> >>>       
> >> field of 
> >>     
> >>> optimization, I am just conforming to what matlab 
> documentation 
> >>> suggests, namely: "Algorithm used for the fitting procedure: 
> >>> Trust-Region -- This is the default algorithm and must be used 
> >>>       
> >> if you 
> >>     
> >>> specify coefficient constraints. Levenberg-Marquardt -- If the 
> >>> trust-region algorithm does not produce a reasonable fit, and 
> >>>       
> >> you do not 
> >>     
> >>> have coefficient constraints, you should try the Levenberg-
> >>>       
> >> Marquardt 
> >>     
> >>> algorithm. Gauss-Newton --THIS ALGORITHM IS POTENTIALLY FASTER 
> >>>       
> >> THAN THE 
> >>     
> >>> OTHER ALGORITHMS, BUT IT ASSUMES THAT THE RESIDUALS ARE CLOSE 
> TO 
> >>>       
> >> ZERO. 
> >>     
> >>> IT IS INCLUDED FOR PEDAGOGICAL REASONS AND SHOULD BE THE LAST 
> >>>       
> >> CHOICE FOR 
> >>     
> >>> MOST MODELS AND DATA SETS. I browsed some literature about the 
> >>>       
> >> garchfit 
> >>     
> >>> function, but I did not see the "Trust-Region" algorithm there 
> >>>       
> >> either: 
> >>     
> >>> algorithm = c("sqp", "nlminb", "lbfgsb", "nlminb+nm", 
> >>>       
> >> "lbfgsb+nm"), 
> >>     
> >>> control = list(), title = NULL, description = NULL, ...)
> >>>
> >>> Thank you for your attention. I am looking forward to your reply.
> >>> Regards,
> >>> Martin
> >>>
> >>> ---------------------------------------------------------------
> --
> >>> vbox7.com - ??????? ????? ???????!
> >>>
> >>> ______________________________________________
> >>> R-help at stat.math.ethz.ch mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> PLEASE do read the posting guide http://www.R-
> >>>       
> >> project.org/posting-guide.html
> >>     
> >>> and provide commented, minimal, self-contained, reproducible code.
> >>>
> >>>       
> >> -- 
> >> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> >> Professor of Applied Statistics,  
> http://www.stats.ox.ac.uk/~ripley/>> University of Oxford,         
>    Tel:  +44 1865 272861 (self)
> >> 1 South Parks Road,                     +44 1865 272866 (PA)
> >> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> >>
> >> ______________________________________________
> >> R-help at stat.math.ethz.ch mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-
> project.org/posting-
> >> guide.htmland provide commented, minimal, self-contained, 
> >> reproducible code.
> >>
> >>     
> >
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-
> project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >   
>