[R] nonlinear least squares fitting Trust-Region"
Spencer Graves
spencer.graves at pdf.com
Sun Sep 3 02:41:21 CEST 2006
Hi, Ravi:
Thanks for the reference. I believe that Bates' thesis adviser
was Donald G. Watts. I'm not certain, but I think Watts had a
connection with the University of Wisconsin, but I'm not certain of
that. Maybe someone else will correct / confirm or amplify on this.
Best Wishes,
Spencer
RAVI VARADHAN wrote:
> I think the idea of parameter and intrinsic nonlinearity is due to Beale (JRSSB 1960). Was he Doug Bates' thesis advisor?
>
> Ravi.
>
> ----- Original Message -----
> From: Spencer Graves <spencer.graves at pdf.com>
> Date: Saturday, September 2, 2006 2:05 pm
> Subject: Re: [R] nonlinear least squares fitting Trust-Region"
> To: RAVI VARADHAN <rvaradhan at jhmi.edu>
> Cc: Prof Brian Ripley <ripley at stats.ox.ac.uk>, Martin Ivanov <tramni at abv.bg>, r-help at stat.math.ethz.ch
>
>
>> May I also suggest Bates and Watts (1988) Nonlinear
>> Regression
>> Analysis and Its Applications (Wiley). This book carefully
>> explains the
>> difference between "parameter effects" and "intrinsic" curvature
>> in
>> non-linear fitting. I don't know if this idea was original with
>> Bates or
>> Watts, but I believe that Bates' PhD dissertation made important,
>> original contributions to our understanding of it -- and it helped
>> get
>> him the faculty position in Statistics at the University of
>> Wisconsin,
>> where he still is. Bates is also a leading contributor to R.
>>
>> hope this helps.
>> spencer graves
>>
>> RAVI VARADHAN wrote:
>>
>>> As suggested by Prof. Ripley, you should read a good book in the
>>>
>> optimization area. One that I would highly recommend is the book
>> by Dennis and Schnabel (1983) - Numerical methods for
>> unconstrained optimization, which does a great job of explaining
>> both "line-search" and "trust-region" approaches for achieving
>> globally-convergent versions of a fast numerical scheme such as
>> Gauss-Newton.
>>
>>> Best,
>>> Ravi.
>>>
>>> ----- Original Message -----
>>> From: Prof Brian Ripley <ripley at stats.ox.ac.uk>
>>> Date: Saturday, September 2, 2006 5:51 am
>>> Subject: Re: [R] nonlinear least squares fitting Trust-Region"
>>> To: Martin Ivanov <tramni at abv.bg>
>>> Cc: r-help at stat.math.ethz.ch
>>>
>>>
>>>
>>>> I believe people (including me) did not reply because you
>>>>
>> appeared
>>
>>>> not to
>>>> have done your homework. The help page for ?nls _does_ have a
>>>> reference
>>>> to the 'port' documentation, and RSiteSearch("trust region") is
>>>> informative and leads to an R package that does trust-region
>>>> optimization.
>>>> (So would looking in the R FAQ.)
>>>>
>>>> You say:
>>>>
>>>>
>>>>
>>>>> Since I am not an expert in the field of optimization, I am
>>>>>
>> just
>>
>>>>> conforming to what matlab documentation
>>>>>
>>>>>
>>>> Please note that some of the R developers are really expert in
>>>> that area,
>>>> and their advice (in the R documentation) should be taken as
>>>> seriously as
>>>> that in some commercial package that is merely commenting about
>>>> the very
>>>> sparse choice it offers. Or if R is not in your personal trust
>>>> region,
>>>> just use 'matlab'.
>>>>
>>>> Please
>>>>
>>>> 1) do not shout at your helpers: using all caps is regarded as
>>>> shouting.
>>>> 2) study and follow the posting guide. People are much more
>>>> likely to
>>>> help you if you demonstrate you have made efforts to help yourself.
>>>>
>>>> 3) read the literature. The R FAQ leads to books that cover
>>>> fitting
>>>> non-linear models in S/R in considerable detail.
>>>>
>>>>
>>>> On Sat, 2 Sep 2006, Martin Ivanov wrote:
>>>>
>>>>
>>>>
>>>>> Dear Mr Graves,
>>>>>
>>>>> Thank you very much for your response. Nobody else from this
>>>>>
>>>>>
>>>> mailing
>>>>
>>>>
>>>>> list ventured to reply to me for the two weeks since I posted
>>>>>
>> my
>>
>>>>> question. "nlminb" and "optim" are just optimization
>>>>>
>> procedures.
>>
>>>>>
>>>>>
>>>> What I
>>>>
>>>>
>>>>> need is not just optimization, but a nonlinear CURVE FITTING
>>>>>
>>>>>
>>>> procedure.
>>>> Which is just optimization: usually by least squares (although
>>>>
>> you
>>
>>>> have
>>>> not actually specified that and there are better modern
>>>> statistical
>>>> ideas).
>>>>
>>>>
>>>>
>>>>> If there is some way to perform nonlinear curve fitting with
>>>>>
>> the
>>
>>>>> "Trust-Region" algorithm using any of these functions, I would
>>>>>
>>>>>
>>>> me much
>>>>
>>>>
>>>>> obliged to you if you suggest to me how to achieve that. You
>>>>>
>>>>>
>>>> asked me
>>>>
>>>>
>>>>> why I do not want Gauss-Newton. Since I am not an expert in
>>>>>
>> the
>>
>>>>>
>>>>>
>>>> field of
>>>>
>>>>
>>>>> optimization, I am just conforming to what matlab
>>>>>
>> documentation
>>
>>>>> suggests, namely: "Algorithm used for the fitting procedure:
>>>>> Trust-Region -- This is the default algorithm and must be used
>>>>>
>>>>>
>>>> if you
>>>>
>>>>
>>>>> specify coefficient constraints. Levenberg-Marquardt -- If the
>>>>> trust-region algorithm does not produce a reasonable fit, and
>>>>>
>>>>>
>>>> you do not
>>>>
>>>>
>>>>> have coefficient constraints, you should try the Levenberg-
>>>>>
>>>>>
>>>> Marquardt
>>>>
>>>>
>>>>> algorithm. Gauss-Newton --THIS ALGORITHM IS POTENTIALLY FASTER
>>>>>
>>>>>
>>>> THAN THE
>>>>
>>>>
>>>>> OTHER ALGORITHMS, BUT IT ASSUMES THAT THE RESIDUALS ARE CLOSE
>>>>>
>> TO
>>
>>>>>
>>>>>
>>>> ZERO.
>>>>
>>>>
>>>>> IT IS INCLUDED FOR PEDAGOGICAL REASONS AND SHOULD BE THE LAST
>>>>>
>>>>>
>>>> CHOICE FOR
>>>>
>>>>
>>>>> MOST MODELS AND DATA SETS. I browsed some literature about the
>>>>>
>>>>>
>>>> garchfit
>>>>
>>>>
>>>>> function, but I did not see the "Trust-Region" algorithm there
>>>>>
>>>>>
>>>> either:
>>>>
>>>>
>>>>> algorithm = c("sqp", "nlminb", "lbfgsb", "nlminb+nm",
>>>>>
>>>>>
>>>> "lbfgsb+nm"),
>>>>
>>>>
>>>>> control = list(), title = NULL, description = NULL, ...)
>>>>>
>>>>> Thank you for your attention. I am looking forward to your reply.
>>>>> Regards,
>>>>> Martin
>>>>>
>>>>> ---------------------------------------------------------------
>>>>>
>> --
>>
>>>>> vbox7.com - ??????? ????? ???????!
>>>>>
>>>>> ______________________________________________
>>>>> R-help at stat.math.ethz.ch mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> PLEASE do read the posting guide http://www.R-
>>>>>
>>>>>
>>>> project.org/posting-guide.html
>>>>
>>>>
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Brian D. Ripley, ripley at stats.ox.ac.uk
>>>> Professor of Applied Statistics,
>>>>
>> http://www.stats.ox.ac.uk/~ripley/>> University of Oxford,
>> Tel: +44 1865 272861 (self)
>>
>>>> 1 South Parks Road, +44 1865 272866 (PA)
>>>> Oxford OX1 3TG, UK Fax: +44 1865 272595
>>>>
>>>> ______________________________________________
>>>> R-help at stat.math.ethz.ch mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide http://www.R-
>>>>
>> project.org/posting-
>>
>>>> guide.htmland provide commented, minimal, self-contained,
>>>> reproducible code.
>>>>
>>>>
>>>>
>>> ______________________________________________
>>> R-help at stat.math.ethz.ch mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-
>>>
>> project.org/posting-guide.html
>>
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
More information about the R-help
mailing list