[R] convergence=0 in optim and nlminb is real?

[Adelchi Azzalini]

On Tue, 17 Dec 2013 15:21:57 +0000, Ravi Varadhan wrote:

RV> The optimization algorithms did converge to a limit point.  But,
RV> not to a stationary point, i.e. a point in parameter space where
RV> the first and second order KKT conditions are satisfied.  If you
RV> check the gradient at the solution, you will see that it is quite
RV> large in magnitude relative to 0.  So, why did the algorithms
RV> declare convergence?  Convergence is based on absolute change in
RV> function value and/or relative change in parameter values between
RV> consecutive iterations.  This does not ensure that the KKT
RV> conditions are satisfied.

This makes sense to me. Although I have indicated other possible
explanations, the most plausable was that the selected point is not
at a minimum, as you confirmed.

As in many other cases, the stopping rule of an optimizer can be a
delicate issue. However, since optim computes (on request) the Hessian
matrix, a check on its positive-definiteness seems to me a reasonable
check to be made by optim before declaring successful convergence.

RV> 
RV> Now, to the real issue:  your problem is ill-posed.  As you can
RV> tell from the eigenvalues of the hessian, they vary over 9 orders
RV> of magnitude.  This may indicate a problem with the data in that
RV> the log-likelihood is over-parametrized relative to the information
RV> in the data set.  Get a better data set or formulate a simpler
RV> model, and the problem will disappear.
RV>  

I had noticed this aspect of the relative order of magnitudes of the
eigenvalues. The model is not over-parameterized (in a formal sense),
but in some cases maximization of the log-likelihood can be a delicate
issue, yes. 

I am not specifically interested in fitting these data, nor any other
data. I am working on an update of package 'sn'. 

Thanks for your informative reply.

Adelchi
-- 
Adelchi Azzalini  <azzalini at stat.unipd.it>
Dipart.Scienze Statistiche, Università di Padova, Italia
tel. +39 049 8274147,  http://azzalini.stat.unipd.it/