[R] How to obtain final gradient estimation from optim

[Stéphane Luchini]

Le Jeudi 27 Mars 2003 12:52, ripley at stats.ox.ac.uk a écrit :
> On Thu, 27 Mar 2003, [iso-8859-15] Stéphane Luchini wrote:
> > I use optim to compute maximum likelihood estimations without giving an
> > analytical gradient to optim. However, I would like to
> > get an output of the final numerical gradient vector and the final matrix
> > of contributions to the gradient. But I did not
> > find any mention of this kind of output in help pages. Does anyone know
> > how to do that ?
>
> No, and optim does not even necessarily calculate a gradient.
> But if it does, it is supposed to be zero at a maximum....

It is zero at the theoretical level, it is not zero at a numerical level and 
it can be used to compute the Outer Product of the Gradient as an estimator 
of the information matrix instead of the inverse hessian. For long 
computation, it enables one to get an estimation of standard deviations 
without computing the hessian matrix which can take a long time. 

>
> I don't lnow what you mean by `contributions to the gradient': optim works
> with the (I presume) log-likelihood.

The matrix of the contribution to the gradient with a typical element $G_{ti}$ 
defined as follows:

$$G_{ti}(y,\theta) = \partial \ell_t(u,\theta) / \partial \theta_i$$

where $\ell$ is the contribution i to the log-lokelihood. Using such a matrix, 
one can verify convergence using a Gauss-Newton regression such that

$$Intercept = G b + residuals$$

and the parameters b as to be close to zero if the parameters estimates are 
close to the true parameters. This is why it is usefull to get as an output 
(when it is avalaible) of the gradient and the G matrix. Moreover, when 
analytical gradients are not straightforward to define, it will allow one to 
compare the numerical results of two procedures (one with giving to R the 
analytical gradient and one without it).