[R] Very slow optim()
Deepayan Sarkar
deep@y@n@@@rk@r @end|ng |rom gm@||@com
Sat Mar 13 08:27:57 CET 2021
On Sat, Mar 13, 2021 at 10:08 AM Spencer Graves
<spencer.graves using effectivedefense.org> wrote:
>
> TWO COMMENTS:
>
>
> 1. DID YOU ASSIGN THE OUTPUT OF "optim" to an object, like "est <-
> optim(...)"? If yes and if "optim" terminated normally, the 60,000+
> paramters should be there as est$par. See the documentation on "optim".
>
>
> 2. WHAT PROBLEM ARE YOU TRYING TO SOLVE?
>
>
> I hope you will forgive me for being blunt (or perhaps bigoted), but
> I'm skeptical about anyone wanting to use optim to estimate 60,000+
> parameters. With a situation like that, I think you would be wise to
> recast the problem as one in which those 60,000+ parameters are sampled
> from some hyperdistribution characterized by a small number of
> hyperparameters. Then write a model where your observations are sampled
> from distribution(s) controlled by these random parameters. Then
> multiply the likelihood of the observations by the likelihood of the
> hyperdistribution and integrate out the 60,000+ parameters, leaving only
> a small number hyperparameters.
Just a comment on this comment: I think it's perfectly reasonable to
optimize 60k+ parameters with conjugate gradient. CG was originally
developed to solve linear equations of the form Ax=b. If x was not
large in size, one would just use solve(A, b) instead of an iterative
method.
Use of CG is quite common in image processing. A relatively small
300x300 image will give you 90k parameters.
-Deepayan
More information about the R-help
mailing list