[R] Dealing with -Inf in a maximisation problem.
pgilbert902 at gmail.com
Mon Nov 7 17:45:20 CET 2016
> I am trying to deal with a maximisation problem in which it is possible
> for the objective function to (quite legitimately) return the value
(Just to add to the pedantic part of the discuss by those of us that do
not qualify as younger and wiser:)
Setting log(0) to -Inf is often convenient but really I think the log
function is undefined at zero, so I would not refer to this as "legitimate".
>which causes the numerical optimisers that I have tried to fall over.
In theory as well as practice. You need to have a function that is
defined on the whole domain.
> The -Inf values arise from expressions of the form "a * log(b)", with b
> = 0. Under the *starting* values of the parameters, a must equal equal
> 0 whenever b = 0, so we can legitimately say that a * log(b) = 0 in
This also is undefined and not "legitimate". I think there is no reason
it should be equal zero. We tend to want to set it to the value we think
of as the "limit": for a=0 the limit as b goes to zero would be zero,
but the limit of a*(-inf) is -inf as a goes to zero.
So, you really do need to avoid zero because your function is not
defined there, or find a redefinition that works properly at zero. I
think you have a solution from another post.
> these circumstances. However as the maximisation algorithm searches
> over parameters it is possible for b to take the value 0 for values of
> a that are strictly positive. (The values of "a" do not change during
> this search, although they *do* change between "successive searches".)
> Clearly if one is *maximising* the objective then -Inf is not a value of
> particular interest, and we should be able to "move away". But the
> optimising function just stops.
> It is also clear that "moving away" is not a simple task; you can't
> estimate a gradient or Hessian at a point where the function value is -Inf.
> Can anyone suggest a way out of this dilemma, perhaps an optimiser that
> is equipped to cope with -Inf values in some sneaky way?
> Various ad hoc kludges spring to mind, but they all seem to be fraught
> with peril.
> I have tried changing the value returned by the objective function from
> "v" to exp(v) --- which maps -Inf to 0, which is nice and finite.
> However this seemed to flatten out the objective surface too much, and
> the search stalled at the 0 value, which is the antithesis of optimal.
> The problem arises in a context of applying the EM algorithm where the
> M-step cannot be carried out explicitly, whence numerical optimisation.
> I can give more detail if anyone thinks that it could be relevant.
> I would appreciate advice from younger and wiser heads! :-)
> Rolf Turner
> -- Technical Editor ANZJS Department of Statistics University of
> Auckland Phone: +64-9-373-7599 ext. 88276
More information about the R-help