[Rd] Numerical optimisation and "non-feasible" regions

Patrick Burns pburns at pburns.seanet.com
Thu Aug 7 18:57:25 CEST 2008

If I understand your proposal correctly, then it
probably isn't a good idea.

A derivative-based optimization algorithm is going
to get upset whenever it sees negative infinity.
Genetic algorithms, simulated annealing (and I think
Nelder-Mead) will be okay when they see infinity
but if all infeasible solutions have value negative infinity,
then you are not giving the algorithm a clue about what
direction to go.


Mathieu Ribatet wrote:
> Dear Patrick (and other),
> Well I used the Sylvester's criteria (which is equivalent) to test for 
> this. But unfortunately, this is not the only issue!
> Well, to sum up quickly, it's more or less like geostatistics. 
> Consequently, I have several unfeasible regions (covariance, margins 
> and others).
> The problem seems that the unfeasible regions may be large and 
> sometimes lead to optimization issues - even when the starting values 
> are well defined.
> This is the reason why I wonder if setting by myself a $-\infty$ in 
> the composite likelihood function is appropriate here.
> However, you might be right in setting a tolerance value 'eps' instead 
> of the theoretical bound eigen values > 0.
> Thanks for your tips,
> Best,
> Mathieu
> Patrick Burns a écrit :
>> If the positive definiteness of the covariance
>> is the only issue, then you could base a penalty on:
>> eps - smallest.eigen.value
>> if the smallest eigen value is smaller than eps.
>> Patrick Burns
>> patrick at burns-stat.com
>> +44 (0)20 8525 0696
>> http://www.burns-stat.com
>> (home of S Poetry and "A Guide for the Unwilling S User")
>> Mathieu Ribatet wrote:
>>> Thanks Ben for your tips.
>>> I'm not sure it'll be so easy to do (as the non-feasible regions
>>> depend on the model parameters), but I'm sure it's worth giving a try.
>>> Thanks !!!
>>> Best,
>>> Mathieu
>>> Ben Bolker a écrit :
>>>> Mathieu Ribatet <mathieu.ribatet <at> epfl.ch> writes:
>>>>> Dear list,
>>>>> I'm currently writing a C code to compute the (composite) 
>>>>> likelihood -
>>>>> well this is done but not really robust. The C code is wrapped in 
>>>>> an R
>>>>> one which call the optimizer routine - optim or nlm. However, the
>>>>> fitting procedure is far from being robust as the parameter space
>>>>> depends on the parameter - I have a covariance matrix that should 
>>>>> be a
>>>>> valid one for example.
>>>>   One reasonably straightforward hack to deal with this is
>>>> to add a penalty that is (e.g.) a quadratic function of the
>>>> distance from the feasible region, if that is reasonably
>>>> straightforward to compute -- that way your function will
>>>> get gently pushed back toward the feasible region.
>>>>   Ben Bolker
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel

More information about the R-devel mailing list