[R] L-BFGS-B needs finite values of 'fn'
Paul Smith
phhs80 at gmail.com
Thu Apr 3 22:00:08 CEST 2008
Thanks, Ravi, for your insight. My initial experiments were conducted
with b = 0.7, and my procedure (similar to yours) worked reasonably
well. Then I wanted to check the quality of my procedure with lower
b's, and the weak result motivated my initial post here.
I have meanwhile got a reasonably accurate solution with R through the
DEoptim package. The number of iterations was 50000 and the code the
following:
library(DEoptim)
k <- 10000
b <- 0.3
f <- function(x) {
n <- length(x)
if (sum(x) > k)
r <- Inf
else r <- -sum((b^(0:(n-1)))*log(x))
return(r)
}
nvar <- 10
(sols <- DEoptim(f, lower=rep(0.000000001,nvar), upper=rep(k,nvar),
control=list(itermax=50000)))
summary(sols)
The result was:
***** summary of DEoptim object *****
best member : 6999.892 2099.949 630.0255 189.0694 56.85699 16.95149
5.12811 1.527 0.46193 0.13798
best value : -11.91101
after : 50001 iterations
FUN evaluated : 2500050 times
*************************************
I am still struggling to find a fast way of getting the solution to my
problem at least as accurate as the one returned by DEoptim. With so
many iterations, DEoptim takes longer than 15 minutes to complete the
job.
Paul
On Thu, Apr 3, 2008 at 4:38 PM, Ravi Varadhan <rvaradhan at jhmi.edu> wrote:
> Paul,
>
> After looking at your objective function and the penalty, I realized that
> since the contribution of each term to the objective function decreases
> geometrically, the later terms contribute relatively less to the overall
> maximum. Hence the numerical estimation of those terms is much less precise
> compared to the dominant terms. This explains why the ratios of x[k]/x[k-1]
> is not close to b, for small values of b.
>
> Now let us try a larger value of b, say, b=0.8.
>
> Here are the results from optim, with "BFGS".
>
> > k <- 10000
> > b <- 0.8
> >
> > f <- function(x, pen, k, b) {
> + n <- length(x)
> + r <- sum((b^(0:(n-1)))*log(x)) - pen*(sum(x)-k)^2
> + return(r)
> + }
> >
> > gr <- function(x, pen, k, b) {
> + n <- length(x)
> + r <- (b^(0:(n-1)))*(1/x) - 2*pen*(sum(x)-k)
> + return(r)
> + }
> >
> >
> > nvar <- 10
> > p0 <- runif(nvar, 0, 20)
> > sols <- optim(p0, f, gr, method="BFGS", control=list(fnscale=-1), k=k,
> b=b, pen=1)
> > sum(sols$par)
> [1] 10000
> > sols$par[2:nvar] / sols$par[1:(nvar-1)]
> [1] 0.8038902 0.8021216 0.7974323 0.7999681 0.7983918 0.8006497 0.8013940
> [8] 0.8044702 0.7879365
> >
> >
>
> As you can see, things are much better!
>
> Hope this helps,
>
> Ravi.
> ----------------------------------------------------------------------------
> -------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvaradhan at jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
> ----------------------------------------------------------------------------
> --------
>
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
>
>
> Behalf Of Ravi Varadhan
> Sent: Wednesday, April 02, 2008 2:03 PM
> To: 'Paul Smith'; 'R-help'
> Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
>
> Yes, that is very important. If you look at the ratios x[k]/x[k-1], they
> are very close to 0.3 for the first few components, and then they start
> slowly diverging (ratio becomes smaller than 0.3) from that.
>
> So, optim is indeed finding a correct solution to the problem that you
> "actually" posed. You could increase your penalty to get a solution that is
> closer to the analytical solution you are expecting.
>
> Ravi.
>
> ----------------------------------------------------------------------------
> -------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvaradhan at jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
> ----------------------------------------------------------------------------
> --------
>
>
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
> Behalf Of Paul Smith
> Sent: Wednesday, April 02, 2008 1:57 PM
> To: R-help
> Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
>
> But let me add the following: the part
>
> - 2000000*(sum(x)-k)^2
>
> of my function is a penalty. In truth, I want to maximize
>
> sum((b^(0:(n-1)))*log(x))
>
> s.t.
>
> sum(x) = k.
>
> Paul
>
>
> On Wed, Apr 2, 2008 at 6:48 PM, Paul Smith <phhs80 at gmail.com> wrote:
> > Thanks, Ravi. The analytical solution, (x_1,x_2,...,x_10), should
> > satisfy this equality:
> >
> > x_t / x_(t-1) = 0.3.
> >
> > Unfortunately, the procedure that you suggest does not lead to a
> > solution that satisfies such an equality.
> >
> > Paul
> >
> >
> >
> >
> >
> > On Wed, Apr 2, 2008 at 5:12 PM, Ravi Varadhan <rvaradhan at jhmi.edu> wrote:
> > > Paul,
> > >
> > > Have you tried using "BFGS" without bounds?
> > >
> > > sols <- optim(rep(20,nvar), f, gr, method="BFGS",
> control=list(fnscale=-1))
> > >
> > > This converges to a solution, although I don't know if the converged
> > > solution is what you want.
> > >
> > > Ravi.
> > >
> > >
> ----------------------------------------------------------------------------
> > > -------
> > >
> > > Ravi Varadhan, Ph.D.
> > >
> > > Assistant Professor, The Center on Aging and Health
> > >
> > > Division of Geriatric Medicine and Gerontology
> > >
> > > Johns Hopkins University
> > >
> > > Ph: (410) 502-2619
> > >
> > > Fax: (410) 614-9625
> > >
> > > Email: rvaradhan at jhmi.edu
> > >
> > > Webpage:
> http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
> > >
> > >
> > >
> > >
> ----------------------------------------------------------------------------
> > > --------
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: r-help-bounces at r-project.org
> [mailto:r-help-bounces at r-project.org] On
> > > Behalf Of Paul Smith
> > >
> > > Sent: Monday, March 31, 2008 2:25 PM
> > > To: R-help
> > >
> > >
> > > Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
> > >
> > > On Mon, Mar 31, 2008 at 2:57 PM, Zaihra T <zaihra at uwindsor.ca> wrote:
> > > > try something like this before wrapping up your function else i
> guess
> > > u'll
> > > > have to stick to Prof Brian Ripley suggestion his suggestions are
> usually
> > > > best bet .
> > > >
> > > > f <- function(x) {
> > > >
> > > > n <- length(x)
> > > >
> > > > r <- sum((b^(0:(n-1)))*log(x)) - 2000000*(sum(x)-k)^2
> > > > if(!is.finite(r))
> > > >
> > > > r<-1e+20 return(r)
> > > >
> > > > }
> > > >
> > > > have a nice day.
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, 31 Mar 2008 12:24:09 +0100 "Paul Smith" wrote:
> > > > > Dear All,
> > > > >
> > > > > I am trying to solve the optimization problem below, but I am
> always
> > > > > getting the following error:
> > > > >
> > > > > Error in optim(rep(20, nvar), f, gr, method = "L-BFGS-B", lower =
> rep(0,
> > > :
> > > > > L-BFGS-B needs finite values of 'fn'
> > > > >
> > > > > Any ideas?
> > > > >
> > > > > Thanks in advance,
> > > > >
> > > > > Paul
> > > > >
> > > > > -----------------------------------------! ------
> > > > >
> > > > > k <- 10000
> > > > > b <- 0.3
> > > > >
> > > > > f <- function(x) {
> > > > >
> > > > > n <- length(x)
> > > > >
> > > > > r <- sum((b^(0:(n-1)))*log(x)) - 2000000*(sum(x)-k)^2
> > > > >
> > > > > return(r)
> > > > >
> > > > > }
> > > > >
> > > > > gr <- function(x) {
> > > > >
> > > > > n <- length(x)
> > > > >
> > > > > r <- (b^(0:(n-1)))*(1/x) - 4000000*(sum(x)-k)
> > > > >
> > > > > return(r)
> > > > >
> > > > > }
> > > > >
> > > > > nvar <- 10
> > > > > (sols <-
> > > > >
> > > >
> > >
> optim(rep(20,nvar),f,gr,method="L-BFGS-B",lower=rep(0,nvar),upper=rep(k,nvar
> > >
> ),control=list(fnscale=-1,parscale=rep(2000,nvar),factr=1e-300,pgtol=1e-300)
> > > ))
> > >
> > > Not much progress, Zaihra. Unfortunately! I am wondering whether one
> > > can transform the original problem into an equivalent one and solvable
> > > with optim.
> > >
> > > I know the analytical solution; I am just trying to check how far can
> > > R go regarding optimization problems.
> > >
> > > Paul
> > >
> > > ______________________________________________
> > >
> > > R-help at r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >
> > >
> > > and provide commented, minimal, self-contained, reproducible code.
> > >
> >
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
More information about the R-help
mailing list