[R] memory problems with lm

[Liaw, Andy]

I believe Prof. Ripley is right.  The problem is

>  $ std.dev: num [, 1:6] 7.40e-01 9.06e+00 4.70e+05 1.02e+06 
> 1.37e+07 ...
>   ..- attr(*, "dimnames")=List of 1

which looks like an array, rather than a vector.

Andy

> From: Adrian Dragulescu [mailto:adrian_d at eskimo.com] 
> 
> If I enforce the variables to be numeric it works fine.
> 
> > str(eff.fro)
> `data.frame':   6 obs. of  2 variables:
>  $ std.dev: num [, 1:6] 7.40e-01 9.06e+00 4.70e+05 1.02e+06 
> 1.37e+07 ...
>   ..- attr(*, "dimnames")=List of 1
>   .. ..$ : chr  "NSTRDSP" "CPFGEP" "WSWOLF" "NPILGRIM" ...
>  $ mean   : num  1.22e-01 1.82e+00 1.11e+05 2.13e+05 1.89e+06 ...
> > gc()
>          used (Mb) gc trigger (Mb)
> Ncells 578941 15.5    1166886 31.2
> Vcells 589444  4.5    2377385 18.2
> > eff.fro
>               std.dev         mean
> NSTRDSP  7.403749e-01 1.215686e-01
> CPFGEP   9.056763e+00 1.815686e+00
> WSWOLF   4.703588e+05 1.112832e+05
> NPILGRIM 1.017640e+06 2.134335e+05
> WSNMILE  1.367312e+07 1.892021e+06
> WSHIDESL 1.830811e+07 1.892021e+06
> > reg <- lm(log(as.numeric(mean)) ~ log(as.numeric(std.dev)),
> data=eff.fro)
> > reg
> 
> Call:
> lm(formula = log(as.numeric(mean)) ~ 
> log(as.numeric(std.dev)),     data =
> eff.fro)
> 
> Coefficients:
>              (Intercept)  log(as.numeric(std.dev))
>                  -1.6368                    0.9864
> 
> 
> Adrian
> 
> 
> On Thu, 29 Apr 2004, Liaw, Andy wrote:
> 
> > Can you show us the output of str(eff.fro)?  Do you have 
> other things in the
> > global environment or the search path that's taking up 
> memory?  What does
> > gc() say?
> >
> > Andy
> >
> > > From: Adrian Dragulescu
> > >
> > > Hello list,
> > >
> > > I've seen the recent discussions documenting problems with lm.
> > >
> > > I have encountered the following problem.  I use WinXP Pro with
> > > service pack 1, and R 1.9.0, on a XEON 2GHz, with 1GB of RAM.
> > >
> > > > eff.fro
> > >               std.dev         mean
> > > NSTRDSP  7.403749e-01 1.215686e-01
> > > CPFGEP   9.056763e+00 1.815686e+00
> > > WSWOLF   4.703588e+05 1.112832e+05
> > > NPILGRIM 1.017640e+06 2.134335e+05
> > > WSNMILE  1.367312e+07 1.892021e+06
> > > WSHIDESL 1.830811e+07 1.892021e+06
> > > > reg <- lm(log(mean) ~ log(std.dev), data=eff.fro)
> > > Error in model.matrix.default(mt, mf, contrasts) :
> > >         cannot allocate vector of length 1074790452
> > > > log(eff.fro$mean)
> > > [1] -2.1072763  0.5964635 11.6198339 12.2710808 14.4531561
> > > [6] 14.4531561
> > > > reg <- lm(log(eff.fro$mean) ~ log(eff.fro$std.dev))
> > > Error: cannot allocate vector of size 3360077 Kb
> > > > lef <- log(eff.fro)
> > > > lef
> > >             std.dev       mean
> > > NSTRDSP  -0.3005986 -2.1072763
> > > CPFGEP    2.2035117  0.5964635
> > > WSWOLF   13.0612512 11.6198339
> > > NPILGRIM 13.8329973 12.2710808
> > > WSNMILE  16.4309427 14.4531561
> > > WSHIDESL 16.7228546 14.4531561
> > > > lef <- log(eff.fro)
> > > > reg <- lm(lef$mean ~ lef$std.dev)
> > >
> > > Here the my computer completely crashed.  A window 
> poped-up and said
> > > memory problem at address ..., and if I want to debug.
> > >
> > > I ran the same code one more time, and it worked but it 
> did not work
> > > how I wanted (where is the slope?):
> > > > reg <- lm(lef$mean ~ lef$std.dev)
> > > > reg
> > >
> > > Call:
> > > lm(formula = lef$mean ~ lef$std.dev)
> > >
> > > Coefficients:
> > > (Intercept)
> > >       8.548
> > >
> > > >
> > > > summary(reg)
> > >
> > > Call:
> > > lm(formula = lef$mean ~ lef$std.dev)
> > >
> > > Residuals:
> > >       1       2       3       4       5       6
> > > -10.655  -7.951   3.072   3.723   5.905   5.905
> > >
> > > Coefficients:
> > >             Estimate Std. Error t value Pr(>|t|)
> > > (Intercept)    8.548      2.999    2.85   0.0358 *
> > > ---
> > > Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1
> > >
> > > Residual standard error: 7.346 on 5 degrees of freedom
> > >
> > > I ran again:
> > >
> > >   reg <- lm(log(mean) ~ log(std.dev), data=eff.fro)
> > >
> > > and I get the pop-up:
> > > The instruction at "0x6b4c45a5" referenced memory at "0x0032374a".
> > > The memory could not be "read".  Click OK to terminate 
> the program.
> > >
> > >
> > > Any ideas?  Thank you,
> > > Adrian
> > >
> > > ______________________________________________
> > > R-help at stat.math.ethz.ch mailing list
> > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide!
> > > http://www.R-project.org/posting-guide.html
> > >
> > >
> >
> >
> > 
> --------------------------------------------------------------
> ----------------
> > Notice:  This e-mail message, together with any 
> attachments, contains
> > information of Merck & Co., Inc. (One Merck Drive, 
> Whitehouse Station, New
> > Jersey, USA 08889), and/or its affiliates (which may be 
> known outside the
> > United States as Merck Frosst, Merck Sharp & Dohme or MSD 
> and in Japan as
> > Banyu) that may be confidential, proprietary copyrighted 
> and/or legally
> > privileged. It is intended solely for the use of the 
> individual or entity
> > named on this message.  If you are not the intended 
> recipient, and have
> > received this message in error, please notify us 
> immediately by reply e-mail
> > and then delete it from your system.
> > 
> --------------------------------------------------------------
> ----------------
> >
> 
>