[R] "lean and mean" regression {was "Memory size"}

[Roger Koenker]

If you can get model.matrix to make an X matrix for you, then you could
try to use slm in SparseM to estimate the model.  Once again, I would
make a plea that it would be nice to have a version of model.matrix that
returned a sparse form of X, since there are bound to be problems for
which X itself creates memory problems even before one tries to hit
it with the QR hammer.


url:	www.econ.uiuc.edu/~roger/my.html	Roger Koenker
email	rkoenker at uiuc.edu			Department of Economics
vox: 	217-333-4558				University of Illinois
fax:   	217-244-6678				Champaign, IL 61820

On Mon, 14 Jul 2003, Martin Maechler wrote:

> >>>>> "AndyL" == Liaw, Andy <andy_liaw at merck.com>
> >>>>>     on Mon, 14 Jul 2003 09:33:31 -0400 writes:
>
>     AndyL> How *exactly* did you "run the regression" in R?
>     AndyL> There are several ways, and it can make a big
>     AndyL> difference for large data sets.  lm() would be the
>     AndyL> most expensive option.  If I'm not mistaken, lsfit()
>     AndyL> is more "lean and mean".
> as a matter of fact, rather use  lm.fit() which is the ``work horse''
> of lm().  lm.fit() and lsfit() are very similar (relying on the
> same Fortran QR decomposition, but lm.fit() has now been tested
> {by lm() usage} much more extensively.
>
>     AndyL> You can even do it more or less by hand, by calling
>     AndyL> qr() directly.  There's also a disussion in Venables
>     AndyL> & Ripley's "S Programming" on this subject (for Splus).
>
> Section 7.2, (actually the relevant code is not at all S-plus specific,
> 	      just the final "resources(.)" [CPU, Memory]
> 	      measuring of the solution.)
>
> It's for the case of one factor with many (107!)levels and continuous
> covariates otherwise. There, one can solve without constructing
> the large matrices that all of lm(), lsfit() or lm.fit() would use.
>
> It becomes really "interesting" if you have (several) factors
> with (many) levels...
>
> Regards,
> Martin
>
>     >> -----Original Message----- From: Silika Tereshchenko
>     >> [mailto:silika at access.unizh.ch] Sent: Sunday, July 13,
>     >> 2003 8:55 AM To: R-help at stat.math.ethz.ch Subject: [R]
>     >> Memory size
>     >>
>     >>
>     >>
>     >> Daer all,
>     >>
>     >> I have the problem. I could not run the regression,
>     >> because I have always the warning message
>     >> "memory.size". from the help file I learned that it is
>     >> possible to increase the memory size, but I did not
>     >> undestand how could I do it. Could you please explaine it
>     >> to me. I would be very grateful for it.
>     >>
>     >>
>     >> The second question: I obtained from the regression the
>     >> coefficient "6.003e-3" and "0.0345e+3". What daos it
>     >> mean?
>     >>
>     >>
>     >>
>     >> Thanks a lot, Silika
>     >>
>     >> ______________________________________________
>     >> R-help at stat.math.ethz.ch mailing list
>     >> https://www.stat.math.ethz.ch/mailman/listinfo> /r-help
>     >>
>
>     AndyL> ------------------------------------------------------------------------------
>     AndyL> Notice: This e-mail message, together with any
>     AndyL> attachments, ...{{dropped}}
>
>     AndyL> ______________________________________________
>     AndyL> R-help at stat.math.ethz.ch mailing list
>     AndyL> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>