[R] Regression with sparse matricies
Frank E Harrell Jr
f.harrell at Vanderbilt.Edu
Sat May 22 23:59:42 CEST 2010
On 05/22/2010 02:19 PM, Robin Jeffries wrote:
> I would like to run a logistic regression on some factor variables (main
> effects and eventually an interaction) that are very sparse. I have a
> moderately large dataset, ~100k observations with 1500 factor levels for one
> variable (x1) and 600 for another (X2), creating ~19000 levels for the
> interaction (X1:X2).
> I would like to take advantage of the sparseness in these factors to avoid
> using GLM. Actually glm is not an option given the size of the design
> I have looked through the Matrix package as well as other packages without
> much help.
> Is there some option, some modification of glm, some way that it will
> recognize a sparse matrix and avoid large matrix inversions?
It is doubtful that fixed effects are appropriate for your situation,
but if you do want to use them there is experimental code in the lrm
function in the rms package to handle "strat" (strata) factors that
makes use of the sparse matrix representation. Not sure if it handles
more than one factor, and you'll have to play with the code to make sure
this method is activated. Take a look at lrm.fit.strat.s that comes
with the source package, the see what is needed in lrm to use it.
Frank E Harrell Jr Professor and Chairman School of Medicine
Department of Biostatistics Vanderbilt University
More information about the R-help