[R] Big data (over 2GB) and lmer

[Michal Figurski]

Dear R-helpers

I have a data set of roughly 10 million records, 7 columns. It has only 
about 500MB as a csv, so it fits in the memory. It's painfully slow to 
do anything with it, but it's possible. I also have another dataset of 
covariates that I would like to explore - with about 4GB of data...

I would like to merge the two datasets and use lmer to build a mixed 
effects model. Is there a way, for example using 'bigmemory' or 'ff', or 
any other trick, to enable lmer to work on this data set?

I will greatly appreciate any suggestions.

-- 
Michal J. Figurski, PhD
HUP, Pathology & Laboratory Medicine
Biomarker Research Laboratory
3400 Spruce St. 7 Maloney S
Philadelphia, PA 19104
tel. (215) 662-3413