[R] Running out of memory when importing SPSS files

[Uwe Ligges]

dobomode wrote:
> Hello R-help,
> 
> I am trying to import a large dataset from SPSS into R. The SPSS file
> is in .SAV format and is about 1GB in size. I use read.spss to import
> the file and get an error saying that I have run out of memory. I am
> on a MAC OS X 10.5 system with 4GB of RAM. Monitoring the R process
> tells me that R runs out of memory when reaching about 3GB of RAM so I
> suppose the remaining 1GB is used up by the OS.
> 
> Why would a 1GB SPSS file take up more than 3GB of memory in R? 

Because SPSS stores data in a compressed way?

 > Is it
> perhaps because R is converting each SPSS column to a less memory-
> efficient data type? In general, what is the best strategy to load
> large datasets in R?

Use a 64-bit version of R and have sufficient amount of RAM in your system.

Uwe Ligges

> Thanks!
> 
> P.S.
> 
> I exported the SPSS .SAV file to .CSV and tried importing the comma
> delimited file. Same results – the import was much slower but
> eventually I ran out of memory again...
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.