[R] mmap error on macbook pro

[David Winsemius]

On Mar 17, 2010, at 4:57 PM, Steve Lianoglou wrote:

> Hi,
>
> On Wed, Mar 17, 2010 at 8:14 AM, Shyamasree Saha [shs]  
> <shs at aber.ac.uk> wrote:
>> Dear List,
>>
>> I am trying to read some files using read.csv and total size of  
>> those files are 3.99 GB. I am using MacBook Pro with 4GB RAM(snow  
>> leopard). I also tried to run a chunk from those files and  
>> altogether the size was 1.33 GB. But every time I was getting the  
>> following error
>>
>> R(1200) malloc: *** mmap(size=16777216) failed (error code=12)
>> *** error: can't allocate region
>> *** set a breakpoint in malloc_error_break to debug
>>
>> I managed to run smaller chunks (400MB) and I also saved the  
>> object. But problem is when I try to load some of those  
>> objects(which is not more than 1.5 GB altogether), I get same error  
>> again. Can someone please help? why am I getting this error? does R  
>> need more space than the actual file size? I may buy new machine if  
>> its something related to RAM size. but if it is some R problem then  
>> it will be very useless to buy new machine. So, I really need to  
>> know why this is happening.
>
> Most like it's a "RAM problem," as you say.
>
> I believe read.csv has better RAM characteristics if you set its
> "colClasses" parameter ... so if you know that some columns in your
> data are numeric, instead of strings, then tell it.
>
> If I'm not mistaken, read.csv will assume everything is a string
> first, and store it like that.

read.csv is just a wrapper for read.table() which is in turn a wrapper  
for scan(). You may be thinking of readLines() which does turn each  
line into a string.

 > read.csv
function (file, header = TRUE, sep = ",", quote = "\"", dec = ".",
     fill = TRUE, comment.char = "", ...)
read.table(file = file, header = header, sep = sep, quote = quote,
     dec = dec, fill = fill, comment.char = comment.char, ...)
<environment: namespace:utils>

-- 

David Winsemius, MD
West Hartford, CT