[R] ERROR : cannot allocate vector of size (in MB & GB)

[Henrik Singmann]

However, this wouldn't help much with Win XP, as this only allows for 2GB (maximum of 3 GB):
http://cran.r-project.org/bin/windows/base/rw-FAQ.html#There-seems-to-be-a-limit-on-the-memory-it-uses_0021

If you want to use more RAM with windows you need to use a 64bit Version.

Cheers,
Henrik

Am 24.07.2012 19:59, schrieb Sarah Goslee:
> Sure, get more RAM. 2GB is a tiny amount if you need to load files of
> 1GB into R, and as you've discovered won't work.
>
> You can try a few simpler things, like making sure there's nothing
> loaded into R except what you absolutely need.
>
> It looks like there's no reason to read the entire file into R at once
> for what you want to do, so you could also load a chunk, process that,
> then move onto the next one.
>
> Sarah
>
> On Tue, Jul 24, 2012 at 9:45 AM, Rantony <antony.akkara at ge.com> wrote:
>> Hi,
>>
>> Here in R, I need to load a huge file(.csv) , its size is 200MB. [may come
>> more than 1GB sometimes].
>> When i tried to load into a variable it taking too much of time and after
>> that when i do cbind by groups,
>> getting an error like this
>>
>> " Error: cannot allocate vector of size 82.4 Mb"
>>
>> My requirement is, spilt data from Huge-size-file(.csv) to no. of small csv
>> files.
>> Here i will give no of lines to be 'split by' as input.
>>
>> Below i give my code
>> -------------------------------
>>                  SplitLargeCSVToMany <- function(DataMatrix,Destination,NoOfLineToGroup)
>>                  {
>>                          test <- data.frame(read.csv(DataMatrix))
>>
>>                          # create groups No.of rows
>>                          group <- rep(1:NROW(test), each=NoOfLineToGroup)
>>                          new.test <- cbind(test, group=group)
>>                          new.test2 <- new.test
>>                          new.test2[,ncol(new.test2)] <- NULL
>>
>>                          # now get indices to write out
>>                          indices <- split(seq(nrow(test)), new.test[, 'group'])
>>
>>                          # now write out the files
>>                          for (i in names(indices))
>>                          {
>>                          write.csv(new.test2[indices[[i]],], file=paste(Destination,"data.", i,
>> ".csv", sep=""),row.names=FALSE)
>>                          }
>>                  }
>>
>> -----------------------------------------------------
>> My system Configuration is,
>> Intel Core2 Duo
>> speed : 3GHz
>> 2 GB RAM
>> OS: Windows-XP [ServicePack-3]
>> ---------------------------------------------------
>>
>> Any hope to solve this issue ?
>>
>> Thanks in advance,
>> Antony.
>>
>>
>>
>>
>> --
>

-- 
Dipl. Psych. Henrik Singmann
PhD Student
Albert-Ludwigs-Universität Freiburg, Germany
http://www.psychologie.uni-freiburg.de/Members/singmann