[R] memory error with 64-bit R in linux
lzhtom at hotmail.com
Thu Jul 19 05:49:55 CEST 2007
Thanks for replying!
i don't think i'm paging. i tried to use a smaller version of my matrix and
do all the checkings as suggested by jim. The smaller matrix caused another
problem, for which I've opened another thread. But i've found something
about memory that I don't understand.
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 269577 14.4 5570995 297.6 8919855 476.4
Vcells 3353395 25.6 9493567 72.5 15666095 119.6
Does this mean the maximum memory I can use for variables is only 120 M?
However, when I tried to check the memory limits:
Here it seems the maximum memory is not limited?
When there is no R function is being executed, I checked the system process
PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
7821 0.0 0.1 10048 2336 pts/0 Ss Jul18 0:00 -bash
8076 2.9 24.5 523088 504004 pts/0 S+ Jul18 2:46 /usr/lib64/R/bi
8918 1.5 0.1 9912 2328 pts/1 Ss 00:44 0:00 -bash
8962 0.0 0.0 3808 868 pts/1 R+ 00:45 0:00 ps u
Does this mean R is using 25% of my memory? But my RAM is 2 GB and the
objects in R only occupy 40 MB from gc().
Did I interpret it wrong?
Thanks a lot!
>From: "jim holtman" <jholtman at gmail.com>
>To: "zhihua li" <lzhtom at hotmail.com>
>CC: r-help at stat.math.ethz.ch
>Subject: Re: [R] memory error with 64-bit R in linux
>Date: Wed, 18 Jul 2007 17:50:31 -0500
>Are you paging? That might explain the long run times. How much
>are your other objects taking up? The matrix by itself should only
>require about 13MB if it is numeric. I would guess it is some of
>other objects that you have in your working space. Put some gc() in
>your loop to see how much space is being used. Run it with a subset
>of the data and see how long it takes. This might give you an
>estimate of the time, and space, that might be needed for the entire
>Do a 'ps' to see how much memory your process is using. Do one
>couple of minutes to see if it is growing. You can alway use
>to get an idea of where time is being spent (use it on a small
>On 7/18/07, zhihua li <lzhtom at hotmail.com> wrote:
>>I'm using the 64-bit R-2.5.0 on a x86-64 cpu, with an RAM of 2 GB.
>>operating system is SUSE 10.
>>The system information is:
>>Linux someone 2.6.13-15.15-smp #1 SMP Mon Feb 26 14:11:33 UTC 2007
>>x86_64 x86_64 GNU/Linux
>>I used heatmap to process a matrix of the dim [16000,100]. After 3
>>of desperating waiting, R told me:
>>cannot allocate vector of size 896 MB.
>>I know the matrix is very big, but since I have 2 GB of RAM and in
>>system, there should be no problem to deal with a vector smaller
>>than 1 GB?
>>(I was not running any other applications in my system)
>>Does anyone know what's going on? Is there a hardware limit where
>>to add more RAM, or is there some way to resolve it softwarely?
>>Also is it
>>possible to speed up the computing (I don't wanna wait another 3
>>know I get another error message)
>>Thank you in advance!
>>ÏíÓÃÊÀ½çÉÏ×î´óµÄµç×ÓÓÊ¼þÏµÍ³¡ª MSN Hotmail¡£
>>R-help at stat.math.ethz.ch mailing list
>>PLEASE do read the posting guide
>>and provide commented, minimal, self-contained, reproducible code.
>+1 513 646 9390
>What is the problem you are trying to solve?
More information about the R-help