[R] Memory use in R

[Duncan Murdoch]

On 29/10/2010 12:46 PM, DM2010 wrote:
>
> Dear R Users
>
> I have two questions about how R makes use of memory on a Windows computer.
>
> On my machine certain R jobs seem to stop with messages such as...
> "Error: cannot allocate vector of size 215.0 Mb"
> ...when, according to Windows Task Manager, there are still hundreds of
> megabytes of physical memory available. These jobs usually stop when the
> peak commit charge is around 2.6GB (in Windows-speak the "commit charge" is
> all the memory allocated by system, drivers, applications etc.).
>
> Machine characteristics
>
> Processor: Pentium-D
> 4GB of installed memory (Task Manager reports 3584MB = 4GB - 512MB total
> memory)
> Windows booted with /3GB switch
> R started with --max-mem-size=3071M
> memory.limit() as reported by R: 3071
>
> My questions are:
> 1. When it starts, does R ask the system to allocate memory for it in a
> single chunk?

No, it can handle fragmented allocations.

> 2. Can R allocate memory either side of the system-preallocated memory in
> the middle of address space?

I think so.  However, when it asks for 215 Mb, it needs to get that 
allocation in a single block of contiguous addresses, and it won't move 
things around to make space.  When all you've got is 3 GB to play in and 
you've allocated 2.6 GB, there's a good chance that the remaining space 
is spread out in small pieces and no available unused address ranges are 
big enough.

If you run R in a 64 bit OS you'll find it handles this situation 
better, because the OS can assign the physical memory to whatever 
address it likes, and you'll have lots of big open ranges.  On the other 
hand, with bigger pointers everything takes a bit more space, so you 
might run out sooner.

Duncan Murdoch

> Best wishes
>
> David Max