[Rd] A question on R memory management in .Fortran() calls under Windows

[Duncan Murdoch]

On 9/9/2005 1:04 PM, Simone Giannerini wrote:
> Dear R community,
> 
> I have a question on how R manages memory allocation in .Fortran()
> calls under Windows.
> In brief, apparently, it is not possible to allocate large matrices
> inside a Fortran subroutine
>   unless you pass them as arguments. If you do not act in this way
> RGUI crashes with a stack overflow error and acting  on memory through
> vsize nsize ppsize and memory.limit does not help at all.

It looks as though your Fortran compiler is allocating the new matrix on 
the stack.  R doesn't give you a huge stack, and that's causing the 
overflow.  When you get R to do the allocation, it does it on the heap, 
which has no artificial limits.  Only a pointer to the object ends up on 
the stack.

I'd say your only reasonable workarounds are to tell your compiler to 
use the heap for the local matrix allocation (if that's possible), or do 
your allocations in R.

There might be some utility somewhere which can modify Rgui.exe to tell 
it to start with a larger stack, and there is probably some linker 
option to put one in place, but it's not really reasonable to increase 
R's stack, because in almost all other situations, writing that much 
data to the stack is a sign of an infinite recursion.

Duncan Murdoch

> 
> ****************************************************************************************
> Details of the configurations on which I performed testing follow:
> R 2.1.1 on WinXP Pro SP2 ITA 
> PC1: AMD 64 3700+ 1GB RAM 
> PC2: AMD AthlonXP 2400+ 512Mb RAM
> Compaq Visual Fortran pro 6.6C
> ****************************************************************************************
> 
> To give an idea I attach a brief example on how to reproduce this:
> Create two simple subroutines 'foo' and 'foobis' that, say, give the
> sum of the elements of a matrix:
> 
> *** file foo.f90 starts
> ***********************************************************************
> SUBROUTINE foo(X,M,N,S)
> !DEC$ ATTRIBUTES DLLEXPORT,C,REFERENCE,ALIAS:'foo_' :: FOO
> 
>     IMPLICIT NONE
>     integer:: M,N
>     real*8:: X(M,N),S
> 
>     S  = sum(X);
> 
> END SUBROUTINE foo
> 
> SUBROUTINE foobis(M,N,S)
> !DEC$ ATTRIBUTES DLLEXPORT,C,REFERENCE,ALIAS:'foobis_' :: FOOBIS
> 
>     IMPLICIT NONE
>     integer:: M,N
>     real*8:: X(M,N),S
>     X = 1;
>     S  = sum(X);
> 
> END SUBROUTINE foobis
> 
> *** file foo.f90 ends
> ***********************************************************************
> 
> Notice that the matrix X is an input argument in foo and an internal
> variable in foobis.
> After compiling and linking turn to R:
> **************************************************************************
>> dyn.load("foo.dll");
>> is.loaded(symbol.For("foo"));
> [1] TRUE
>> is.loaded(symbol.For("foobis"));
> [1] TRUE
>> M <- 10;
>> N <- 10;
>> X <- matrix(1,M,N);
>> .Fortran("foo",X,as.integer(M),as.integer(N),S=as.double(0))$S;
> [1] 100 
>> .Fortran("foobis",as.integer(M),as.integer(N),S=as.double(0))$S;
> [1] 100
> 
> ## no problem here with small matrices, let's increase the size
> 
>> M <- 3000;
>> N <- 100;
>> X <- matrix(1,M,N);
>> .Fortran("foo",X,as.integer(M),as.integer(N),S=as.double(0))$S;
> [1] 3e+05  ## OK
> 
> 
> .Fortran("foobis",as.integer(M),as.integer(N),S=as.double(0))$S;
> ##  *** R GUI CRASHES WITH A STACK OVERFLOW ERROR ***
> ****************************************************************************
> Any suggestion would be greatly appreciated, I apologize if this
> problem has already been addressed previously,  I did not notice it.
>  Many thanks in advance,
>  
>  Simone
> ______________________________________________________ 
> 
> Simone Giannerini
> Dipartimento di Scienze Statistiche "Paolo Fortunati"
> Universita' di Bologna
> Via delle belle arti 41 - 40126  Bologna,  ITALY
> Tel: +39 051 2098248  Fax: +39 051 232153
> E-mail: giannerini at stat.unibo.it
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel