[R] Increasing Max Vector Length

Prof Brian D Ripley ripley at stats.ox.ac.uk
Fri Jun 14 12:18:18 CEST 2002


On Wed, 12 Jun 2002, Jeremy Fox wrote:

> I'm looking at doing some really large linear algebra problems. I
> followed the helpful instructions in the back of the installation
> guide for rather aggressive compilation options on 64-bit
> Solaris. Happily, I can address more than 6GB of RAM at once in R, so
> I the 64 bit part works fine. With the Sun Performance libraries, R is
> also very fast.
>
> One thing that still bugs me is the maximum vector length, which
> apparently is 268435455. Can I increase this? 32 bit SPlus has the
> same limitations, but Fortran, C and 32 bit Matlab do not, so it's not

Any 32-bit machine can at most address 2^32bytes (by definition)
and that is 536870912 doubles.  It's pretty unlikely that one could
allocate all of that to one object, and it's only a factor of two.
So on 32-bit platforms it is not much of a restriction.

We are in the process of altering R internally to use unsigned longs and
not ints for quantities holding memory sizes, and this will help on 64-bit
machines, and slightly on 32-bit machines.  However, vector lengths will
remain limited to 2^31 - 1, as vectors are indexed by ints.

Almost all of R's management of the heap is done in units of 8 bytes,
and size (as used in allocVector) is a long.  There is a line

	    if (length > INT_MAX / sizeof(double))
		errorcall(R_GlobalContext->call,
			  "cannot allocate vector of length %d", length);

in src/main/memory.c that provides the limit (assuming you are quoting
numeric vectors), and this is one of the things which will get altered
shortly (to ULONG_MAX/sizeof(double), so unlimited by this on a 64-bit
machine).  I think even now you could change those limits to LONG_MAX on a
64-bit machine as size is a long.

R-devel will have the changes in a week or two.  (I've tried them once, and
backed them out until we are finished with 1.5.1.)

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._



More information about the R-help mailing list