[R] Increasing Max Vector Length
Prof Brian D Ripley
ripley at stats.ox.ac.uk
Fri Jun 14 12:18:18 CEST 2002
On Wed, 12 Jun 2002, Jeremy Fox wrote:
> I'm looking at doing some really large linear algebra problems. I
> followed the helpful instructions in the back of the installation
> guide for rather aggressive compilation options on 64-bit
> Solaris. Happily, I can address more than 6GB of RAM at once in R, so
> I the 64 bit part works fine. With the Sun Performance libraries, R is
> also very fast.
> One thing that still bugs me is the maximum vector length, which
> apparently is 268435455. Can I increase this? 32 bit SPlus has the
> same limitations, but Fortran, C and 32 bit Matlab do not, so it's not
Any 32-bit machine can at most address 2^32bytes (by definition)
and that is 536870912 doubles. It's pretty unlikely that one could
allocate all of that to one object, and it's only a factor of two.
So on 32-bit platforms it is not much of a restriction.
We are in the process of altering R internally to use unsigned longs and
not ints for quantities holding memory sizes, and this will help on 64-bit
machines, and slightly on 32-bit machines. However, vector lengths will
remain limited to 2^31 - 1, as vectors are indexed by ints.
Almost all of R's management of the heap is done in units of 8 bytes,
and size (as used in allocVector) is a long. There is a line
if (length > INT_MAX / sizeof(double))
"cannot allocate vector of length %d", length);
in src/main/memory.c that provides the limit (assuming you are quoting
numeric vectors), and this is one of the things which will get altered
shortly (to ULONG_MAX/sizeof(double), so unlimited by this on a 64-bit
machine). I think even now you could change those limits to LONG_MAX on a
64-bit machine as size is a long.
R-devel will have the changes in a week or two. (I've tried them once, and
backed them out until we are finished with 1.5.1.)
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272860 (secr)
Oxford OX1 3TG, UK Fax: +44 1865 272595
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
More information about the R-help