[R] Computing very large distance matrix
sheth at economics.rutgers.edu
Thu Feb 26 01:22:08 CET 2004
Thank you all for your replies.
I have just discovered that now, for some reason, R does not let me increase
its memory limit beyond 4095 Mb.
Also, I get a different error message when computing the matrix using
daisy() as opposed to dist():
Error: Cannot allocate vector of size 135168 Kb
Does this mean that there is no way in which I can compute this distance
matrix on the machine I described below?
Is there any possible solution to this problem?
Thanks again to all,
----- Original Message -----
From: "Prof Brian Ripley" <ripley at stats.ox.ac.uk>
To: "Arnav Sheth" <sheth at economics.rutgers.edu>
Cc: "R-Help" <R-help at stat.math.ethz.ch>
Sent: Wednesday, February 25, 2004 2:12 PM
Subject: Re: [R] Computing very large distance matrix
> On Wed, 25 Feb 2004, Arnav Sheth wrote:
> > Hello All,
> > I have a 131072x132 matrix for which I need to compute a regular
euclidean distance matrix, which I then need to transform and run agnes() on
this transformed matrix. I am having trouble computing the distance matrix
as it is fairly large and I am sure I have gone over the max.
> > The specific error I am getting is:
> > Error in vector("double", length) : negative length vectors are not
> We've improved that message for the next release.
> > I have increased the memory limit to the maximum capacity of my hard
> > drive (which is around 20gb), with no success.
> You'd need to increase the 32-bit limit of Windows, too ....
> > I am running the RGUI on Windows XP with 512 mb of RAM.
> > Would anyone have any suggestions as to how I can overcome this problem?
> > I would be most grateful for any help.
> No. The distance matrix is 8Gb all by itself.
> Brian D. Ripley, ripley at stats.ox.ac.uk
> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel: +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UK Fax: +44 1865 272595
More information about the R-help