[R] Computing very large distance matrix
Roger D. Peng
rpeng at jhsph.edu
Thu Feb 26 02:28:24 CET 2004
The problem is not with R; the problem is with Windows, as well
as the x86 architecture. You'll never really be able to access
more than 3GB of physical memory (for a single process).
Accessing more than 4GB requires a 64 bit processor.
-roger
Arnav Sheth wrote:
> Hello,
>
> Thank you all for your replies.
>
> I have just discovered that now, for some reason, R does not let me increase
> its memory limit beyond 4095 Mb.
>
> Also, I get a different error message when computing the matrix using
> daisy() as opposed to dist():
>
> Error: Cannot allocate vector of size 135168 Kb
>
> Does this mean that there is no way in which I can compute this distance
> matrix on the machine I described below?
>
> Is there any possible solution to this problem?
>
> Thanks again to all,
> Arnav.
>
>
>
> ----- Original Message -----
> From: "Prof Brian Ripley" <ripley at stats.ox.ac.uk>
> To: "Arnav Sheth" <sheth at economics.rutgers.edu>
> Cc: "R-Help" <R-help at stat.math.ethz.ch>
> Sent: Wednesday, February 25, 2004 2:12 PM
> Subject: Re: [R] Computing very large distance matrix
>
>
>
>>On Wed, 25 Feb 2004, Arnav Sheth wrote:
>>
>>
>>>Hello All,
>>>
>>>I have a 131072x132 matrix for which I need to compute a regular
>
> euclidean distance matrix, which I then need to transform and run agnes() on
> this transformed matrix. I am having trouble computing the distance matrix
> as it is fairly large and I am sure I have gone over the max.
>
>>>The specific error I am getting is:
>>>Error in vector("double", length) : negative length vectors are not
>
> allowed
>
>>We've improved that message for the next release.
>>
>>
>>>I have increased the memory limit to the maximum capacity of my hard
>>>drive (which is around 20gb), with no success.
>>
>>You'd need to increase the 32-bit limit of Windows, too ....
>>
>>
>>>I am running the RGUI on Windows XP with 512 mb of RAM.
>>>
>>>Would anyone have any suggestions as to how I can overcome this problem?
>>>I would be most grateful for any help.
>>
>>No. The distance matrix is 8Gb all by itself.
>>
>>--
>>Brian D. Ripley, ripley at stats.ox.ac.uk
>>Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
>>University of Oxford, Tel: +44 1865 272861 (self)
>>1 South Parks Road, +44 1865 272866 (PA)
>>Oxford OX1 3TG, UK Fax: +44 1865 272595
>>
>>
>
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>
More information about the R-help
mailing list