[R] Using very large matrix

[Jay Emerson]

Corrado,

Package bigmemory has undergone a major re-engineering and will be available
soon (available now in Beta version upon request).  The version
currently on CRAN
is probably of limited use unless you're in Linux.

bigmemory may be useful to you for data management, at the very least, where

x <- filebacked.big.matrix(80000, 80000, init=n, type="double")

would accomplish what you want using filebacking (disk space) to hold
the object.
But even this requires 64-bit R (Linux or Mac, or perhaps a Beta
version of Windows 64-bit
R that REvolution Computing is working on).

Subsequent operations (e.g. extraction of a small portion for analysis) are then
easy enough:

y <- x[1,]

would give you the first row of x as an object y in R.  Note that x is
not itself an R matrix,
and most existing R analytics can't work on x directly (and would max
out the RAM if they
tried, anyway).

Feel free to email me for more information (and this invitation
applies to anyone who is
interested in this).

Cheers,

Jay

#Dear friends,
#
#I have to use a very large matrix. Something of the sort of
#matrix(80000,80000,n) .... where n is something numeric of the sort 0.xxxxxx
#
#I have not found a way of doing it. I keep getting the error
#
#Error in matrix(nrow = 80000, ncol = 80000, 0.2) : too many elements specified
#
#Any suggestions? I have searched the mailing list, but to no avail.
#
#Best,
#--
#Corrado Topi
#
#Global Climate Change & Biodiversity Indicators
#Area 18,Department of Biology
#University of York, York, YO10 5YW, UK
#Phone: + 44 (0) 1904 328645, E-mail: ct529 at york.ac.uk



-- 
John W. Emerson (Jay)
Assistant Professor of Statistics
Department of Statistics
Yale University
http://www.stat.yale.edu/~jay