[R] Memory question
Prof Brian Ripley
ripley at stats.ox.ac.uk
Thu Jul 14 13:29:25 CEST 2005
On Thu, 14 Jul 2005, Kenneth Cabrera wrote:
> Thank you Dr. Spencer Graves for your answer.
>
> What kind of matrices? They come form an image of about 3000x5000, and
> I need to generate arround 1024 matrices of the same size, they are not
> sparse
> matrices.
>
> What function can I use to, once generated one matrix, I could save into disk
> and then use the same space for the following, and so on.
You can use either save or .saveRDS/serialize followed by rm() and gc().
You cannot use the same space, but you can free up the space.
Then when you need the data again, load/.readRDS/unserialize can pull the
object back. (If you arrange this right the object will only go into a
temporary frame and so only be needed one at a time.)
>
> Thank you very much for your help
>
> Kenneth
>
> Spencer Graves wrote:
>
>> What kinds of matrices? There are facilities in the Matrix and
>> SparseM packages that might help for sparse matrices. If they are N x k
>> where N is large and k is not, can you compute something like the QR
>> decomposition and get away with keeping only the R part for most of your
>> matrices?
>>
>> One could potentially define a class of matrices that are only kept
>> in memory only when needed; I think S-Plus may do that. It would take a
>> lot of work to make that work generally, but you might be able to
>> accomplish what you need with a much smaller effort.
>>
>> spencer graves
>>
>> Kenneth Roy Cabrera Torres wrote:
>>
>>
>>> Hi R users and developers:
>>>
>>> I want to know how can I save memory in R
>>> for example:
>>> - saving on disk a matrix.
>>> - using again the matrix (changing their values)
>>> - saving again the matrix on disk in a different file.
>>>
>>> The idea is that I have a process that generate several
>>> matrices, but if I keep them all in memory it will overflow.
>>>
>>> How can I save them in different files, so I use the same
>>> amount of memory for each processed matrix?
>>>
>>> Thank you for your help.
>>>
>>>
>>
>>
>
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
More information about the R-help
mailing list