[R] arrays in binary files
Timothy H. Keitt
tklistaddr at keittlab.bio.sunysb.edu
Wed Jul 10 21:52:21 CEST 2002
You need TPIE; see http://www.cs.duke.edu/TPIE/. I've long thought it
would be cool to write an R wrapper for this package.
On Wed, 2002-07-10 at 13:07, Duncan Murdoch wrote:
> >In the writing of functions for a package have to deal with (very)
> > large arrays and I was considering that
> >having them in files rather than in memory would help me a bit
> >on the memory side of the things. I was thinking of accessing the
> >elements in the arrays by 'seek' and friends in C.
> My gut feeling is that this is a bad thing to do. Unless you go to a
> lot of trouble to optimize, it's generally better to let the operating
> system create "virtual memory" and handle the seeking and such.
> The main case where this is not true is the case of large static
> arrays that change little from run to run, of which you only use a
> small proportion of the entries. Unless you can do memory mapped
> files, you'll have to read the whole file into virtual memory on each
> run, and that's slow. If you're accessing every element anyway, then
> it's not such a penalty to read the whole thing at the start; if they
> are temporary for the run, or change frequently, then you'll be
> writing the whole thing every time anyway.
> Duncan Murdoch
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
More information about the R-help