R-beta: Re: S Compatibility

Ross Ihaka ihaka at stat.auckland.ac.nz
Wed Apr 30 04:27:07 CEST 1997


Z. Todd Taylor writes:

> I have a question/comment on this topic.  If I understand what
> I've read, R's lexical scoping rules are what require that all
> of R's data be held in memory (as opposed to S's method of
> storing each object on disk and reading it as often as
> necessary).

> My question is whether there will ever be a way around having to
> store everything in memory.  I make heavy use of S's
> user-defined database mechanism to gain transparent access to
> some huge datasets.  I store tens of thousands of
> individually-compressed S objects that I can access
> transparently via my user-defined database.

> In S, I attach my user-defined database(s) and all those objects
> are available; but they don't use any memory until I actually
> use them.  Obviously, I rarely access all of them in one
> session, but usually access more than I could hold in memory
> at once.

> So, how could I do this in R?


You can't (yet).  Here is a possible approach I have been thinking
about (thinking not doing).  Lazy evaluation is implemented using
"promise-to-evaluate" objects.  We could use something quite similar
(perhaps identical) to implement what you want.  We will introduce
"promise-to-fetch" kind of object which contains information on how to
fetch an object and a slot which would point to the real object if is
currently in memory.  When the evaluator encounters such an object it
checks the value slot to see if the value is present.  If it is it
returns it, otherwise it uses the "fetch" recipe to first bring it into
memory and then it returns it.

When memory runs low, the real objects could be freed and the recipes
retained.  How this would work (timestamps?) and how it would work with
the garbage collector is an open question.

Because the objects are referenced through these "promise" gizmos,
scoping is no problem.

Its something we definitely need (almost as much as REAL libraries).
If only there were another 24 hours in each day ...
	Ross
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=



More information about the R-help mailing list