[R] memory management

[Roger Bivand]

On Fri, 10 Nov 2000 Pan_Yuming at aam.de wrote:

> read.table() is slower than scan(), but i wont use scan() if
> read.table() can do it satifactoriely.  Kjetil gave me the right
> direction. i tend to use
>  a lot of loops in the program and that s not efficient.
> 
I have a feeling that there is an underlying issue concerning the
treatment of character strings in read.table(), both for factors and as
row names. A lot of con cells seem to be used up - that's where I've
typically hit memory limits on reading largish files. If you can
manage with scan(), and convert your character vectors to numeric (for
later conversion to factor) before you read the file, you can use more
memory for heap and less for con cells. If you are really stuck, then for
special files, like images, it's best to write a small C function to suck
in the data - this is much less challenging than it might seem, and gives
you a chance to see how elegant R really is under the hood!

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no
and: Department of Geography and Regional Development, University of
Gdansk, al. Mar. J. Pilsudskiego 46, PL-81 378 Gdynia, Poland.

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._