[R] Garbage Collecting

[Kjetil Kjernsmo]

Dear all,

Me and the Garbage Collector doesn't understand each other... 
I'm on R-1.0.1 (I have requested an upgrade to the right local people), an
osf1 system with an alphaev6 CPU. I have fair amounts of memory to use,
but it seems to be used up pretty fast nevertheless. 
I'm now trying to see what the garbage collector does, I have inserted a
gc(v=T)
call at a spot in my code.
I also discovered gctorture(), but I don't claim to be an R hacker for
that matter. :-)

I do a call 
> l4 <- t(apply(cbind(rep(1000000, 4)), 1, lineprofile, 1, 80, 
                smap01n$histogram))
and this will create a 101 * 4 element matrix. My first line in
'lineprofile' is gc(v=T), so it reports while running
Garbage collection [nr. 362602]...
629255 cons cells free (78%)
155163 Kbytes of heap free (76%)
Garbage collection [nr. 362603]...
627739 cons cells free (78%)
149224 Kbytes of heap free (73%)
Garbage collection [nr. 362604]...
626223 cons cells free (78%)
143286 Kbytes of heap free (70%)
Garbage collection [nr. 362605]...
624707 cons cells free (78%)
137347 Kbytes of heap free (67%)

So, it seems like close to 6 MB is used for time lineprofile is run. I
thought that the garbage collector would come and remove everything that
wasn't needed, so that the difference in free heap size would be
corresponding to the size of the array produced by lineprofile, that is,
202 bytes. And that when the call returns with the expected object, the gc
would free memory, so that what's in the memory would correspond to the
size of the objects, which again would correspond to what I have in my
.RData file. Currently, my .RData file (saved shortly before the previous
call) is slightly less than 450 kb, but 
> gc()
           free    total  (Mb)
Ncells   629305   800000  30.6
Vcells 19860721 26214400 200.0

Last night (before this call), I did a run with gctorture(T) but on a 
slightly more complex piece of my code (yes, it is _slow_), and since it
was started just after starting R, it starts with almost all the heap free
Garbage collection [nr. 2111]...
641457 cons cells free (80%)
203427 Kbytes of heap free (99%)
Garbage collection [nr. 43375]...
639941 cons cells free (79%)
197488 Kbytes of heap free (96%)
[...]
Garbage collection [nr. 303218]...
630828 cons cells free (78%)
161854 Kbytes of heap free (79%)
Garbage collection [nr. 356743]...
629295 cons cells free (78%)
155912 Kbytes of heap free (76%)

At this point, I halted it with BREAK, and did 
> gc()
           free    total  (Mb)
Ncells   629306   800000  30.6
Vcells 19860723 26214400 200.0

I would expect it to get rid of everything that went on in the last call,
but 19860723/26214400 is 76%, so that didn't happen. 

Now, this isn't too much to go on, I realize, but I want to hear if my
understanding of what the garbage collector is supposed to be doing is
totally wrong before I get deeper into this. If I'm not totally
wrong, and anybody cares to have a look, I have saved the objects that is
sufficient to reproduce the problem on my system in
<URL:http://www.astro.uio.no/~kjetikj/tmp/kjetil.RData>

I had a core dump (warning, 291MB), after killing R with wildly giving all
kinds of signals... (BREAK, QUIT, STOP, KILL) :-)
<URL:http://www.astro.uio.no/~kjetikj/tmp/Rcore1>, I don't know if it is
of much use. 

Yours Confusedly,

Kjetil
-- 
Kjetil Kjernsmo
Graduate astronomy-student                    Problems worthy of attack
University of Oslo, Norway            Prove their worth by hitting back
E-mail: kjetikj at astro.uio.no                                - Piet Hein
Homepage <URL:http://www.astro.uio.no/~kjetikj/>
Webmaster at skepsis.no 


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._