[Rd] Memory "leak" in readChar (PR#1483)

ripley@stats.ox.ac.uk ripley@stats.ox.ac.uk
Fri, 26 Apr 2002 11:47:04 +0200 (MET DST)


It's not really a memory leak, more that R_alloc was at the time
incorrectly documented and do_readchar needs to reset vmax.  (Rstreams
used .C, and that did reset vmax, whereas .Internal does not.)

It's actually being done rather inefficiently, as this seems an unusual
case.  For 1.6.0 I am planning on using alloca (where available) for this
sort of short-term buffer.

I'll put the two-line fix in for 1.5.0.


On Fri, 26 Apr 2002 hcp@met.ed.ac.uk wrote:

> Full_Name: Hugh C. Pumphrey
> Version: 1.4.1
> OS: Linux (Debian Woody)
> Submission from: (NULL) (129.215.133.170)
>
>
> The function readChar() appears to have some type of problem with memory
> allocation.
> I don't know if "memory leak" is the correct term but if one uses readChar()
> many
> times, the R binary grows in size until it eats all your memory and swap space.
>
> The code enclosed below demonstrates the problem. As-is, it causes the size
> of R.bin to grow from 11MB to 50MB, even though there are no large arrays.
> The old Rstreams package did not do this. Also, the function readBin() does
> not suffer from this problem.
>
> ## Stress read/write of files in binary mode to see if there is a memory leak
> ## Now, readBin seems to be innocent. The memory leak is in readChar
> header <-paste("This is a very long and boring text header which appears",
>                "at the beginning of each chunk of an even longer and duller",
>                "file which contains mostly binary data")
> nchars <- nchar(header)
> nrecs <-2000
> ntries <- 100
>
> ## Write a test file
> stream <- file("/tmp/gunge")
> open(stream,open="wb")
> for(irec in 1:nrecs){
>   ## This writes null-ended strings unless you use the eos=NULL option
>   writeChar(header,stream,eos=NULL)
> }
> close(stream)
>
> ## Read in the file ntries times. In real applications one would be reading
> ## ntries _different_ files and calculating some summary statistics
> for(itry in 1:ntries){
>   stream <- file("/tmp/gunge")
>   open(stream,open="rb")
>   if(itry %% 10== 0) print(itry)
>   for(irec in 1:nrecs){
>     iheader <- readChar(stream,nchars)
>   }
>   close(stream)
> }
>
>
>
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._