[Rd] parse_Rd and/or lazyload problem

[Mark.Bravington at csiro.au]

The errors are generated inside lazyLoadDBfetch; I don't call object.size, so unfortunately that's not the issue (unless o.s. is called somewhere inside the .Primitive for lazyLoadDBfetch).

> > I'm encountering problems when making lazy-loadable databases of the output from 'parse_Rd'. The lazy-
> > load database is of seemingly limitless size when I try to reload it... Admittedly I am using functions
> > that I'm not really supposed to use, which is why this isn't a bug report, but there does seem to be
> > something strange going on; my code is very similar to code that lives inside
> > 'tools:::.install_package_Rd_objects'. The problems occur with just-released R2.10.0 on Windows.
> 
> object.size() has problems when working on Rd objects, because it counts
> every environment separately, even though they may all be references to
> the same one. I haven't looked at your code, but that could be a problem.
> 
> 

Mark

________________________________________
From: Duncan Murdoch [murdoch at stats.uwo.ca]
Sent: 31 October 2009 08:59
To: Bravington, Mark (CMIS, Hobart)
Cc: r-devel at r-project.org
Subject: Re: [Rd] parse_Rd and/or lazyload problem

On 30/10/2009 12:50 AM, Mark.Bravington at csiro.au wrote:
> I'm encountering problems when making lazy-loadable databases of the output from 'parse_Rd'. The lazy-load database is of seemingly limitless size when I try to reload it... Admittedly I am using functions that I'm not really supposed to use, which is why this isn't a bug report, but there does seem to be something strange going on; my code is very similar to code that lives inside 'tools:::.install_package_Rd_objects'. The problems occur with just-released R2.10.0 on Windows.

object.size() has problems when working on Rd objects, because it counts
every environment separately, even though they may all be references to
the same one. I haven't looked at your code, but that could be a problem.

Duncan Murdoch

>
> The examples below use files which can be found at ftp://ftp.csiro.au/MarkBravington, but you'll obviously need to modify the paths. The file "scrunge.Rd" is just "Rdiff.Rd" from the 'tools' package. The file "fakepack.7z" should unzip to create a fake package with a DESCRIPTION file and a "man" directory that contains two Rd files.
>
> Transcript of first example:
> eglist <- list( scrunge=tools:::prepare_Rd(  'd:/temp/scrunge.Rd',
>     defines=.Platform$OS.type, stages='install', warningCalls=FALSE))
> tools:::makeLazyLoadDB( eglist, 'd:/temp/ll', compress=FALSE)
> tools:::fetchRdDB( 'd:/temp/ll')
> # Error: cannot allocate vector of size 1.4 Gb
>
> The second example triggers an error with 'tools:::.install_package_Rd_objects' itself. It doesn't happen the first time, but frequently does after the Rd files have been changed. Transcript:
>
> # Make sure d:/temp/help/ is empty, then
> test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
> test> tools:::fetchRdDB( 'd:/temp/help/temp')
> # All good. Next, I *removed* one of the two Rd files in "d:/temp/fakepack/man", ...
> # ...deleted "d:/temp/help/temp*", and tried again
> test> tools:::.install_package_Rd_objects( 'd:/temp/fakepack', 'd:/temp')
> test> tools:::fetchRdDB( 'd:/temp/help/temp')
> Warning: Reached total allocation of 1535Mb: see help(memory.size)
> Warning: Reached total allocation of 1535Mb: see help(memory.size)
> Warning: Reached total allocation of 1535Mb: see help(memory.size)
> Warning: Reached total allocation of 1535Mb: see help(memory.size)
> Error: cannot allocate vector of size 1.9 Gb
> # Or on other occasions I get
> Error: internal error -3 in R_decompress1
>
> Mark Bravington
> CSIRO CMIS
> Hobart
> Australia
>
> --please do not edit the information below--
> Version:
>  platform = i386-pc-mingw32
>  arch = i386
>  os = mingw32
>  system = i386, mingw32
>  status =
>  major = 2
>  minor = 10.0
>  year = 2009
>  month = 10
>  day = 26
>  svn rev = 50208
>  language = R
>  version.string = R version 2.10.0 (2009-10-26)
> Windows XP (build 2600) Service Pack 2
> Locale:
> LC_COLLATE=English_Australia.1252;LC_CTYPE=English_Australia.1252;LC_MONETARY=English_Australia.1252;LC_NUMERIC=C;LC_TIME=English_Australia.1252
> Search Path:
>  .GlobalEnv, ROOT, package:grDevices, package:ad, package:chstuff, package:handy2, package:tweedie, package:statmod, package:handy, package:debug, package:mvbutils, mvb.session.info, package:tools, package:tcltk, package:stats, package:graphics, package:utils, package:methods, Autoloads, package:base
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel