[Rd] parse_Rd and/or lazyload problem

[luke at stat.uiowa.edu]

On Thu, 5 Nov 2009, Mark.Bravington at csiro.au wrote:

> Great-- thanks for the info.
>
> For now, hopefully I can get the behaviour I want by sticking a .Call( 'R_lazyLoadDBflush'...) [as per 'detach'] before calling 'lazyLoad'. Seems to work on my examples, but please let me know if you don't think it'll work generally-- if not, I could presumably create the files under different names and then change them.
>
> Is there merit in making 'makeLazyLoadDB' public, just as 'lazyLoad' already is? It's useful.

The internals of the lazy load mechanism are intended to be private.
This is now emphasized in the lazyLoad help page.  This is under
active development, and we need the freedom to be able to change these
internals as needed.  They may change without notice, so using them
directly in user code or packages is not a good idea and is strongly
discouraged.

luke

>
> Mark
>
> --
> Mark Bravington
> CSIRO Mathematical & Information Sciences
> Marine Laboratory
> Castray Esplanade
> Hobart 7001
> TAS
>
> ph (+61) 3 6232 5118
> fax (+61) 3 6232 5012
> mob (+61) 438 315 623
>
> luke at stat.uiowa.edu wrote:
>> Here is a more stripped down variant generates and error on OS X for
>> me:
>>
>>      mkEg <- function(tm) list(scrunge = as.POSIXct(tm))
>>
>>      extract <- function(db) {
>>  	e<- new.env()
>>  	lazyLoad(db, e)
>>  	as.list( e)
>>      }
>>
>>      eg <- mkEg("2009-11-04 12:49:53")
>>      eg1 <- mkEg("2009-11-04 12:49:28")
>>
>>      tools:::makeLazyLoadDB( eg, '/tmp/ll')
>>      extract('/tmp/ll') # force; OK
>>
>>      tools:::makeLazyLoadDB( eg1, '/tmp/ll')
>>      extract('/tmp/ll')
>>
>> Changing the second set of /tmp/ll makes the symptom go away.
>>
>> I believe this comes down to unintended use of the lazyload mechanism
>> -- in particular it is not intended that a database be rewritten
>> after it has been loaded.  There is a chaching mechanism for improved
>> performance on slow file systems, and I believe what is happening is
>> that the new indices are being used to look in the old chached data.
>> There might be some merit in having lazyLoad call R_lazyLoadDBflush.
>>
>> luke
>>
>> On Tue, 3 Nov 2009, Seth Falcon wrote:
>>
>>> Hi,
>>>
>>> On 11/3/09 6:51 PM, Mark.Bravington at csiro.au wrote:
>>>>
>>>> file.copy( 'd:/temp/Rdiff.Rd', 'd:/temp/scrunge.Rd') # Rdiff.Rd
>>>> from 'tools' package source
>>>>
>>>> eglist<- list( scrunge=parse_Rd(  'd:/temp/scrunge.Rd'))
>>>> tools:::makeLazyLoadDB( eglist, 'd:/temp/ll')
>>>> e<- new.env()
>>>> lazyLoad( 'd:/temp/ll', e)
>>>> as.list( e) # force; OK
>>>>
>>>> eglist1<- list( scrunge=parse_Rd(  'd:/temp/Rdiff.Rd'))
>>>> tools:::makeLazyLoadDB( eglist1, 'd:/temp/ll')
>>>> e<- new.env()
>>>> lazyLoad( 'd:/temp/ll', e)
>>>> as.list( e) # Splat
>>>>
>>>> It doesn't make any difference which file I process first; the error
>>>> comes the second time round.
>>>
>>> If I adjust this example in terms of paths and run on OS X, I get the
>>> following error on the second run:
>>>
>>>> as.list(e) # Splat
>>> Error in as.list.environment(e) : internal error -3 in R_decompress1
>>>
>>> I haven't looked further yet.
>>>
>>> + seth
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu