[Rd] Should package version requirements assume installation from sources?

[Duncan Murdoch]

No, it looks as though this is an unrelated bit of caching being done by 
rstan.  During startup, it saves some ggplot defaults in an internal 
environment ".rstanvis_defaults"; that captured the ggplot2 3.3.6 
defaults, which caused problems when being run under ggplot2 3.4.0.

Duncan Murdoch

On 23/09/2022 5:54 a.m., Duncan Murdoch wrote:
> This issue may be the culprit in an obscure bug that's been reported on
> R-package-devel:  see
> https://stat.ethz.ch/pipermail/r-package-devel/2022q3/008481.html.  It
> appears that some ggplot2 version 3.4.0 code is being run even though
> 3.3.6 is the version on CRAN, and 3.3.6 is *also* being run in the same
> check session.
> 
> Duncan Murdoch
> 
> On 14/09/2022 6:04 a.m., Duncan Murdoch wrote:
>> On 13/09/2022 5:45 p.m., Mikael Jagan wrote:
>>> [Arguably also appropriate for R-package-devel, but posted to R-devel
>>>         as the discussion is aimed primarily at "experts" ... ]
>>>
>>> We, the authors of Matrix, have encountered a somewhat subtle issue
>>> induced by caching of S4 classes and methods in package namespaces.
>>>
>>> The namespaces of three reverse dependent packages (SeuratObject, conText,
>>> mcmcsae) cache the formal definition of our virtual class Matrix (and some
>>> subclasses).  For example:
>>>
>>>     > ns <- asNamespace("SeuratObject")
>>>     > grep("^[.]__C__.*Matrix$", names(ns), value = TRUE)
>>> [1] ".__C__dMatrix"       ".__C__compMatrix"    ".__C__AnyMatrix"
>>> [4] ".__C__generalMatrix" ".__C__CsparseMatrix" ".__C__sparseMatrix"
>>> [7] ".__C__dsparseMatrix" ".__C__Matrix"
>>>
>>> The cached definition (which includes a _validity method_) is obtained from
>>> the version of Matrix available when the reverse dependent package was built
>>> from sources.  For example, if SeuratObject was built under Matrix 1.4-1,
>>> then we get:
>>>
>>>     > getValidity(ns$.__C__Matrix)
>>> function (object)
>>> {
>>>         if (!isTRUE(r <- .Call(Dim_validate, object, "Matrix")))
>>>             r
>>>         else .Call(dimNames_validate, object)
>>> }
>>> <bytecode: 0x11e7ca508>
>>> <environment: namespace:Matrix>
>>>
>>> whereas if SeuratObject was built under Matrix >= 1.5-0, then we get:
>>>
>>>     > getValidity(ns$.__C__Matrix)
>>> function (object)
>>> .Call(Matrix_validate, object)
>>> <bytecode: 0x107dc1698>
>>> <environment: namespace:Matrix>
>>>
>>> There are two "questions" here:
>>>
>>> 1.  The symbol 'Matrix_validate' is not defined until Matrix 1.5-0.
>>>         Is it necessary, for this reason alone, for SeuratObject to have
>>>         'Imports: Matrix (>= 1.5-0)'?  Or can SeuratObject continue using
>>>         'Imports: Matrix (>= 1.3-3)', at the risk of errors like
>>>
>>>         > Error: object 'Matrix_validate' not found
>>>
>>>         (as already seen here: https://stackoverflow.com/questions/73700130)?
>>>
>>>         Note that this error would not occur for anyone installing SeuratObject
>>>         from sources, unless they decide to _downgrade_ Matrix after doing so.
>>>         Hence this primarily concerns Windows and macOS where R users would
>>>         typically install a binary built by CRAN (i.e., not on their system).
>>>
>>>         We are aware that package TMB tests in .onLoad() that the current Matrix
>>>         version is equal to or greater than the version available at build time,
>>>         thus avoiding a "strict" version requirement, but do not want this practice
>>>         to spread ...
>>>
>>> 2.  For how long should Matrix retain the superceded 'Dim_validate' and
>>>         'dimNames_validate', in order to ensure that "stale" cached validity
>>>         methods continue to work?
>>>
>>> We hope that this discussion will highlight the potential ramifications
>>> of importing classes and methods from other packages, and having one's
>>> classes and methods imported _by_ other packages, especially for version
>>> requirements.
>>
>> This sounds like a bug or bad design in the S4 system, i.e. caching
>> things without a way to detect or update when the cache becomes stale.
>>
>> Is it really necessary to cache things as part of the binary package, or
>> could they be put in place when needed using lazy loading, getting a
>> copy from the loaded copy of Matrix?
>>
>> Duncan Murdoch
>