[Rd] if(--as-cran)?
Martin Maechler
maechler at stat.math.ethz.ch
Wed Sep 5 09:48:04 CEST 2012
>>>>> Kasper Daniel Hansen <kasperdanielhansen at gmail.com>
>>>>> on Tue, 4 Sep 2012 17:41:38 -0400 writes:
> On Tue, Sep 4, 2012 at 5:32 PM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
>> On 04/09/2012 5:21 PM, John Fox wrote:
>>>
>>> Dear all,
>>>
>>> I'd like to second this fairly simple request. I currently enclosed some
>>> of
>>> the examples in the effects package in \donttest{} blocks to satisfy the
>>> CRAN timing requirements for examples. It would be nice to have something
>>> like a \donttestcran{} block that suppresses the tests when --as-cran is
>>> set
>>> (and on CRAN itself).
>>>
>>> I'm sure that I've missed many of the nuances in this discussion, but this
>>> seems like a simple solution to me.
>>
>>
>> That would work for examples, but not tests. Many packages have scripts
>> that are in the tests directory, not just test code in the .Rd files. What
>> I think you should use is my suggested HowMuchTimeLeftBeforeTimeout()
>> function, but if that's not writeable, then simply having a TIMELIMIT
>> environment variable or similar, so your code could see if the there's a
>> timeout pending.
> I don't like this idea, because then I need to think about the order
> in which the tests are run.
> You are right that we could simulate this by having an environment
> variable, and if that variable is set, we could do the full tests.
> Like Dirk is mentioning, the (sensible, I might add) requirement that
> the tests should be "quick" suddenly makes the quick testing the
> default.
> What I - and I think a lot of other people - would like, is to do this
> in a standard way so it would be uniform across packages. I don't
> think it would be good if I start using R_LONG_TESTS and Dirk starts
> to use __NON_CRAN_TESTS or whatever. Then all of us need to know the
> package-specific variables when we test other people packages (and
> that happens occasionally, for example when you have a package that
> other packages depend upon).
> In this use case, standardization would be hugely beneficial in my
> opinion. In my reading of the discussion, this is really what all of
> us are saying.
> Kasper
Yes!
... plus the idea to go one step further:
Instead of just two levels ( quick tests / extensive tests ),
one could use 3 or 4 (or 5) such levels, e.g.,
1: very quick
2: CRAN-ok-quick
3: package-developer-usual
4: extensive
*and* the CRAN maintainers would say which level is the one
that runs daily on CRAN and hence needs to meet Duncan's
TIMELIMIT.
So instead of just setting an environment variable to
non-zero length, or "true" (as in my example), one could set
that environment variable to an 1, 2, ...
Martin
>>> > -----Original Message-----
>>> > From: r-devel-bounces at r-project.org [mailto:r-devel-bounces at r-
>>> > project.org] On Behalf Of Kasper Daniel Hansen
>>> > Sent: Tuesday, September 04, 2012 5:12 PM
>>> > To: Warnes, Gregory
>>> > Cc: Terry Therneau; r-devel at r-project.org
>>> > Subject: Re: [Rd] if(--as-cran)?
>>> >
>>> > On Tue, Sep 4, 2012 at 4:53 PM, Warnes, Gregory
>>> > <gregory.warnes at novartis.com> wrote:
>>> > >
>>> > > On 9/4/12 3:58 PM, "Duncan Murdoch" <murdoch.duncan at gmail.com> wrote:
>>> > >
>>> > >>On 04/09/2012 3:44 PM, Terry Therneau wrote:
>>> > >>>ly in
>>> > >>> On 09/04/2012 01:57 PM, Duncan Murdoch wrote:
>>> > >>> > On 04/09/2012 2:36 PM, Warnes, Gregory wrote:
>>> > >>> >> On 9/4/12 8:38 AM, "Duncan Murdoch" <murdoch.duncan at gmail.com>
>>> > wrote:
>>> > >>> >>
>>> > >>> >>
>>> > >>> >> >On 04/09/2012 8:20 AM, Terry Therneau wrote:
>>> > >>> >> >>
>>> > >>> >> >> On 09/04/2012 05:00 AM, M
>>> > >>><mailto:r-devel-request at r-project.org>artin wrote:
>>> > >>> >> >> > The issue is not just about "CRAN" vs "off CRAN".
>>> > >>> >> >> > It is good to think about a more general scheme of
>>> > >>> >> >> > "light testing" vs "normal testing" vs "extensive testing",
>>> > >>> >> >> > e.g., for the situation where the package implements
>>> > >>> >> >> > (simulation/bootstrap/ ..) based inference, and the
>>> > developer
>>> > >>> >> >> > (but not only) should be able to run the extensive tests.
>>> > >>> >> >> >
>>> > >>> >> >> > Martin
>>> > >>> >> >>
>>> > >>> >> >> I agree with Martin. A mechanism to specify testing level
>>> > would
>>> > >>>be the
>>> > >>> >> >> best. Then CRAN can choose to set that variable to "3" say,
>>> > with
>>> > >>>level
>>> > >>> >> >> 1 for extensive and 2 for usual.
>>> > >>>>> >>
>>> > >>>
>>> > >>>[snip]
>>> > >
>>> > >>The testingLevel() function is supposed to be a way to know that a
>>> > >>certain level of testing is being done, to allow such tailoring.
>>> > >>However, I don't think it's practical. I think you can ask whether a
>>> > >>specific test is being run (my "D" %in% tests() example), but you
>>> > can't
>>> > >>reasonably convert the set of tests chosen by a tester into a single
>>> > >>number.
>>> > >>
>>> > >>What I think you and Greg are talking about is something different.
>>> > You
>>> > >>are asking that we set up more suites of tests, corresponding to
>>> > >>numerical levels. Currently we have two suites: the default, and the
>>> > >>--as-cran suite. But we also have completely customized suites, set
>>> > by
>>> > >>users who want to check specific things. They can do that the way you
>>> > >>do (by calling the tests explicitly), or by setting environment
>>> > >>variables (as described in the Tools chapter of the R Internals
>>> > manual).
>>> > >
>>> > > No! We're not asking for the r-core to create more test suites, or
>>> > even
>>> > > to do anything different based on the test intensity level.
>>> > >
>>> > > We're just asking for a standard way to control the intensity of the
>>> > tests
>>> > > *we* write to prevent us from duplicating this functionality in our
>>> > own
>>> > > packages, probably in incompatible ways.
>>> >
>>> > And given that CRAN recently put down timing requirements (and
>>> > Bioconductor has had them for a long time), it could be extremely
>>> > useful to have one system. It is not clear to me whether it needs
>>> > more than 2 levels ("slow" and "fast"), but I'll leave that up to
>>> > people who have thought longer about this.
>>> >
>>> > I could certainly use it in several packages to differentiate between
>>> > slow and quick tests.
>>> >
>>> > Kasper
>>> >
>>> > ______________________________________________
>>> > R-devel at r-project.org mailing list
>>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
More information about the R-devel
mailing list