[Rd] if(--as-cran)?

[Terry Therneau]

On 09/04/2012 01:57 PM, Duncan Murdoch wrote:
> On 04/09/2012 2:36 PM, Warnes, Gregory wrote:
>> On 9/4/12 8:38 AM, "Duncan Murdoch" <murdoch.duncan at gmail.com> wrote:
>>
>>
>> >On 04/09/2012 8:20 AM, Terry Therneau wrote:
>> >>
>> >> On 09/04/2012 05:00 AM, r-devel-request at r-project.org wrote:
>> >> > The issue is not just about "CRAN" vs "off CRAN".
>> >> > It is good to think about a more general scheme of
>> >> > "light testing" vs "normal testing" vs "extensive testing",
>> >> > e.g., for the situation where the package implements
>> >> > (simulation/bootstrap/ ..) based inference, and the developer
>> >> > (but not only) should be able to run the extensive tests.
>> >> >
>> >> > Martin
>> >>
>> >> I agree with Martin. A mechanism to specify testing level would be the
>> >>best.
>> >> Then CRAN can choose to set that variable to "3" say, with level 1 for
>> >>extensive and 2 for
>> >> usual.
>> >> I'm quite willing to put up with the nuisance of print()
>> >>enclosures. I prefer it to
>> >> having yet another way to subvert the evaluation model.
>> >>
>> >> I'm a believer in testing everything possible in my packages, and
>> >>wear it it as a badge
>> >> of honor that the survival package has 4 lines of R code in the tests
>> >>directory for every
>> >> 3 in the R directory. But CRAN only needs to run a small subset of
>> >>this.
>> >
>> >We have a mechanism to specify testing level: the --as-cran flag. We
>> >could presumably make it more elaborate by adding other flags, or option
>> >levels, or whatever.
>> >
>> >What I think we shouldn't do is try to create an R-level test that says
>> >
>> > if (testingLevel() > 3) {
>> > doSomething
>> >}
>> >
>> >because tests can be turned on and off, individually. If testingLevel 3
>> >specified tests (A, B, C), then is our testingLevel higher if we are
>> >running tests (A, B, D, E, F, G)? Why not just test for the presence of
>> >whichever test is most relevant to that particular code block, e.g.
>> >
>> > if ("D" %in% tests()) {
>> > doSomething
>> >}
>>
>>
>> I would prefer the testingLevel() approach of the "D" %in% tests()
>> approach, since testingLevel() provides a natural way to add successively
>> greater test details without having to dig into the code to determine the
>> set of tests.
>
> I don't see how you could possibly calculate a single number in a reasonable way. What is
> the number that should be returned for (A, B, D, E, F, G)?
>
> Duncan Murdoch

Duncan is leapfrogging ahead to another level that I hadn't thought of.  An example would 
be to divide my survival package as "cox", "parametric", "survfit", "all", some of whaich 
overlap. Interesting idea, but beyond what I'd use.  When I'm focused on a detail I run 
the test of interest directly, not through CMD check.  For me low, med, high intensity 
suffices with as-cran invoking the first level and high including those that take an 
exceptionally long time.
  If you went to an A, B, C, ... approach what would be the as-cran default?

Of course there is then the furthest level, which I recently instituted for survival due
to the large number of dependencies: before posting a change download all the other
dependent packages and run their tests too.  It's an overnighter, and in that case I'd 
want level=high.  Forewarned is forearmed.

Terry T.