From kasperdanielhansen at gmail.com  Tue Sep  4 23:12:14 2012
From: kasperdanielhansen at gmail.com (Kasper Daniel Hansen)
Date: Tue, 4 Sep 2012 17:12:14 -0400
Subject: [Rd] if(--as-cran)?
In-Reply-To: <0CAEE2B1235DF643B0F5E1EAE1487B530946D655@023-CH1MPN1-043.023d.mgd.msft.net>
References: <50465D7F.5060004@gmail.com>
	<0CAEE2B1235DF643B0F5E1EAE1487B530946D655@023-CH1MPN1-043.023d.mgd.msft.net>
Message-ID: <CAC2h7utRwFeb5_z5_gLUF=6b_cMw+CAgPnfqU_utMR7TPwu7zg@mail.gmail.com>

On Tue, Sep 4, 2012 at 4:53 PM, Warnes, Gregory
<gregory.warnes at novartis.com> wrote:
>
> On 9/4/12 3:58 PM, "Duncan Murdoch" <murdoch.duncan at gmail.com> wrote:
>
>>On 04/09/2012 3:44 PM, Terry Therneau wrote:
>>>ly in
>>> On 09/04/2012 01:57 PM, Duncan Murdoch wrote:
>>> > On 04/09/2012 2:36 PM, Warnes, Gregory wrote:
>>> >> On 9/4/12 8:38 AM, "Duncan Murdoch" <murdoch.duncan at gmail.com> wrote:
>>> >>
>>> >>
>>> >> >On 04/09/2012 8:20 AM, Terry Therneau wrote:
>>> >> >>
>>> >> >> On 09/04/2012 05:00 AM, M
>>><mailto:r-devel-request at r-project.org>artin wrote:
>>> >> >> > The issue is not just about "CRAN" vs "off CRAN".
>>> >> >> > It is good to think about a more general scheme of
>>> >> >> > "light testing" vs "normal testing" vs "extensive testing",
>>> >> >> > e.g., for the situation where the package implements
>>> >> >> > (simulation/bootstrap/ ..) based inference, and the developer
>>> >> >> > (but not only) should be able to run the extensive tests.
>>> >> >> >
>>> >> >> > Martin
>>> >> >>
>>> >> >> I agree with Martin. A mechanism to specify testing level would
>>>be the
>>> >> >> best. Then CRAN can choose to set that variable to "3" say, with
>>>level
>>> >> >> 1 for extensive and 2 for usual.
>>>>> >>
>>>
>>>[snip]
>
>>The testingLevel() function is supposed to be a way to know that a
>>certain level of testing is being done, to allow such tailoring.
>>However, I don't think it's practical.  I think you can ask whether a
>>specific test is being run (my "D" %in% tests() example), but you can't
>>reasonably convert the set of tests chosen by a tester into a single
>>number.
>>
>>What I think you and Greg are talking about is something different. You
>>are asking that we set up more suites of tests, corresponding to
>>numerical levels.  Currently we have two suites:  the default, and the
>>--as-cran suite.  But we also have completely customized suites, set by
>>users who want to check specific things.  They can do that the way you
>>do (by calling the tests explicitly), or by setting environment
>>variables (as described in the Tools chapter of the R Internals manual).
>
> No!  We're not asking for the r-core to create more test suites, or even
> to do anything different based on the test intensity level.
>
> We're just asking for a standard way to control the intensity of the tests
> *we* write to prevent us from duplicating this functionality in our own
> packages, probably in incompatible ways.

And given that CRAN recently put down timing requirements (and
Bioconductor has had them for a long time), it could be extremely
useful to have one system.  It is not clear to me whether it needs
more than 2 levels ("slow" and "fast"), but I'll leave that up to
people who have thought longer about this.

I could certainly use it in several packages to differentiate between
slow and quick tests.

Kasper



[Rd] if(--as-cran)?
Kasper Daniel Hansen