From kasperdanielhansen at gmail.com  Thu Sep  6 03:51:01 2012
From: kasperdanielhansen at gmail.com (Kasper Daniel Hansen)
Date: Wed, 5 Sep 2012 21:51:01 -0400
Subject: [Rd] if(--as-cran)?
In-Reply-To: <50474F2E.9000600@gmail.com>
References: <50465D7F.5060004@gmail.com>
	<0CAEE2B1235DF643B0F5E1EAE1487B530946D655@023-CH1MPN1-043.023d.mgd.msft.net>
	<20550.28445.956167.112449@max.nulle.part> <504671FC.9030800@gmail.com>
	<20550.30155.399375.438614@max.nulle.part> <50467A5E.2080106@gmail.com>
	<CAC2h7usPuepziN4ngNg2F_9jOPPF_doTJR7bnfFbpX_0gowHtw@mail.gmail.com>
	<5046ED0B.4060506@gmail.com>
	<CADfFDC55XL5P8ytALSeCT5QK5DCXfbfo7z7AmJRaOZiU_-R3EQ@mail.gmail.com>
	<20551.10411.168210.290864@stat.math.ethz.ch>
	<50474F2E.9000600@gmail.com>
Message-ID: <CAC2h7uvFWmjUfA0yxMNXtGwR4rZGdXZW_reBvP9n74=vOgwH0g@mail.gmail.com>

Duncan,

you are right that from the perspective of a single developer: it is
entirely possible to make tests conditional and ensure that 'long'
tests does not get run by CRAN.

What I, and several others, advocate is a uniform set of conventions,
so we avoid having multiple package specific solutions.

Martin has explained one reason for why using _R_CHECK_TIMINGS_ is not
ideal (the fact that we - at least I and Martin - are most interested
in the timings for the long tests).  Another reason is that I (and I
suspect most others) would prefer something that looks a bit more
official and supported.  High-jacking an environment variable set for
(what appears to be) a different reason does not seem like something
that could be considered supported long-term, and which have a uniform
interface (across packages).

Of course, adding an extra environment variable or an extra function
(as suggested by Henrik) is additional work, and I know it is easy for
us to suggest things.  My perspective is that I would like something
that will continue to be supported in the future, and where I can use
the same interface across the packages I maintain and others outside
my control that I occasionally test.  Testing is not just being done
by the package author(s) or CRAN.   I suspect many packages already
have half-baked solutions to the 'long' test vs 'short' test issue,
and I don't think we will see any movement toward a unified interface
without support in base R.

And let me end by saying that I do agree with Martin: having numerical
levels is better (together with a clear statement what --as-cran sets
the level to).  But I would rather have a binary state than nothing.

Kasper


On Wed, Sep 5, 2012 at 9:10 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
> This post contains two incorrect statements.  Since Martin is too busy to
> post a retraction, I will point them out:
>
>  --timings and --as-cran do not set _R_CHECK_TIMINGS_ in the same way.
>
> As far as I recall, nobody has suggested that package writers be limited to
> two choices for test suites.  I certainly haven't.
>
> Duncan Murdoch
>
>
> On 05/09/2012 6:25 AM, Martin Maechler wrote:
>>
>> >>>>> Deepayan Sarkar <deepayan.sarkar at gmail.com>
>> >>>>>     on Wed, 5 Sep 2012 11:49:37 +0530 writes:
>>
>>      > On Wed, Sep 5, 2012 at 11:41 AM, Duncan Murdoch
>>      > <murdoch.duncan at gmail.com> wrote:
>>      >> On 12-09-04 8:19 PM, Kasper Daniel Hansen wrote:
>>      >>>
>>      >>> On Tue, Sep 4, 2012 at 6:02 PM, Duncan Murdoch
>> <murdoch.duncan at gmail.com>
>>      >>> wrote:
>>      >>>>
>>      >>>> On 04/09/2012 5:42 PM, Dirk Eddelbuettel wrote:
>>      >>>>>
>>      >>>>>
>>      >>>>> On 4 September 2012 at 17:26, Duncan Murdoch wrote:
>>      >>>>> | On 04/09/2012 5:14 PM, Dirk Eddelbuettel wrote:
>>      >>>>> | > An add-on argument to the already established option
>> --as-cran may
>>      >>>>> be
>>      >>>>> the
>>      >>>>> | > best.
>>      >>>>> | >
>>      >>>>> | > And to iterate, what bugs me is that for _me_ on _my_
>> machine
>>      >>>>> developing _my_
>>      >>>>> | > package I have remember how to enable what is now (as per
>> CRAN's
>>      >>>>> decree)
>>      >>>>> | > "non-standard behaviour" of full testing.  I fully agree
>> with what
>>      >>>>> Terry had
>>      >>>>> | > said: more tests are better (when we develop).  I want the
>> full
>>      >>>>> suite
>>      >>>>> at my
>>      >>>>> | > end; that is after all why we wrote it!
>>      >>>>> |
>>      >>>>> | You don't have to remember that, you need to figure it out
>> once, write
>>      >>>>> a
>>      >>>>> | script that sets the environment variables that enable it,
>> and then
>>      >>>>> you
>>      >>>>> | can forget it.
>>      >>>>>
>>      >>>>> "In theory, theory and practice are the same. In practice, they
>> are
>>      >>>>> not."
>>      >>>>>
>>      >>>>> The main test script long had exactly such a setting; I wrote
>> what I
>>      >>>>> wrote
>>      >>>>> because it is _still the wrong way around_ and as I happen to
>> have added
>>      >>>>> to
>>      >>>>> unit tests this weekend _having suffered through precisely this
>>      >>>>> setting_.
>>      >>>>>
>>      >>>>> But we are on different wavelengths here and I evidently do not
>> get my
>>      >>>>> point
>>      >>>>> across to you.  And as you are the one who could make a change
>> where it
>>      >>>>> matters, I have no choice but to rest my case in frustration.
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>> If you want to give up, then give up, but then don't complain
>> about the
>>      >>>> current behaviour.  If you want to fix it, then continue the
>> discussion.
>>      >>>>
>>      >>>> You're right that we're on different wavelengths.  If you want
>> some tests
>>      >>>> to
>>      >>>> run at home but not on CRAN, then somewhere there has to be a
>>      >>>> conditional.
>>      >>>> I'm suggesting that the conditional should be "if there's a
>> tight time
>>      >>>> limit, skip this".
>>      >>>>
>>      >>>> I don't remember if this was your suggestion, but someone has
>> suggested
>>      >>>> "if
>>      >>>> we're running with the --as-cran option, skip this" and others
>> have
>>      >>>> suggested "if we're running on CRAN, skip this".   I don't see
>> why you
>>      >>>> find
>>      >>>> my suggestion so objectionable.  If you want, I'll repeat why I
>> find the
>>      >>>> other two suggestions objectionable.
>>      >>>
>>      >>>
>>      >>> I agree with Duncan that having an option long/short makes more
>> sense
>>      >>> than with/without cran, as long as cran sets that option to be
>> short.
>>      >>
>>      >>>
>>      >>>
>>      >>> I would also prefer a command line switch to R CMD check to an
>>      >>> environment variable, but I'll be very happy with a standardized
>>      >>> environment variable.
>>      >>
>>      >>
>>      >> I honestly don't see the need for a standardized variable.  I've
>> told you
>>      >> how to detect that you are running --as-cran; if that isn't
>> sufficient
>>      >> information, then you, the package author, need to set up
>> something more
>>      >> elaborate, and assume that if it's not set up, then someone else
>> (maybe
>>      >> CRAN) is running the test.
>>
>>      > So maybe documenting that (_R_CHECK_TIMINGS_) more formally in
>> R-exts
>>      > would be sufficient?
>>
>>      > -Deepayan
>>
>> yes and no.
>> As Duncan said very early,  --as-cran is just turning on several
>> already existing options, one of them being the
>> _R_CHECK_TIMINGS_  -- which you can *also* enable by  --timings.
>>
>> So, checking for  _R_CHECK_TIMINGS_ is *not* checking for
>> the presence of  --as-cran !
>> .. in the sense that it is necessary but not sufficient.
>> Also, if I run the "long checks" I may still want to use
>>   --timings. For me as package developer, the timings may be even more
>> important in the "long" than in the "short" checks case.
>>
>> So, one solution that is little work would be that  --as-cran
>> sets an *extra* environment variable that is *only* set by
>> --as-cran, but by no other command line switch.
>>
>> Still a pity that it seems people want to live in a  0/1
>> setting when it would be very natural to adopt a  0/1/2/3
>> (or so) one.
>> It's a bit like prefering  verbose = FALSE/TRUE  as argument to
>> an R function where eventually I often found that using a
>> tracing = 0/1/2  (or then  'verbose' = 0/1/2  which is almost
>> back compatible to FALSE/TRUE)
>> was more useful.
>>
>>
>>
>>      >> I asked the CRAN powers-that-be about the possibility of querying
>> the amount
>>      >> of time remaining before a timeout; since the different platforms
>> all use
>>      >> different mechanisms to enforce a timeout, that's not really
>> practical.  So
>>      >> the best you could hope for is to know that a timeout is in
>> effect.  Before
>>      >> I wrote any code, I'd need to hear why --as-cran detection isn't
>> sufficient.
>>
>> I agree that *reliable*  --as-cran  detection  solves the OP's
>> and most other correspondents problem.
>> But as argued above,  _R_CHECK_TIMINGS_  is not sufficient.
>>
>> Martin
>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



[Rd] if(--as-cran)?
Kasper Daniel Hansen