From sfalcon at fhcrc.org  Fri Aug 31 17:02:17 2007
From: sfalcon at fhcrc.org (Seth Falcon)
Date: Fri, 31 Aug 2007 08:02:17 -0700
Subject: [Rd] R CMD check recursive copy of tests/
In-Reply-To: <59d7961d0708300731x15a363b3w34db4fb4d3ac0310@mail.gmail.com>
	(Henrik Bengtsson's message of "Thu\,
	30 Aug 2007 07\:31\:02 -0700")
References: <59d7961d0708291511r36a130d9vdd5c1f32b277e670@mail.gmail.com>
	<200708301004.l7UA4eHv016956@lynne.ethz.ch>
	<59d7961d0708300731x15a363b3w34db4fb4d3ac0310@mail.gmail.com>
Message-ID: <m2myw7rbly.fsf@fhcrc.org>

"Henrik Bengtsson" <hb at stat.berkeley.edu> writes:
>> intentional I'd say: I did not implement it, but it seems much
>> more logical to keep the previous rule: All *.R files in
>> ./tests/ are run <period>
>> Subdirectories can be useful for organization, notably storing
>> test data.  I don't think it's a good idea to use so very many test files
>> that you need subdirectories, unless maybe you are thinking
>> about unit tests; and then, see below.
>
> Examples of subdirectories (some overlapping) are:
>
> units/ - tests of minimal code modules
> integration/ - tests of integrating the above units
> system/ - "real-world" scenarios/use cases
>
> requirements/ - every requirement should have at least on test.
> bugs/ - every bug fix should come with a new test.
> regression/ - every update should have a regression test to validate
> backward compatibility etc.
>
> robustness/ - Testing the robustness of estimators against outliers as
> well as extreme parameter settings.
> validation/ - validation of numeric results compared with alternative
> implementations or summaries.
>
> benchmarking/ - actually more measuring time, but can involve
> validation that a method is faster than an alternative.
> crossplatform/ - validate correctness across platforms.
> torture/ - pushing the limits.

Those all seem like reasonable examples, but the fact that R CMD check
doesn't recurse really isn't a problem.  You can have a driver script
at the top-level that runs as many of the tests in subdirs as you
want.  And this is really a good thing since as you mentioned later in
your response, some tests take a long time to run and probably are
best not automatically run during R CMD check.

+ seth

-- 
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
BioC: http://bioconductor.org/
Blog: http://userprimary.net/user/



[Rd] R CMD check recursive copy of tests/
Seth Falcon