From brian at braverock.com  Wed Sep  3 21:58:04 2014
From: brian at braverock.com (Brian G. Peterson)
Date: Wed, 03 Sep 2014 14:58:04 -0500
Subject: [Rd] timings for examples in R CMD check
Message-ID: <540772CC.3070108@braverock.com>

I'm having a very hard time making R CMD check produce a clean check on 
examples because of the timings inserted into examples by R CMD check.

I am getting a difference on every example output caused by timing 
information being inserted by 'R CMD check'.

The current 'Writing R Extensions' manual[1] states on p. 14:

    If directory tests has a subdirectory Examples containing a
    file pkg-Ex.Rout.save, this is  compared to the output file
    for running the examples when the latter are checked.
    Reference  output should be produced without having the
    --timings option set (and note that --as-cran  sets it).

As is suggested above, if I run

    R CMD check --as-cran mypkg.tar.gz

or

    R CMD check --timings mypkg.tar.gz

the my.pkg.Rcheck/mypkg-Ex.Rout file contains timing lines inserted by 
'R CMD check'

If, instead, I follow my reading of the manual, and run

R CMD check mypkg.tar.gz

the mypkg.Rcheck/mypkg-Ex.Rout file does not contain timings.  If I then 
copy the mypkg.Rcheck/mypkg-Ex.Rout file to

mypkg/tests/Examples/mypkg-Ex.Rout.save

as described by footnote 17 on the previously referenced p. 14, I would 
expect to now have no differences, as demonstrated by 'diff' or 'diff 
-bw' as used by 'R CMD check'.

To my surprise, whether I copy a version of the mypkg-Ex.Rout file with 
or without timing rows, I get differences reported for every example by 
the (spurious?) insertion of timings by R CMD check.

e.g.
8169a8549,8550
 > > base::assign(".dptime", (proc.time() - get(".ptime", pos = 
"CheckExEnv")), pos = "CheckExEnv")
 > > base::cat("textplot", base::get(".format_ptime", pos = 
'CheckExEnv')(get(".dptime", pos = "CheckExEnv")), "\n", 
file=base::get(".ExTimings", pos = 'CheckExEnv'), append=TRUE, sep="\t")
8175a8557



Is this expected?

It is certainly not the behavior we had for many years, where a check of 
examples could actually be clean, producing differences only on the 
'Time elapsed:' line, which has been present for many R versions.  I 
have tried testing against a package with both the timing lines includes 
and excluded in /mypkg-Ex.Rout.save, and with R CMD check run without 
additional options, with --timings, and with --as-cran.   They produce 
equivalent check files. on their example checks.

This behavior is on R 3.1.0, 3.1.1, and R-devel.

I would assume from the documentation that there should be a way to get 
a clean check, without the timing diffs.

Advice appreciated,

Brian

Ref:
[1] http://cran.r-project.org/doc/manuals/R-exts.pdf

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock



[Rd] timings for examples in R CMD check
Brian G. Peterson