From michael.weylandt at gmail.com  Thu Mar 20 02:55:18 2014
From: michael.weylandt at gmail.com (Michael Weylandt)
Date: Wed, 19 Mar 2014 21:55:18 -0400
Subject: [Rd] [RFC] A case for freezing CRAN
In-Reply-To: <CAPPM_gR4kvVxk+sWqYGRxLyaPzw5bMr=kM==-o7sTKGpJy1iug@mail.gmail.com>
References: <CABFfbXt0PjHEw-4=VfeWeeosm_wOBH-fVsj85j9LMe81Lx0gWQ@mail.gmail.com>
	<53299314.6080608@gmail.com>
	<CABFfbXuTvefJ0Nmc25JJEwhmUQge+K1Fi=oaTwoO_jfK75mUZg@mail.gmail.com>
	<CAPPM_gRP4xKEE1bFiR8Aaeo1XOX0kpu4p=EyOm2z+6fkVkJaag@mail.gmail.com>
	<CABFfbXtmFyoyaRSU6LyWrUzyXvvkSV14yD3vc0c2fzRcHbtNzQ@mail.gmail.com>
	<CAPPM_gR7pP7eq2Gg3nT4osSw1S69hvEW=yCx3-Q7T1V0fb19Vg@mail.gmail.com>
	<CABFfbXteYwBwE018xf9aa_z95h3k3OquT2_BQmrsL6n4W57tWA@mail.gmail.com>
	<CAPPM_gR4kvVxk+sWqYGRxLyaPzw5bMr=kM==-o7sTKGpJy1iug@mail.gmail.com>
Message-ID: <A5F1114B-D870-478F-B11F-2EB7419F6BCE@gmail.com>



On Mar 19, 2014, at 18:42, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:

> On Wed, Mar 19, 2014 at 5:16 PM, Jeroen Ooms <jeroen.ooms at stat.ucla.edu> wrote:
>> On Wed, Mar 19, 2014 at 2:59 PM, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:
>>> 
>>> So implementation isn't a problem.  The problem is that you need a way
>>> to force people not to be able to use different package versions than
>>> what existed at the time of each R release.  I said this in my
>>> previous email, but you removed and did not address it: "However, you
>>> would need to find a way to actively _prevent_ people from installing
>>> newer versions of packages with the stable R releases."  Frankly, I
>>> would stop using CRAN if this policy were adopted.
>> 
>> I am not proposing to "force" anything to anyone, those are your
>> words. Please read the proposal more carefully before derailing the
>> discussion. Below *verbatim* a section from the paper:
> <snip>
> 
> Yes "force" is too strong a word.  You want a barrier (however small)
> to prevent people from installing newer (or older) versions of
> packages than those that correspond to a given R release.


Jeroen,

Reading this thread again, is it a fair summary of your position to say "reproducibility by default is more important than giving users access to the newest bug fixes and features by default?" It's certainly arguable, but I'm not sure I'm convinced: I'd imagine that the ratio of new work being done vs reproductions is rather high and the current setup optimizes for that already. 

What I'm trying to figure out is why the standard "install the following list of package versions" isn't good enough in your eyes? Is it the lack of CRAN provided binaries or the fact that the user has to proactively set up their environment to replicate that of published results?

In your XML example, it seems the problem was that the reproducer didn't check that the same package versions as the reproducee and instead assumed that 'latest' would be the same. Annoying yes, but easy to solve. 

Michael



[Rd] [RFC] A case for freezing CRAN
Michael Weylandt