From bbo|ker @end|ng |rom gm@||@com  Wed Sep 14 02:31:33 2022
From: bbo|ker @end|ng |rom gm@||@com (Ben Bolker)
Date: Tue, 13 Sep 2022 20:31:33 -0400
Subject: [Rd] 
 Should package version requirements assume installation from
 sources?
In-Reply-To: <07ec1390-3e67-82cf-09b4-6668db099c31@gmail.com>
References: <b3e53fa9-b0e0-a8c8-147a-c1c56348f3e9@gmail.com>
 <07ec1390-3e67-82cf-09b4-6668db099c31@gmail.com>
Message-ID: <70bacfe0-58f2-c656-3a91-061901f4a3f8@gmail.com>

   Interesting.

   How is the version number bumping communicated back to package 
maintainers? Or is there a rule like "third digit-blocks of package 
versions are reserved for Bioconductor modifications" ?

On 2022-09-13 7:11 p.m., Herv? Pag?s wrote:
> FWIW this happens all the time in Bioconductor, where S4 is used very 
> intensively, when we make that kind of change to a core infrastructure 
> package. Then a bunch of Windows and Mac binary packages need to be 
> rebuilt because they contain cached stuff that is now out-of-sync with 
> the new state of affairs. The way we deal with this is by bumping the 
> version numbers of all the affected packages. This triggers automatic 
> rebuilt and propagation of the new binaries.
> 
> Note that these problems also affect packages that were already 
> installed on the user machines _prior_ to the changes to the upstream 
> package. Because the caching we are talking about is actually in the 
> package installation folder (and the fact that it ends up in the 
> binaries is just a consequence of that). So even on a system where 
> packages are installed from source, the affected packages need to be 
> reinstalled. Bumping the version numbers of all the affected packages 
> also solves that.
> 
> I don't see how this could be handled or mitigated via package 
> requirements.
> 
> Best,
> 
> H.
> 
> 
> On 13/09/2022 14:45, Mikael Jagan wrote:
>> [Arguably also appropriate for R-package-devel, but posted to R-devel
>> ??? as the discussion is aimed primarily at "experts" ... ]
>>
>> We, the authors of Matrix, have encountered a somewhat subtle issue
>> induced by caching of S4 classes and methods in package namespaces.
>>
>> The namespaces of three reverse dependent packages (SeuratObject, 
>> conText,
>> mcmcsae) cache the formal definition of our virtual class Matrix (and 
>> some
>> subclasses).? For example:
>>
>> > ns <- asNamespace("SeuratObject")
>> > grep("^[.]__C__.*Matrix$", names(ns), value = TRUE)
>> [1] ".__C__dMatrix"?????? ".__C__compMatrix"??? ".__C__AnyMatrix"
>> [4] ".__C__generalMatrix" ".__C__CsparseMatrix" ".__C__sparseMatrix"
>> [7] ".__C__dsparseMatrix" ".__C__Matrix"
>>
>> The cached definition (which includes a _validity method_) is obtained 
>> from
>> the version of Matrix available when the reverse dependent package was 
>> built
>> from sources.? For example, if SeuratObject was built under Matrix 1.4-1,
>> then we get:
>>
>> > getValidity(ns$.__C__Matrix)
>> function (object)
>> {
>> ??? if (!isTRUE(r <- .Call(Dim_validate, object, "Matrix")))
>> ??????? r
>> ??? else .Call(dimNames_validate, object)
>> }
>> <bytecode: 0x11e7ca508>
>> <environment: namespace:Matrix>
>>
>> whereas if SeuratObject was built under Matrix >= 1.5-0, then we get:
>>
>> > getValidity(ns$.__C__Matrix)
>> function (object)
>> .Call(Matrix_validate, object)
>> <bytecode: 0x107dc1698>
>> <environment: namespace:Matrix>
>>
>> There are two "questions" here:
>>
>> 1.? The symbol 'Matrix_validate' is not defined until Matrix 1.5-0.
>> ??? Is it necessary, for this reason alone, for SeuratObject to have
>> ??? 'Imports: Matrix (>= 1.5-0)'?? Or can SeuratObject continue using
>> ??? 'Imports: Matrix (>= 1.3-3)', at the risk of errors like
>>
>> ??? > Error: object 'Matrix_validate' not found
>>
>> ??? (as already seen here: https://stackoverflow.com/questions/73700130)?
>>
>> ??? Note that this error would not occur for anyone installing 
>> SeuratObject
>> ??? from sources, unless they decide to _downgrade_ Matrix after doing 
>> so.
>> ??? Hence this primarily concerns Windows and macOS where R users would
>> ??? typically install a binary built by CRAN (i.e., not on their system).
>>
>> ??? We are aware that package TMB tests in .onLoad() that the current 
>> Matrix
>> ??? version is equal to or greater than the version available at build 
>> time,
>> ??? thus avoiding a "strict" version requirement, but do not want this 
>> practice
>> ??? to spread ...
>>
>> 2.? For how long should Matrix retain the superceded 'Dim_validate' and
>> ??? 'dimNames_validate', in order to ensure that "stale" cached validity
>> ??? methods continue to work?
>>
>> We hope that this discussion will highlight the potential ramifications
>> of importing classes and methods from other packages, and having one's
>> classes and methods imported _by_ other packages, especially for version
>> requirements.
>>
>> Mikael, Martin, and Doug
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> 

-- 
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics
 > E-mail is sent at my convenience; I don't expect replies outside of 
working hours.



[Rd] Should package version requirements assume installation from sources?
Ben Bolker