From jmc at r-project.org  Sun Dec 18 20:04:40 2011
From: jmc at r-project.org (John Chambers)
Date: Sun, 18 Dec 2011 11:04:40 -0800
Subject: [Rd] S4 NAMESPACE method imports and exports do not include
 (promoted?) generics
In-Reply-To: <CAOQ5Nyd_ies9vyvRuD78-iAf41C4qPPtbLfZOmaQMLYe8YCbTg@mail.gmail.com>
References: <4EEA841C.5060709@fhcrc.org> <4EEB52D7.9010207@fhcrc.org>
	<4EEBA7E4.70802@r-project.org> <4EEBCCD5.6010405@fhcrc.org>
	<4EEBE7CD.5080502@r-project.org>
	<CAOQ5Nyd_ies9vyvRuD78-iAf41C4qPPtbLfZOmaQMLYe8YCbTg@mail.gmail.com>
Message-ID: <4EEE3948.2080300@r-project.org>



On 12/17/11 6:02 AM, Michael Lawrence wrote:
> I guess what it boils down to is whether it makes sense for PkgB to have
> exportMethods(unique) when PkgB does not export(unique) or have PkgA in
> Depends.  And whether it makes sense for PkgC to have
> importMethodsFrom(PkgB, unique) without importFrom(PkgA, unique). If it
> is not feasible/sensible to support implicit passing of generics up the
> dependency stack, then R should probably emit some sort of warning/error
> when methods are exported or imported without the corresponding generic.
>
> The fact that a generic is being created for an implicit generic defined
> in 'base' is not really the issue here.

On the contrary, it's why we need to do some thinking about what we want.

Try replacing "unique" with "sum" throughout Martin's example, adjusting 
the method definition appropriately.  No problems arise, because sum() 
is a primitive.  Exporting and importing methods works as implied by the 
extensions manual, but via the implicit generic for sum().

The essential point is that the methods being imported are "for" the 
generic implied by the function in package "base".  PkgA is the 
irrelevant aspect if the methods come from PkgB and the implied generic 
comes from "base".  The problem is that there is no "flag" for 
non-primitive functions that says "Methods have been defined for this 
function (base::unique in this case), so some calls should carry out 
method dispatch."

The extensions manual stated that exportMethods() would "export the 
generic".  We could make that true or make it unnecessary, by having 
importMethods() look for a generic in the referenced package and infer 
the implicit generic (maybe with a message).  Otherwise, we're adding 
inconsistent requirements for primitives vs true functions in the base 
package.

John
>
> Thanks,
> Michael
>
> On Fri, Dec 16, 2011 at 4:52 PM, John Chambers <jmc at r-project.org
> <mailto:jmc at r-project.org>> wrote:
>
>     The key point here is that setGeneric("unique") is done that way,
>     without other argument, whoever does it.  That creates the generic
>     from the implicit generic corresponding to base::unique.  If package
>     A had done anything else, the resulting methods tables would NOT
>     refer to package "base" but to package "PkgA" and it's that version
>     of the generic that would need to be imported.
>
>     So it's not particularly relevant that we're dealing with
>     PkgA::unique() if the generic function was created from base::unique
>     by the standard call.  That's what would make an automatic
>     imputation of the generic from importMethods() possible.
>
>
>     On 12/16/11 2:57 PM, Martin Morgan wrote:
>
>         On 12/16/2011 12:19 PM, John Chambers wrote:
>
>             The subject heading is correct if referring to
>             exportMethods() and
>             importMethodsFrom(). They refer to the methods tables, not
>             the generic
>             functions, whatever the extensions manual says.
>
>             Looking into the code of namespaceImportMethods() will
>             illustrate this.
>             It just deals with lists of method tables obtained from
>             .getGenerics()
>             which in spite of its name also only looks for the method
>             table metadata
>             objects.
>
>             As I vaguely recall, there was some concern at one time
>             about having
>             extra copies of the generic version of the function.
>
>             The fundamental problem is that creating methods for
>             unique(), say, does
>             not change the way calls to base::unique() work. Therefore,
>             all packages
>             that want to use methods have to ensure that a generic
>             version of unique
>             gets in between. Primitive functions are an exception
>             because the method
>             dispatch is in the C code and has a rule for checking when
>             given an S4
>             object. There is no corresponding provision for evaluating a
>             call to a
>             regular function.
>
>             If the importing package has a setGeneric() for the relevant
>             function
>             then its own namespace has the generic version of the
>             function. (That is
>             a workaround, but I inferred that was what you were trying
>             to avoid.)
>
>             Fixes seem possible, but some care is needed. If exportMethods
>             automatically exported the generic function, it really is no
>             different
>             from export() for that function.
>
>
>         export() somehow implies ownership of the generic, e.g.,
>         responsibility
>         for documentation. I can see in the scenario below that PkgB
>         might be
>         expected to Depends: PkgA if it intends for the user to access
>         PkgB's
>         methods on PkgA::unique.
>
>             namespaceImportMethods() could try to supply the generic
>             function if it
>             is not already present. If it does not find the generic in
>             the namespace
>             being imported, it would essentially have to call
>             setGeneric(), assuming
>             the non-generic function exists in the specified package
>             (e.g., in base
>             for unique()).
>
>
>         In the example below for PkgC the 'unique' generic is in PkgB's
>         namespace imports
>
>          > getNamespaceImports("PkgB")
>         $base
>         [1] TRUE
>
>         $PkgA
>         unique
>         "unique"
>
>         I guess PkgB could have Depends: PkgA, not importFrom(PkgA,
>         unique), and
>         then defined and exported a method on PkgA::unique found on the
>         search
>         path, so that the generic wasn't available to PkgC. But I'd be
>         happy if
>         the generic found in either PkgB's namespace or namespace
>         imports were
>         imported along with the method. Not sure that I like the idea of
>         calling
>         setGeneric() -- PkgA could have done something non-standard --
>         and would
>         rather an error.
>
>         Thans for your attention.
>
>         Martin
>
>
>             Comments?
>             John
>
>
>
>             On 12/16/11 6:16 AM, Martin Morgan wrote:
>
>                 tar of Pkgs A, B, C attached. Martin
>
>                 On 12/15/2011 03:34 PM, Martin Morgan wrote:
>
>                     In
>
>                      > R.version.string
>                     [1] "R Under development (unstable) (2011-12-15 r57901)"
>
>                     section 1.6.6 of 'Writing R Extensions' says
>
>                     Note that exporting methods on a generic in the
>                     namespace will
>                     also export the generic, and exporting a generic in the
>                     namespace will also export its methods.
>
>                     and
>
>                     Note that importMethodsFrom will also import any
>                     generics defined in
>                     the namespace on those methods
>
>                     However, if PkgA promotes 'unique' to a generic and
>                     exports that
>
>                     DESCRIPTION:
>                     Imports: methods
>
>                     R/f.R:
>                     setGeneric("unique")
>
>                     NAMESPACE:
>                     export(unique)
>
>                     and PkgB creates and exports a method on unique
>
>                     DESCRIPTION
>                     Imports: methods, PkgA
>
>                     R/f.R:
>                     setClass("B", representation(b="numeric"))
>                     setMethod(unique, "B",
>                     function(x, incomparables=FALSE, ...) unique(x at b))
>
>                     NAMESPACE:
>                     importFrom(PkgA, unique)
>                     exportClasses(B)
>                     exportMethods(unique)
>
>                     and PkgC wants to import PkgB's classes and methods
>
>                     DESCRIPTION
>                     Imports: methods, PkgB
>
>                     R/f.R
>                     cunique <- function(x) unique(x)
>
>                     NAMESPACE
>                     importMethodsFrom(PkgB, unique)
>                     export(cunique)
>
>                     then
>
>                     (a) the 'unique' generic is not available to the
>                     user of PkgB
>
>                      > library(PkgB)
>                      > unique(new("B", b=1:5))
>                     Error in unique.default(new("B", b = 1:5)) :
>                     unique() applies only to vectors
>
>                     and (b) the generic has not been imported to PkgC's
>                     namespace
>
>                      > cunique(new("B", b=1:5))
>                     Error in unique.default(b) : unique() applies only
>                     to vectors
>
>                     A workaround is for PkgB to also export(unique), and
>                     for PkgC to also
>                     importFrom(PkgA, unique), but is this the intention?
>
>                     This is arising from Bioconductor efforts to place
>                     commonly promoted
>                     functions and S3 classes into a single package, to
>                     avoid conflicts when
>                     the same function is promoted independently by
>                     several packages.
>
>                     Martin
>
>
>
>
>
>
>     ________________________________________________
>     R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list
>     https://stat.ethz.ch/mailman/__listinfo/r-devel
>     <https://stat.ethz.ch/mailman/listinfo/r-devel>
>
>



[Rd] S4 NAMESPACE method imports and exports do not include (promoted?) generics
John Chambers