From ripley at stats.ox.ac.uk  Tue Jun 26 11:51:03 2007
From: ripley at stats.ox.ac.uk (Prof Brian Ripley)
Date: Tue, 26 Jun 2007 10:51:03 +0100 (BST)
Subject: [Rd] conflict between lme4 and RMySQL packages (PR#9753)
In-Reply-To: <20070626084353.227DD65971@slim.kubism.ku.dk>
References: <20070626084353.227DD65971@slim.kubism.ku.dk>
Message-ID: <Pine.LNX.4.64.0706261002260.19805@gannet.stats.ox.ac.uk>

You have three summary() functions here:

> library(RMySQL)
Loading required package: DBI
> library(lme4)
Loading required package: Matrix
Loading required package: lattice
> find("summary")
[1] "package:Matrix" "package:DBI"    "package:base"

and this is simply a namespace issue (note the environments):

> summary
standardGeneric for "summary" defined from package "base"

function (object, ...)
standardGeneric("summary")
<environment: 0x15122d0>
Methods may be defined for arguments: object

> Matrix:::summary
standardGeneric for "summary" defined from package "base"

function (object, ...)
standardGeneric("summary")
<environment: 0x26c6640>
Methods may be defined for arguments: object

> DBI:::summary
standardGeneric for "summary" defined from package "base"

function (object, ...)
standardGeneric("summary")
<environment: 0x15122d0>
Methods may be defined for arguments: object

so the visible summary() from package:Matrix is a copy of that in the DBI 
namespace, not that in the Matrix namespace.  Since lme4 imports all of 
Matrix, it imports its take-over of summary.  So the issue is not *S4* 
dispatch per se, but how the 'methods' package copes with two or more 
packages presenting the same generic.

If you dig a bit deeper,

> ls(environment(summary)$".AllMTable")
  [1] "ANY"             "DBIObject"       "MySQLConnection" "MySQLDriver"
  [5] "MySQLResult"     "lmer2"           "mer"             "sparseMatrix"
  [9] "summary.lmer2"   "summary.mer"
> ls(environment(summary)$".AllMTable")
  [1] "ANY"             "DBIObject"       "MySQLConnection" "MySQLDriver"
  [5] "MySQLResult"     "lmer2"           "mer"             "sparseMatrix"
  [9] "summary.lmer2"   "summary.mer"
> ls(environment(Matrix:::summary)$".AllMTable")
[1] "ANY"          "sparseMatrix"
> ls(environment(DBI:::summary)$".AllMTable")
  [1] "ANY"             "DBIObject"       "MySQLConnection" "MySQLDriver"
  [5] "MySQLResult"     "lmer2"           "mer"             "sparseMatrix"
  [9] "summary.lmer2"   "summary.mer"

shows the issue: the methods do not get stored on the version in the 
Matrix namespace, but rather on the version in package:Matrix which was 
copied from the existing export and not Matrix:::summary.

Clearly there is going to be a namespace problem one way round: what seems 
undocumented is which.


On Tue, 26 Jun 2007, maechler at stat.math.ethz.ch wrote:

> Thank you, Dale.
>
> I think you've revealed a bug in the method dispatch mechanism.
> I'm adding here a script which confirms your finding and add
> slightly more insight:
>
> library(RMySQL)
> library(lme4)
>
> data(sleepstudy)
> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>
> ## MM: First save, then print -- the error is in "print" i.e. show() :
> s1 <- summary(fm1)
> s1 #-> error in printMer()  which is hidden :
>
> debug(lme4:::printMer)
> s1 # --> printMer(x, ..)  calls  so <- summary(x)  at the very beginning
> ## and indeed,  summary(x) -- *inside* printMer() dispatches wrongly !!!!
> ## whereas top-level, there is no problem :
>
> ## maybe here, or then later
> undebug(lme4:::printMer)
>
> ## This works fine top-level, but gives non-sense when inside  printMer() !!
> ##
> so <- summary(s1) ## does nothing, since s1 *is* already  summary.lmer :
> stopifnot(identical(so, s1)) # ok
>
> ## one other symptom of the same problem:
> showMethods(summary) ## is fine top-level
>
> ## whereas it does not see the correct methods when used from inside printMer()
> ## i.e., also when debugging
>
>
>
>>>>>> "db" == dale barr <dale.barr at ucr.edu>
>>>>>>     on Tue, 26 Jun 2007 00:57:22 +0200 (CEST) writes:
>
>    db> Full_Name: Dale Barr
>    db> Version: 2.5.1 (patched)
>    db> OS: Ubuntu linux x86_64
>    db> Submission from: (NULL) (138.23.70.108)
>
>
>    db> When RMySQL is loaded in before lme4, the summary() function for lmer objects in
>    db> the lme4 packages produces the following error:
>
>    db> Error in printMer(object) : no slot of name "status" for this object of class
>    db> "table"
>
>    db> When RMySQL is loaded AFTER lme4, however, no such error arises.  For example,
>    db> the following code gives the error:
>
>    >> library(RMySQL)
>    db> Loading required package: DBI
>    >> library(lme4)
>    db> Loading required package: Matrix
>    db> Loading required package: lattice
>    >> data(sleepstudy)
>    >> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>    >> summary(fm1)
>    db> Error in printMer(object) : no slot of name "status" for this object of class
>    db> "table"
>
>    db> Now, here is the exact same code except that lme4 is loaded before RMySQL.  The
>    db> summary function works properly.
>
>    >> library(lme4)
>    db> Loading required package: Matrix
>    db> Loading required package: lattice
>    db> library(RMySQL)> library(RMySQL)
>    db> Loading required package: DBI
>    >> data(sleepstudy)
>    >> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>    >> summary(fm1)
>    db> Linear mixed-effects model fit by REML
>    db> Formula: Reaction ~ Days + (Days | Subject)
>    db> Data: sleepstudy
>    db> AIC  BIC logLik MLdeviance REMLdeviance
>    db> 1754 1770 -871.8       1752         1744
>    db> Random effects:
>    db> Groups   Name        Variance Std.Dev. Corr
>    db> Subject  (Intercept) 610.835  24.7151
>    db> Days         35.056   5.9208  0.067
>    db> Residual             655.066  25.5943
>    db> number of obs: 180, groups: Subject, 18
>
>    db> Fixed effects:
>    db> Estimate Std. Error t value
>    db> (Intercept)  251.405      6.820   36.86
>    db> Days          10.467      1.546    6.77
>
>    db> Correlation of Fixed Effects:
>    db> (Intr)
>    db> Days -0.137
>
>
>    db> MY SESSION INFO:
>    db> R version 2.5.1 RC (2007-06-22 r42030)
>    db> x86_64-unknown-linux-gnu
>
>    db> locale:
>    db> LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C
>
>    db> attached base packages:
>    db> [1] "stats"     "graphics"  "grDevices" "utils"     "datasets"  "methods"
>    db> [7] "base"
>
>    db> other attached packages:
>    db> RMySQL         DBI        lme4      Matrix     lattice
>    db> "0.6-0"     "0.2-3" "0.99875-2" "0.99875-2"   "0.15-11"
>
>    db> ______________________________________________
>    db> R-devel at r-project.org mailing list
>    db> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595



[Rd] conflict between lme4 and RMySQL packages (PR#9753)
Prof Brian Ripley