From konr@d@rudo|ph @end|ng |rom gm@||@com  Wed Oct  9 21:22:13 2019
From: konr@d@rudo|ph @end|ng |rom gm@||@com (Konrad Rudolph)
Date: Wed, 9 Oct 2019 20:22:13 +0100
Subject: [Rd] S3 lookup rules changed in R 3.6.1
Message-ID: <CAM2gKPZfigRJL1yfyMq-hC3K6QVcswx05NM8HaJ41HBzom4fOQ@mail.gmail.com>

tl;dr: S3 lookup no longer works in custom non-namespace environments as of
R 3.6.1. Is this a bug?

I am implementing S3 dispatch for generic methods in environments that are
not
packages. I am trying to emulate the R package namespace mechanism by
having a
?namespace? environment that defines generics and methods, but only exposes
the
generics themselves, not the methods.

To make S3 lookup work when using the generics, I am using
`registerS3method`.
While this method itself has no extensive documentation, the documentation
of
`UseMethod` contains this relevant passage:

> Namespaces can register methods for generic functions. To support this,
> ?UseMethod? and ?NextMethod? search for methods in two places: in the
> environment in which the generic function is called, and in the
registration
> data base for the environment in which the generic is defined (typically a
> namespace). So methods for a generic function need to be available in the
> environment of the call to the generic, or they must be registered. (It
does
> not matter whether they are visible in the environment in which the
generic is
> defined.) As from R 3.5.0, the registration data base is searched after
the
> top level environment (see ?topenv?) of the calling environment (but
before
> the parents of the top level environment).

This used to work but it stopped working in R 3.6.1 and I cannot figure out
(a)
why, and (b) how to fix it. Unfortunately I am unable to find the relevant
information by reading the R source code, even when ?diff?ing what seem to
be
the only even remotely relevant changes [1].

The R NEWS merely list the following change for R 3.6.0:

> * S3method() directives in ?NAMESPACE? can now also be used to perform
delayed
>   S3 method registration.
> [?]
> * Method dispatch uses more relevant environments when looking up class
>   definitions.

Unfortunately it is not clear to me what exactly this means.

Here?s a minimal example code that works under R 3.5.3 but breaks under
R 3.6.1
(I don?t know about 3.6.0).

```
# Define ?package namespace?:
ns = new.env(parent = .BaseNamespaceEnv)
local(envir = ns, {
    test = function (x) UseMethod('test')
    test.default = function (x) message('test.default')
    test.foo = function (x) message('test.foo')

    .__S3MethodsTable__. = new.env(parent = .BaseNamespaceEnv)
    .__S3MethodsTable__.$test.default = test.default
    .__S3MethodsTable__.$test.foo = test.foo

    # Or, equivalently:
    # registerS3method('test', 'default', test.default)
    # registerS3method('test', 'foo', test.foo)
})

# Expose generic publicly:
test = ns$test

# Usage:
test(1)
test(structure(1, class = 'foo'))
```

Output in R up to 3.5.3:

```
test.default
test.foo
```

Output in R 3.6.1:

```
Error in UseMethod("test") :
  no applicable method for 'test' applied to an object of class
"c('double', 'numeric')"
```

It?s worth noting that the output of `.S3methods` is the same for all R
versions, and from my understanding of its output, this *should* indicate
that
S3 lookup should behave identically, too. Furthermore, lookup via
`getS3method`
succeeds in all R versions, and (again, in my understanding) the logic of
this
function should be identical to the logic of R?s internal S3 dispatch:

```
getS3method('test', 'default')(1)
getS3method('test', 'foo')(1)
```

Conversely, specialising an existing generic from a loaded package works.
E.g.:

```
local(envir = ns, {
    print.foo = function (x) message('print.foo')
    registerS3method('print', 'foo', print.foo)
})

print(structure(1, class = 'foo'))
```

This prints ?print.foo? in all R versions as expected.

So my question is: Why do the `test(?)` calls in R 3.6.1 no longer trigger
S3
method lookup in the generic function?s environment? Is this behaviour by
design
or is it a bug? If it?s by design, why does `getS3method` still use the old
behaviour? And, most importantly, how can I fix my definition of `ns` to
make
S3 dispatch for non-exposed methods work again?

? actually I just found a workaround:

```
ns$.packageName = 'not important'
```

This marks `ns` as a package namespace. To me, the documentation seems to
imply
that this shouldn?t be necessary (and it previously wasn?t). Furthermore,
the
code for `registerS3method` explicitly supports non-package namespace
environments. Unfortunately this workaround is not satisfactory because
pretending that the environment is a package namespace, when it really
isn?t,
might break other things.

[1] See r75273; there?s also r74625, which changes the actual lookup
mechanism
    used by `UseMethod`, but that seems even less relevant, because it is
    disabled unless a specific environment variable is set.

-- 
Konrad Rudolph

	[[alternative HTML version deleted]]



[Rd] S3 lookup rules changed in R 3.6.1
Konrad Rudolph