From nsosnov at microsoft.com  Tue Jun 27 17:51:15 2017
From: nsosnov at microsoft.com (Nathan Sosnovske)
Date: Tue, 27 Jun 2017 15:51:15 +0000
Subject: [Rd] suggestion to fix packageDescription() for Windows users
In-Reply-To: <22866.13492.183329.958202@stat.math.ethz.ch>
References: <1d01ccce-cb18-30d9-9d93-644ed4dbd995@uw.edu>
 <70548b39-a0ee-80d5-302f-2433a700192d@gmail.com>
 <6b7f9592-1a53-37dd-f8ae-fe805dcbc337@uw.edu>
 <7e843534-d9f2-9cac-1c53-7c2e8b687c39@gmail.com>
 <CA+E84Q4SKEHOA7Gy_wCZs_9twTmzE-M=EeUDABshmREyAutyNg@mail.gmail.com>
 <08f33a36-329c-b40a-202a-80f2e61f53a3@gmail.com>
 <SN1PR21MB0061EB36C510723541CC25B7B7D80@SN1PR21MB0061.namprd21.prod.outlook.com>
 <CY4PR21MB04713D49F2DB7C9BFC2E2A56DED90@CY4PR21MB0471.namprd21.prod.outlook.com>
 <SN1PR21MB0061957F2745E5277E239C69B7DF0@SN1PR21MB0061.namprd21.prod.outlook.com>
 <22866.13492.183329.958202@stat.math.ethz.ch>
Message-ID: <BY2PR21MB0052E33F1F450FFB9005CF10B7DC0@BY2PR21MB0052.namprd21.prod.outlook.com>

> *) Of course I'd hoped you'd find why the underlying
> packageDescription() function is not "getting the right thing" in this case directly -- in Windows only in some locales -- and provide a Windows-only patch for the underlying problem there, rather than the workaround patch in citation().
> The patch does solve the problem at hand, alright, so thank you, Rich and Nathan!

This makes sense. I was asking if we wanted to proceed with the workaround-esque patch Rich suggested because I was trying to understand what was more useful to the community: diving deeper into the issue to see what was going on, or spending that time on another issue that was symptomatic. Either way, I think it makes sense to fix the underlying issue at some point and I do have time set aside this week to do so.

> so ideally almost all of this follow up should have happened there.

Apologies about this. I'm still learning and appreciate the gentle correction. ?

-----Original Message-----
From: Martin Maechler [mailto:maechler at stat.math.ethz.ch] 
Sent: Tuesday, June 27, 2017 3:34 AM
To: Nathan Sosnovske <nsosnov at microsoft.com>
Cc: Rich Calaway <richcala at microsoft.com>; Duncan Murdoch <murdoch.duncan at gmail.com>; Andrie de Vries <apdevries at gmail.com>; R-devel Mailing List (r-devel at r-project.org) <r-devel at r-project.org>; Ben Marwick <bmarwick at uw.edu>; Martin Maechler <maechler at stat.math.ethz.ch>
Subject: Re: [Rd] suggestion to fix packageDescription() for Windows users

>>>>> Nathan Sosnovske via R-devel <r-devel at r-project.org>
>>>>>     on Mon, 26 Jun 2017 18:22:25 +0000 writes:

    > I'd be curious to know what others think of Rich's
    > patch. If it is acceptable, I can spend time that I was
    > going to look at it this week on another bug. 

It is a bit kludgy (*) of course, but I confirm it solves the problem in a "robust" way.

*) Of course I'd hoped you'd find why the underlying
packageDescription() function is not "getting the right thing" in this case directly -- in Windows only in some locales -- and provide a Windows-only patch for the underlying problem there, rather than the workaround patch in citation().
The patch does solve the problem at hand, alright, so thank you, Rich and Nathan!

Note that Duncan Murdoch did mention in this thread to file an official bug report and Ben Marwick gave the URL

> From: Ben Marwick ...
> Subject: Re: [Rd] suggestion to fix packageDescription() for Windows 
> users
> Date: Sun, 18 Jun 2017 08:34:56 +1000

> Thanks very much, I see your bug report here:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.
> r-project.org%2Fbugzilla3%2Fshow_bug.cgi%3Fid%3D17291&data=02%7C01%7Cn
> sosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86
> f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=AFG8tyP5Maqc
> iZwYFqBZ4wylbVJAoyWu4kASsxFZr%2F4%3D&reserved=0

so ideally almost all of this follow up should have happened there.
I have followed up there and also there attached a Windows-only
+ commented version of Rich's patch. As mentioned, I've tested
it and confirmed to work for the use case in the mean time, so plan to commit soon.

This will be too late for the release of R 3.4.1 tomorrow, of course [code freeze was on June 23].

Martin Maechler
ETH Zurich

    > -----Original Message-----
    > From: Rich Calaway 
    > Sent: Friday, June 23, 2017 6:34 PM
    > To: Nathan Sosnovske <nsosnov at microsoft.com>; Duncan Murdoch <murdoch.duncan at gmail.com>; Andrie de Vries <apdevries at gmail.com>
    > Cc: Ben Marwick <bmarwick at uw.edu>; R-devel Mailing List (r-devel at r-project.org) <r-devel at r-project.org>
    > Subject: RE: [Rd] suggestion to fix packageDescription() for Windows users

    > The following patch is not the most elegant, but it restores the Authors when "LC_CTYPE" is set to either "Chinese" or "Arabic":

    >> Sys.setlocale("LC_CTYPE", "Chinese")
    > [1] "Chinese (Simplified)_China.936"
    >> citation("readr")

    > To cite package ?readr? in publications use:

    > (2016). readr: Read Tabular Data. R package version 1.0.0.
    > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0

    > A BibTeX entry for LaTeX users is

    > @Manual{,
    > title = {readr: Read Tabular Data},
    > year = {2016},
    > note = {R package version 1.0.0},
    > url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0},
    > }

    > ATTENTION: This citation information has been auto-generated from the package DESCRIPTION file and may need manual editing, see ?help("citation")?.

    >> Sys.setlocale("LC_CTYPE", "Arabic")
    > [1] "Arabic_Saudi Arabia.1256"
    >> citation("readr")

    > To cite package ?readr? in publications use:

    > (2016). readr: Read Tabular Data. R package version 1.0.0.
    > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0

    > A BibTeX entry for LaTeX users is

    > @Manual{,
    > title = {readr: Read Tabular Data},
    > year = {2016},
    > note = {R package version 1.0.0},
    > url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0},
    > }

    > ATTENTION: This citation information has been auto-generated from the package DESCRIPTION file and may need manual editing, see ?help("citation")?.

    >> citation <- newCitation
    >> citation("readr")

    > To cite package ?readr? in publications use:

    > Hadley Wickham, Jim Hester and Romain Francois (2016). readr: Read
    > Tabular Data. R package version 1.0.0.
    > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0

    > A BibTeX entry for LaTeX users is

    > @Manual{,
    > title = {readr: Read Tabular Data},
    > author = {Hadley Wickham and Jim Hester and Romain Francois},
    > year = {2016},
    > note = {R package version 1.0.0},
    > url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ce3ed6ea675b44c908c9208d4bd4817dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636341564724082662&sdata=C91aZ4qu2CT56qCrqFFBshREabZzfkaLXAABgWQSvXg%3D&reserved=0},
    > }



    > The patch is:

    > Index: citation.R
    > ===================================================================
    > --- citation.R	(revision 72852)
    > +++ citation.R	(working copy)
    > @@ -1162,8 +1162,11 @@
    > if(dir == "")
    > stop(gettextf("package %s not found", sQuote(package)),
    > domain = NA)
    > -        meta <- packageDescription(pkg = package,
    > -                                   lib.loc = dirname(dir))
    > +	    args <- list(pkg = package, lib.loc = dirname(dir))
    > +	    if (!is.na(enc <- packageDescription(pkg = package, lib.loc=dirname(dir), field="Encoding")))
    > +		    args$enc <- enc
    > +        meta <- do.call("packageDescription", args=args)
    > +
    > ## if(is.null(auto)): Use default auto-citation if no CITATION
    > ## available.
    > citfile <- file.path(dir, "CITATION")


    > Nathan says he can look into this further next week...

    > Cheers,

    > Rich Calaway
    > Microsoft R Product Team
    > 24/1341
    > +1 (425) 4219919 X19919

    > -----Original Message-----
    > From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of Nathan Sosnovske via R-devel
    > Sent: Friday, June 23, 2017 7:36 AM
    > To: Duncan Murdoch <murdoch.duncan at gmail.com>; Andrie de Vries <apdevries at gmail.com>
    > Cc: r-devel at r-project.org; Ben Marwick <bmarwick at uw.edu>
    > Subject: Re: [Rd] suggestion to fix packageDescription() for Windows users

    > Hi Duncan,

    > I'm guessing I'll be able to look at this over the weekend/next week (probably closer to next week). It is on my list of things to do and I've just had a few other prior commitments that I have to finish first.

    > Sorry for the delay. I'll chime in with a status update next week.

    > Nathan

    > -----Original Message-----
    > From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of Duncan Murdoch
    > Sent: Friday, June 23, 2017 5:16 AM
    > To: Andrie de Vries <apdevries at gmail.com>
    > Cc: r-devel at r-project.org; Ben Marwick <bmarwick at uw.edu>
    > Subject: Re: [Rd] suggestion to fix packageDescription() for Windows users

    > On 18/06/2017 5:57 AM, Andrie de Vries wrote:
    >> Hi, Duncan
    >> 
    >> i have forwarded this thread to Nathan, who promised to look into it.

    > Any progress on this?

    > Duncan Murdoch

    >> 
    >> Andrie
    >> 
    >> On 17 Jun 2017 17:26, "Duncan Murdoch" <murdoch.duncan at gmail.com 
    >> <mailto:murdoch.duncan at gmail.com>> wrote:
    >> 
    >> On 17/06/2017 9:13 AM, Ben Marwick wrote:
    >> 
    >> Hi Duncan,
    >> 
    >> Thanks for your reply. Yes, it does seem to be specific to the CTYPE
    >> setting to Chinese on Windows. If I set it to English using
    >> Sys.setlocale() there is no problem, then back to Chinese and the
    >> authors disappear:
    >> 
    >> Sys.setlocale("LC_ALL","English")
    >> citation("readr")
    >> 
    >> 
    >> Thanks, that makes the problem reproducible.  I'll submit it as a
    >> bug report.  Maybe someone from Microsoft will fix it.
    >> 
    >> Duncan Murdoch

    [.........]


[Rd] suggestion to fix packageDescription() for Windows users
Nathan Sosnovske