From nsosnov at microsoft.com  Fri Jun 23 16:35:42 2017
From: nsosnov at microsoft.com (Nathan Sosnovske)
Date: Fri, 23 Jun 2017 14:35:42 +0000
Subject: [Rd] suggestion to fix packageDescription() for Windows users
In-Reply-To: <08f33a36-329c-b40a-202a-80f2e61f53a3@gmail.com>
References: <1d01ccce-cb18-30d9-9d93-644ed4dbd995@uw.edu>
 <70548b39-a0ee-80d5-302f-2433a700192d@gmail.com>
 <6b7f9592-1a53-37dd-f8ae-fe805dcbc337@uw.edu>
 <7e843534-d9f2-9cac-1c53-7c2e8b687c39@gmail.com>
 <CA+E84Q4SKEHOA7Gy_wCZs_9twTmzE-M=EeUDABshmREyAutyNg@mail.gmail.com>
 <08f33a36-329c-b40a-202a-80f2e61f53a3@gmail.com>
Message-ID: <SN1PR21MB0061EB36C510723541CC25B7B7D80@SN1PR21MB0061.namprd21.prod.outlook.com>

Hi Duncan,

I'm guessing I'll be able to look at this over the weekend/next week (probably closer to next week). It is on my list of things to do and I've just had a few other prior commitments that I have to finish first.

Sorry for the delay. I'll chime in with a status update next week.

Nathan

-----Original Message-----
From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of Duncan Murdoch
Sent: Friday, June 23, 2017 5:16 AM
To: Andrie de Vries <apdevries at gmail.com>
Cc: r-devel at r-project.org; Ben Marwick <bmarwick at uw.edu>
Subject: Re: [Rd] suggestion to fix packageDescription() for Windows users

On 18/06/2017 5:57 AM, Andrie de Vries wrote:
> Hi, Duncan
>
> i have forwarded this thread to Nathan, who promised to look into it.

Any progress on this?

Duncan Murdoch

>
> Andrie
>
> On 17 Jun 2017 17:26, "Duncan Murdoch" <murdoch.duncan at gmail.com 
> <mailto:murdoch.duncan at gmail.com>> wrote:
>
>     On 17/06/2017 9:13 AM, Ben Marwick wrote:
>
>         Hi Duncan,
>
>         Thanks for your reply. Yes, it does seem to be specific to the CTYPE
>         setting to Chinese on Windows. If I set it to English using
>         Sys.setlocale() there is no problem, then back to Chinese and the
>         authors disappear:
>
>         Sys.setlocale("LC_ALL","English")
>         citation("readr")
>
>
>     Thanks, that makes the problem reproducible.  I'll submit it as a
>     bug report.  Maybe someone from Microsoft will fix it.
>
>     Duncan Murdoch
>
>
>         #' To cite package ?readr? in publications use:
>         #'
>         #'   Hadley Wickham, Jim Hester and Romain Francois (2017).
>         readr: Read
>         #' Rectangular Text Data. R package version 1.1.1.
>         #' https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>         <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>
>         #'
>         #' A BibTeX entry for LaTeX users is
>         #'
>         #' @Manual{,
>         #'   title = {readr: Read Rectangular Text Data},
>         #'   author = {Hadley Wickham and Jim Hester and Romain Francois},
>         #'   year = {2017},
>         #'   note = {R package version 1.1.1},
>         #'   url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>         <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>},
>         #' }
>
>
>         Sys.setlocale("LC_CTYPE", "Chinese")
>         citation("readr")
>
>         #'
>         #' To cite package ?readr? in publications use:
>         #'
>         #'   (2017). readr: Read Rectangular Text Data. R package
>         version 1.1.1.
>         #' https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>         <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>
>         #'
>         #' A BibTeX entry for LaTeX users is
>         #'
>         #' @Manual{,
>         #'   title = {readr: Read Rectangular Text Data},
>         #'   year = {2017},
>         #'   note = {R package version 1.1.1},
>         #'   url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>         <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>},
>         #' }
>         #'
>         #' ATTENTION: This citation information has been auto-generated
>         from the
>         #' package DESCRIPTION file and may need manual editing, see
>         #' ?help("citation")?.
>
>         Where do we go from here? I do want to use the Chinese locale
>         with R on
>         Windows (and perhaps others do too), so switching the locale
>         isn't a fix.
>
>         Thanks,
>
>         Ben
>
>         On 17/06/2017 10:36 PM, Duncan Murdoch wrote:
>
>             On 17/06/2017 7:10 AM, Ben Marwick wrote:
>
>                 Recently I was trying to cite a package where the
>                 authors have ?
>                 and ? in their names. I found that on Windows the
>                 citation() function
>                 did not return the authors' names at all, but on Linux
>                 there was no
>                 problem (sessionInfos at the bottom):
>
>                 On Windows, no author names are returned:
>
>
>             I'm not seeing this.  You have fairly strange localization
>             settings; see
>             comments below.
>
>
>                 #---------------
>
>                  > citation("readr")
>
>                 To cite package ?readr? in publications use:
>
>                    (2017). readr: Read Rectangular Text Data. R package
>                 version 1.1.1.
>                    https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>                 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN
> .R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.co
> m%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQX
> UDbzD3E5EWRM%3D&reserved=0>
>
>                 A BibTeX entry for LaTeX users is
>
>                    @Manual{,
>                      title = {readr: Read Rectangular Text Data},
>                      year = {2017},
>                      note = {R package version 1.1.1},
>                      url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>                 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>},
>                    }
>
>                 ATTENTION: This citation information has been
>                 auto-generated from the
>                 package DESCRIPTION file and may need manual editing, see
>                 ?help("citation")?.
>                 #---------------
>
>                 On Linux we do see the author names:
>
>                 #---------------
>                  > citation("readr")
>
>                 To cite package ?readr? in publications use:
>
>                    Hadley Wickham, Jim Hester and Romain Francois
>                 (2017). readr:
>                    Read Rectangular Text Data. R package version 1.1.1.
>                    https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>                 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN
> .R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.co
> m%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db4
> 7%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQX
> UDbzD3E5EWRM%3D&reserved=0>
>
>                 A BibTeX entry for LaTeX users is
>
>                    @Manual{,
>                      title = {readr: Read Rectangular Text Data},
>                      author = {Hadley Wickham and Jim Hester and Romain
>                 Francois},
>                      year = {2017},
>                      note = {R package version 1.1.1},
>                      url = {https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0
>                 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2FCRAN.R-project.org%2Fpackage%3Dreadr&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=SnbAtaLB%2BUtjGEZkTXwTqnaJtiF3jQXUDbzD3E5EWRM%3D&reserved=0>},
>                    }
>                 #---------------
>
>                 This appears to be an OS-dependent encoding issue. The
>                 citation function
>                 does not take an encoding argument, so it's not possible
>                 to set the
>                 encoding at the point where that function is used. The
>                 citation function
>                 working with the packageDescription function, which does
>                 have an
>                 encoding argument, but the default is not useful for
>                 Windows when there
>                 is an encoding set in the DESCRIPTION of the package (in
>                 this case
>                 UTF-8).
>
>                 We can set the encoding argument in packageDescription
>                 so it works in
>                 Windows to give the authors as expected, but it is very
>                 inconvenient to
>                 generate citations directly from the output of this
>                 function. So I'd
>                 like to propose a solution this problem by changing one
>                 line in the
>                 packageDescription function, like so, from:
>
>                 #---------------
>                 if (missing(encoding) && Sys.getlocale("LC_CTYPE") == "C")
>                 #---------------
>
>                 to:
>
>                 #---------------
>                 if ((missing(encoding) && Sys.getlocale("LC_CTYPE") ==
>                 "C") |
>                 unname(Sys.info()['sysname']) == "Windows")
>                 #---------------
>
>                 If I understand correctly, that will force
>                 ASCII//TRANSLIT encoding when
>                 DESCRIPTION files are read by packageDescription() on
>                 Windows machines.
>                 The upside is that Windows users will get the authors in
>                 the package
>                 citation, unlike the current situation. The downside is
>                 that the exotic
>                 symbols in the authors' names are replaced with common
>                 ones that are
>                 similar.
>
>                 I think getting the citations to easily include the
>                 authors' names is
>                 pretty important, even if their names have exotic
>                 characters, so this is
>                 worth fixing. Is this edit to packageDescription the
>                 best way to solve
>                 this problem of exotic characters preventing the
>                 authors' names from
>                 showing on Windows?
>
>                 thanks,
>
>                 Ben
>
>
>
>
>                 Windows sessionInfo
>
>                 #---------------
>                  > sessionInfo()
>                 R version 3.4.0 Patched (2017-05-10 r72670)
>                 Platform: x86_64-w64-mingw32/x64 (64-bit)
>                 Running under: Windows 7 x64 (build 7601) Service Pack 
> 1
>
>                 Matrix products: default
>
>                 locale:
>                 [1] LC_COLLATE=English_Australia.1252
>                 [2] LC_CTYPE=Chinese (Simplified)_People's Republic of
>                 China.936
>                 [3] LC_MONETARY=English_Australia.1252
>                 [4] LC_NUMERIC=C
>                 [5] LC_TIME=English_Australia.1252
>
>
>             I don't know what English_Australia.1252 does that's
>             different from what
>             I use (English_Canada.1252), but the Chinese locale setting
>             could cause
>             trouble.  Could you try setting this (presumably in the
>             Windows control
>             panel) to be consistent?  You're using a much simpler
>             setting on Linux.
>
>             Duncan Murdoch
>
>
>                 attached base packages:
>                 [1] stats     graphics  grDevices utils     datasets
>                 methods   base
>
>                 loaded via a namespace (and not attached):
>                   [1] readr_1.1.1    compiler_3.4.0 R6_2.2.1       hms_0.3
>                 tools_3.4.0
>                   [6] tibble_1.3.3   yaml_2.1.14    Rcpp_0.12.11
>                  knitr_1.16
>                 rlang_0.1.1
>                 [11] fortunes_1.5-4
>                 #---------------
>
>                 Linux sessionInfo:
>
>                 #---------------
>                  > sessionInfo()
>                 R version 3.3.1 (2016-06-21)
>                 Platform: x86_64-pc-linux-gnu (64-bit)
>                 Running under: Ubuntu 16.10
>
>                 locale:
>                   [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>                   [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>                   [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>                   [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
>                   [9] LC_ADDRESS=C               LC_TELEPHONE=C
>                 [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
>                 attached base packages:
>                 [1] stats     graphics  grDevices utils     datasets
>                 methods   base
>
>                 loaded via a namespace (and not attached):
>                 [1] tools_3.3.1 yaml_2.1.14 knitr_1.16
>                 #---------------
>
>                 ______________________________________________
>                 R-devel at r-project.org <mailto:R-devel at r-project.org>
>                 mailing list
>                 https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-devel&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=8o%2FDxEx7SSzw9rTkmz0h5ztEUZ8f7wasDh9RQ7p2DK8%3D&reserved=0
>                 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat
> .ethz.ch%2Fmailman%2Flistinfo%2Fr-devel&data=02%7C01%7Cnsosnov%40micro
> soft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C636338169899157582&sdata=8o%2FDxEx7SSzw9rTkmz0h5ztE
> UZ8f7wasDh9RQ7p2DK8%3D&reserved=0>
>
>
>
>         ______________________________________________
>         R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list
>         https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-devel&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=8o%2FDxEx7SSzw9rTkmz0h5ztEUZ8f7wasDh9RQ7p2DK8%3D&reserved=0
>         
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat
> .ethz.ch%2Fmailman%2Flistinfo%2Fr-devel&data=02%7C01%7Cnsosnov%40micro
> soft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C636338169899157582&sdata=8o%2FDxEx7SSzw9rTkmz0h5ztE
> UZ8f7wasDh9RQ7p2DK8%3D&reserved=0>
>
>

______________________________________________
R-devel at r-project.org mailing list
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-devel&data=02%7C01%7Cnsosnov%40microsoft.com%7Ccf07101e770643227da008d4ba31aa85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338169899157582&sdata=8o%2FDxEx7SSzw9rTkmz0h5ztEUZ8f7wasDh9RQ7p2DK8%3D&reserved=0


[Rd] suggestion to fix packageDescription() for Windows users
Nathan Sosnovske