From murdoch@dunc@n @ending from gm@il@com  Wed May  9 14:52:19 2018
From: murdoch@dunc@n @ending from gm@il@com (Duncan Murdoch)
Date: Wed, 9 May 2018 08:52:19 -0400
Subject: [Rd] 
 download.file does not process gz files correctly (truncates them?)
In-Reply-To: <CABdHhvHpdywicYS17PaPho46V12F4SGFKVB0aRZuH4WbKTe57A@mail.gmail.com>
References: <CAO1zAVbXOY-8PSCeHj5DRGj=MOoL5OmMX-ZcdY-P-yU6yVXVXw@mail.gmail.com>
 <CAFDcVCTcwYdpBayHAUuvncZ00RWTHtLGujf6-8_EUOLanQtOuQ@mail.gmail.com>
 <CABFfbXvBWLiFjCbpGtDN_HwRmYmh5Hpy6SFCfaS1meQmwC5hyw@mail.gmail.com>
 <CAO1zAVaGZYKFt12MhvUJFG_UjzkFnTToJjD2S6Mkpq=nLwdjAg@mail.gmail.com>
 <CAFDcVCSqj1woeDOdJmgP2rRwEpgTk0ZagFrHAUNkwRvk1cXnKQ@mail.gmail.com>
 <0a82df6b-556f-41ae-101f-8a95eaa64f75@gmail.com>
 <CABdHhvE-H68updR0gs49h4bLYVW_WT6e+An41WmHKCP=-jVWtg@mail.gmail.com>
 <CABdHhvHpdywicYS17PaPho46V12F4SGFKVB0aRZuH4WbKTe57A@mail.gmail.com>
Message-ID: <0b8d19ea-7529-bc4e-3895-4cd16f74e535@gmail.com>

On 08/05/2018 4:47 PM, Hadley Wickham wrote:
> On Tue, May 8, 2018 at 8:15 AM, Hadley Wickham <h.wickham at gmail.com> wrote:
>> On Thu, May 3, 2018 at 11:34 PM, Tomas Kalibera
>> <tomas.kalibera at gmail.com> wrote:
>>> On 05/03/2018 11:14 PM, Henrik Bengtsson wrote:
>>>>
>>>> Also, as mentioned in my
>>>> https://stat.ethz.ch/pipermail/r-devel/2012-August/064739.html, when
>>>> not specifying the mode argument, the default on Windows is mode = "w"
>>>> *except* for certain, case-sensitive, filename extensions:
>>>>
>>>>       if(missing(mode) && length(grep("\\.(gz|bz2|xz|tgz|zip|rda|RData)$",
>>>> url)))
>>>>           mode <- "wb"
>>>>
>>>> Just like the need for mode = "wb" on Windows, the above
>>>> special-file-extension-hack is only happening on Windows, and is only
>>>> documented in ?download.file if you're on Windows; so someone who's on
>>>> Linux/macOS trying to help someone on Windows may not be aware of
>>>> this. This adds to even more confusions, e.g. "works for me".
>>>
>>> If we were designing the API today, it would probably make more sense not to
>>> convert any line endings by default. Today's editors _usually_ can cope with
>>> different line endings and it is probably easier to detect that a text file
>>> has incorrect line endings rather than detecting that a binary file has been
>>> corrupted by an attempt to convert line endings. But whether to change
>>> existing, documented behavior is a different question. In order to help
>>> users and programmers who do not read the documentation carefully we would
>>> create problems for users and programmers who do. The current heuristic/hack
>>> is in line with the compatibility approach: it detects files that are
>>> obviously binary, so it changes the default behavior only for cases when it
>>> would obviously cause damage.
>>
>>  From a purely utilitarian standpoint, there are far more users who do
>> not carefully read the documentation than users who do ;)
>>
>> (I'd also argue that basing the decision on the file extension is
>> suboptimal, and it would be better to use the mime type if provided by
>> the server)
> 
> Also note that MS just announced support for unix line endings in notepad
> 
> https://blogs.msdn.microsoft.com/commandline/2018/05/08/extended-eol-in-notepad/

Perhaps soon RStudio will follow Notepad's lead, and not convert line 
endings when it saves a non-native file.

Duncan Murdoch



[Rd] download.file does not process gz files correctly (truncates them?)
Duncan Murdoch