From luke-tierney at uiowa.edu  Sat Dec 16 21:09:50 2017
From: luke-tierney at uiowa.edu (luke-tierney at uiowa.edu)
Date: Sat, 16 Dec 2017 14:09:50 -0600 (CST)
Subject: [Rd] cannot destroy connection (?) created by readLines in a
 tryCatch
In-Reply-To: <CABtg=KkADSVkwsEN=nm_8Xs5eONABqfy9fZwVW__wcmzP_jS5Q@mail.gmail.com>
References: <CABtg=KnWVVSvq6dogNG9fnYWhTJNNadJcyaMDif0_EV7Gd_VaA@mail.gmail.com>
 <CADwqtCPwpURQqLs0NrX7Hg9CtEX4huLj+Z=DTKvzTb9bHX=vbA@mail.gmail.com>
 <CABtg=KnTE+5n-4wQgty0yMVr1_vuUhvYgJ2fJGux143Pz-T0bQ@mail.gmail.com>
 <alpine.OSX.2.20.1712141746070.1706@lukes-macbook-air.local>
 <CABtg=KkADSVkwsEN=nm_8Xs5eONABqfy9fZwVW__wcmzP_jS5Q@mail.gmail.com>
Message-ID: <alpine.DEB.2.20.1712161407190.8786@luke-Latitude>

On Fri, 15 Dec 2017, G?bor Cs?rdi wrote:

> Thanks for tracking this down. Yeah, I should use suppressWarnings(),
> you are right.
> Although, readLines() might throw another warning, e.g. for incomplete
> last lines,

Do whatever makes sense for your context.

My main point is: if you want your computation to be able to handle a
warning and possibly continue after that then you need to use a
calling handler, not an exiting one.

Best,

luke

> and you don't necessarily want to suppress that.
>
> TBH I am not sure why that warning is given:
>
> ? con <- file(tempfile())
> ? open(con)
> Error in open.connection(con) : cannot open the connection
> In addition: Warning message:
> In open.connection(con) :
>  cannot open file
> '/var/folders/59/0gkmw1yj2w7bf2dfc3jznv5w0000gn/T//RtmpilJLXL/filed0ab5adb9a18':
> No such file or directory
>
> It seems that open() also throws an error, so why give the warning?
> Because it is more specific?
> Would it make sense to turn that warning into an error?
>
> Gabor
>
> On Thu, Dec 14, 2017 at 11:46 PM,  <luke-tierney at uiowa.edu> wrote:
>> This has nothing to do with on.exit. It is an iteraction between where
>> the warning is signaled in 'file' and your _exiting_ warning handler.
>> This combination has the same issue,
>>
>> tryCatch(file(tempfile(), "r"), warning = identity)
>> showConnections(all = TRUE)
>>
>> as does
>>
>> options(warn=2)
>> file(tempfile(), "r")
>> showConnections(all = TRUE)
>>
>> I haven't looked at the internals of 'file' but it looks like
>> what it does is
>>
>>     add an entry to connections table
>>     warn about non-existent file
>>     realize it has to fail
>>     remove the connections table entry
>>     signal an error
>>
>> This misses the possibility that the warning can result in a jump
>> if it is turned into a error or handled by an exiting handler.
>> It's worth filing a bug report on 'file'.
>>
>> It's not clear what you are really trying to do, but establishing
>> an _exiting_ handler for warnings is usually not what you
>> want. If you are trying to suppress warnings you need to use a
>> calling handler, e.g. via suppressWarnings. If you want to do
>> something more sophisticated that does not terminate the
>> computation on a warniing you can build on what suppressWarnigns
>> does.
>>
>> Best,
>>
>> luke
>>
>>
>> On Thu, 14 Dec 2017, G?bor Cs?rdi wrote:
>>
>>> On Thu, Dec 14, 2017 at 7:56 PM, Gabriel Becker <gmbecker at ucdavis.edu>
>>> wrote:
>>>>
>>>> Gabor,
>>>>
>>>> You can grab the connection and destroy it via getConnection and then a
>>>> standard close call.
>>>
>>>
>>> Yeah, that's often a possible workaround, but since this connection
>>> was opened by
>>> readLines() internally, I don't necessarily know which one it is. E.g.
>>> I might open multiple
>>> connections to the same file, so I can't choose based on the file name.
>>>
>>> Btw. this workaround seems to work for me:
>>>
>>> read_lines <- function(con, ...) {
>>>  if (is.character(con)) {
>>>    con <- file(con)
>>>    on.exit(close(con))
>>>  }
>>>  readLines(con, ...)
>>> }
>>>
>>> This is basically the same as readLines(), but on.exit() does its job
>>> here.
>>> That's another clue that it might be an on.exit() issue. Wild guess:
>>> on.exit() does not run if an internal function errors.
>>>
>>>> (it actually lists that it is "closed" already, but
>>>> still in the set of existing connections. I can't speak to that
>>>> difference).
>>>
>>>
>>> It is closed but not destroyed.
>>>
>>> G.
>>>
>>>>> tryCatch(
>>>>
>>>>
>>>> +   readLines(tempfile(), warn = FALSE)[1],
>>>>
>>>> +   error = function(e) NA,
>>>>
>>>> +   warning = function(w) NA
>>>>
>>>> + )
>>>>
>>>> [1] NA
>>>>
>>>>> rm(list=ls(all.names = TRUE))
>>>>
>>>>
>>>>> gc()
>>>>
>>>>
>>>>          used (Mb) gc trigger (Mb) max used (Mb)
>>>>
>>>> Ncells 257895 13.8     592000 31.7   416371 22.3
>>>>
>>>> Vcells 536411  4.1    8388608 64.0  1795667 13.7
>>>>
>>>>>
>>>>
>>>>> showConnections(all = TRUE)
>>>>
>>>>
>>>>   description
>>>>
>>>> 0 "stdin"
>>>>
>>>> 1 "stdout"
>>>>
>>>> 2 "stderr"
>>>>
>>>> 3
>>>>
>>>> "/var/folders/79/l_n_5qr152d2d9d9xs0591lh0000gn/T//RtmpZRcxmh/file128a13bffc77"
>>>>
>>>>   class      mode text   isopen   can read can write
>>>>
>>>> 0 "terminal" "r"  "text" "opened" "yes"    "no"
>>>>
>>>> 1 "terminal" "w"  "text" "opened" "no"     "yes"
>>>>
>>>> 2 "terminal" "w"  "text" "opened" "no"     "yes"
>>>>
>>>> 3 "file"     "r"  "text" "closed" "yes"    "yes"
>>>>
>>>>> con = getConnection(3)
>>>>
>>>>
>>>>> con
>>>>
>>>>
>>>> A connection with
>>>>
>>>> description
>>>>
>>>> "/var/folders/79/l_n_5qr152d2d9d9xs0591lh0000gn/T//RtmpZRcxmh/file128a13bffc77"
>>>>
>>>> class       "file"
>>>>
>>>> mode        "r"
>>>>
>>>> text        "text"
>>>>
>>>> opened      "closed"
>>>>
>>>> can read    "yes"
>>>>
>>>> can write   "yes"
>>>>
>>>>> close(con)
>>>>
>>>>
>>>>> showConnections(all=TRUE)
>>>>
>>>>
>>>>   description class      mode text   isopen   can read can write
>>>>
>>>> 0 "stdin"     "terminal" "r"  "text" "opened" "yes"    "no"
>>>>
>>>> 1 "stdout"    "terminal" "w"  "text" "opened" "no"     "yes"
>>>>
>>>> 2 "stderr"    "terminal" "w"  "text" "opened" "no"     "yes"
>>>>
>>>>
>>>>
>>>> HTH,
>>>> ~G
>>>>
>>>> On Thu, Dec 14, 2017 at 10:02 AM, G?bor Cs?rdi <csardi.gabor at gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> Consider this code. This is R 3.4.2, but based on a quick look at the
>>>>> NEWS, this has not been fixed.
>>>>>
>>>>> tryCatch(
>>>>>   readLines(tempfile(), warn = FALSE)[1],
>>>>>   error = function(e) NA,
>>>>>   warning = function(w) NA
>>>>> )
>>>>>
>>>>> rm(list=ls(all.names = TRUE))
>>>>> gc()
>>>>>
>>>>> showConnections(all = TRUE)
>>>>>
>>>>> If you run it, you'll get a connection you cannot close(), i.e. the
>>>>> last showConnections() call prints:
>>>>>
>>>>> ? showConnections(all = TRUE)
>>>>>   description
>>>>> 0 "stdin"
>>>>> 1 "stdout"
>>>>> 2 "stderr"
>>>>> 3
>>>>>
>>>>> "/var/folders/59/0gkmw1yj2w7bf2dfc3jznv5w0000gn/T//Rtmpc7JqVS/filecc2044b2ccec"
>>>>>   class      mode text   isopen   can read can write
>>>>> 0 "terminal" "r"  "text" "opened" "yes"    "no"
>>>>> 1 "terminal" "w"  "text" "opened" "no"     "yes"
>>>>> 2 "terminal" "w"  "text" "opened" "no"     "yes"
>>>>> 3 "file"     "r"  "text" "closed" "yes"    "yes"
>>>>>
>>>>> AFAICT, readLines should close the connection:
>>>>>
>>>>> ? readLines
>>>>> function (con = stdin(), n = -1L, ok = TRUE, warn = TRUE, encoding =
>>>>> "unknown",
>>>>>     skipNul = FALSE)
>>>>> {
>>>>>     if (is.character(con)) {
>>>>>         con <- file(con, "r")
>>>>>         on.exit(close(con))
>>>>>     }
>>>>>     .Internal(readLines(con, n, ok, warn, encoding, skipNul))
>>>>> }
>>>>> <environment: namespace:base>
>>>>>
>>>>> so maybe this just a symptom of an on.exit() issue?
>>>>>
>>>>> Or am I missing something and it is possible to close the connection?
>>>>>
>>>>> Thanks,
>>>>> Gabor
>>>>>
>>>>> ______________________________________________
>>>>> R-devel at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Gabriel Becker, PhD
>>>> Scientist (Bioinformatics)
>>>> Genentech Research
>>>
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>> --
>> Luke Tierney
>> Ralph E. Wareham Professor of Mathematical Sciences
>> University of Iowa                  Phone:             319-335-3386
>> Department of Statistics and        Fax:               319-335-3017
>>    Actuarial Science
>> 241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
>> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu


[Rd] cannot destroy connection (?) created by readLines in a tryCatch
luke-tierney at uiowa.edu