From csardi.gabor at gmail.com  Fri Nov 11 14:12:33 2016
From: csardi.gabor at gmail.com (=?UTF-8?B?R8OhYm9yIENzw6FyZGk=?=)
Date: Fri, 11 Nov 2016 13:12:33 +0000
Subject: [Rd] Memory leak with tons of closed connections
In-Reply-To: <CAPvvxJU2v9XaxtUp1Nt9W5dEhRLKEvtZQpHfh=cXRMCKkYL8Zg@mail.gmail.com>
References: <CAPvvxJV3922JWG--M3CPFMu5n4_+nN+CU7umaRKQ8yJPF5t=dw@mail.gmail.com>
	<22565.42645.436216.547440@stat.math.ethz.ch>
	<CAPvvxJU2v9XaxtUp1Nt9W5dEhRLKEvtZQpHfh=cXRMCKkYL8Zg@mail.gmail.com>
Message-ID: <CABtg=K=FrfpKi67N-ChH6HuuEqY6c-4ymQ9+YCRtbSx8CsUopA@mail.gmail.com>

On Fri, Nov 11, 2016 at 12:46 PM, Gergely Dar?czi
<daroczig at rapporter.net> wrote:
[...]
>> I've changed the above to *print* the gc() result every 1000th
>> iteration, and after 100'000 iterations, there is still no
>> memory increase from the point of view of R itself.

Yes, R does not know about it, it does not manage this memory (any
more), but the R process requested this memory from the OS, and never
gave it back, which is basically the definition of a memory leak. No?

I think the leak is because 'stdin' is special and R opens it with fdopen():
https://github.com/wch/r-source/blob/f8cdadb769561970cc42776f563043ea5e12fe05/src/main/connections.c#L561-L579

and then it does not close it:
https://github.com/wch/r-source/blob/f8cdadb769561970cc42776f563043ea5e12fe05/src/main/connections.c#L636

I understand that R cannot fclose the FILE*, because that would also
close the file descriptor, but anyway, this causes a memory leak. I
think.

It seems that you cannot close the FILE* without closing the
descriptor, so maybe a workaround would be to keep one FILE* open,
instead of calling fdopen() to create new ones every time. Another
possible workaround is to use dup(), but I don't know enough about the
details to be sure.

Gabor

[...]



[Rd] Memory leak with tons of closed connections
=?UTF-8?B?R8OhYm9yIENzw6FyZGk=?=