From edd @end|ng |rom deb|@n@org  Fri Aug  2 16:24:07 2019
From: edd @end|ng |rom deb|@n@org (Dirk Eddelbuettel)
Date: Fri, 2 Aug 2019 09:24:07 -0500
Subject: [Rd] Infrequent but steady NULL-pointer caused segfault in
 as.POSIXlt.POSIXct (R 3.4.4)
In-Reply-To: <CAL109eM4576ye7Jo9LUfPqncMSLb1Bg4XzMiC7e=w+h7ig4usw@mail.gmail.com>
References: <CAL109eM4576ye7Jo9LUfPqncMSLb1Bg4XzMiC7e=w+h7ig4usw@mail.gmail.com>
Message-ID: <23876.18311.906487.977359@rob.eddelbuettel.com>


On 2 August 2019 at 16:23, Sun Yijiang wrote:
| The R script I run daily for hours looks like this:
| 
| while (!finish) {
|     Sys.sleep(0.1)
|     time = as.integer(format(Sys.time(), "%H%M")) # always crash here
|     if (new.data.timestamp() <= time)
|         next
|     # ... do some jobs for about 2 minutes ...
|     gc()
| }
| 
| Basically it waits for new data, which comes in every 10 minutes, and
| do some jobs, then gc(), then loop again.  It works great most of the
| time, but crashes strangely once a month or so.  Although infrequent,
| it always crashes at the same place and gives the same error info,
| like this:

A really long time ago in a galaxy not too far away I also shephearded such
jobs where the job were left running for a long time from a single "outer"
call, and aimed to control resources via gc(), Sys.sleep()and alike.

Doing that, I learned a different session.  As R is an fact _an environment_
taking care of a great many things, robustness can be had more easily via
fresh processes.  Since then I mostly control the jobs _from the outside_ via
cron, and aim to have well-defined taks (with few dependencies, a different
topic).  I do the same for e.g. package testing and loading/unloading of
shared libraries: hard to get right, easier to test in vanilla sessions.

These are also some of the reasons I joined Jeff Horner in his then-nascient
littler project, and continue to look after it. I run many (automated) jobs
with it from cron, and this generally works _great_.  Rscript came just a
little bit later, has also improved over time and is a fine alternative.

Lastly, I would recommend to take advantage of the fact that POSIXct is in
fact numeric internally and rewrite this as

     currentTime <- as.numeric(Sys.time())
     if (currentTime > prevTime + minDelta) {
        # do stuff ...
        prevTime <- currentTime
     }

That alone may help you as you no longer need all the temporary string
objects you were chasing through the debugger.

Good luck, and keep re-engineering your scripts. R can be a very, very
reliable tool when driven the right way.

Cheers, Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org



[Rd] Infrequent but steady NULL-pointer caused segfault in as.POSIXlt.POSIXct (R 3.4.4)
Dirk Eddelbuettel