From i.costigan at me.com  Sun Oct  6 00:47:56 2013
From: i.costigan at me.com (Imanuel Costigan)
Date: Sun, 06 Oct 2013 09:47:56 +1100
Subject: [Rd] Possible POSIXlt / wday glitch & bugs.r-project.org status
In-Reply-To: <C1A6679C-91B6-4EAB-A630-EFD2ABA641CB@r-project.org>
References: <7C58F7EE-8786-45B4-B47A-7BC19A1CECCB@me.com>
	<CAE3=dmeXTKV=dmFD5PO30dK0spHzrb1k-F4ndiURp8_zEBxRng@mail.gmail.com>
	<3CF8C970-05CC-45C4-8CE0-E078C84121AF@me.com>
	<CAPPM_gQBb_B5ukeBoCkd8rqwya-LdvUS0LGd7eRDzR=Scz8ZpA@mail.gmail.com>
	<CA+MmmTKJRwWegc=yPHo0mZ-=arhkMpw3PvyEspYS7jeH86gFOQ@mail.gmail.com>
	<C1A6679C-91B6-4EAB-A630-EFD2ABA641CB@r-project.org>
Message-ID: <E29B6F11-2628-46E3-8A6D-0F450F021F21@me.com>

Thanks for the clarification Simon. Having a look at the code in datetime.c, I can't say that I envy its maintainer. Looks like there are a lot of "hacks" to get things working. 

For the time being, I am going to override the TZ environment variable to be UTC as at this time I don't need to worry about converting between time zones - although I will in the future.

Also - I can't submit a new bug to your website's hosting of R bugs. The "submit new bug" page doesn't load. I also get a certificate error when visiting your site's mirror.

Thanks!


On 06/10/2013, at 5:18 AM, Simon Urbanek <simon.urbanek at r-project.org> wrote:

> On Oct 5, 2013, at 4:51 PM, Sean O'Riordain <seanpor at acm.org> wrote:
> 
>> Some people (luckily not me anymore!) working with mortgages and
>> pensions need to calculate up to 40 years into the future for the
>> payment schedule.
>> 
> 
> Just to clarify since the Joshua's comment was ambiguous (and in part plain wrong) - R's POSIXct has no such limit since it doesn't use integers, so that is not really the issue here. As the original post suggested there may be a bug in handing some cases where conversions hit the system libraries (that may truncate to integers) and some cases may be worked around - and that remains to be investigated.
> 
> Cheers,
> Simon
> 
> 
> 
>> On 5 October 2013 02:37, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:
>>> On Fri, Oct 4, 2013 at 8:02 PM, Imanuel Costigan <i.costigan at me.com> wrote:
>>>> Thanks for the responses and quoting the timezone help file.
>>>> 
>>>> I am assuming that in order to determine the wday element of POSIXlt, R does the necessary calculations in Julian time (via POSIXct). Based on this excerpt from ?DateTimeClasses, it looks like R is responsible for determining time zones post 2037 (the example I gave was in 2038). So it could be an R issue.
>>>> 
>>> It's an issue with size of the largest number you can store in a
>>> signed integer, which is not specific to R.
>>> 
>>>> .POSIXct(.Machine$integer.max, tz="UTC")
>>> [1] "2038-01-19 03:14:07 UTC"
>>> 
>>> Dates larger than that cannot be represented by a signed integer.  It
>>> could be worked around, but it's not trivial because R would have to
>>> use something other than the tm C struct.  Luckily, there's a decade
>>> or two before it starts to become a pressing issue. :)
>>> 
>>>>>    ?"POSIXct"? objects may also have an attribute ?"tzone"?, a
>>>>>    character vector of length one.  If set to a non-empty value, it
>>>>>    will determine how the object is converted to class ?"POSIXlt"?
>>>>>    and in particular how it is printed.  This is usually desirable,
>>>>>    but if you want to specify an object in a particular timezone but
>>>>>    to be printed in the current timezone you may want to remove the
>>>>>    ?"tzone"? attribute (e.g. by ?c(x)?).
>>>>> 
>>>>>    Unfortunately, the conversion is complicated by the operation of
>>>>>    time zones and leap seconds (24 days have been 86401 seconds long
>>>>>    so far: the times of the extra seconds are in the object
>>>>>    ?.leap.seconds?).  **The details of this are entrusted to the OS
>>>>>    services where possible.  This always covers the period 1970-2037,
>>>>>    and on most machines back to 1902 (when time zones were in their
>>>>>    infancy).  Outside the platform limits we use our own C code.
>>>> 
>>>> 
>>>> On 05/10/2013, at 12:59 AM, Scott Kostyshak <skostysh at princeton.edu> wrote:
>>>> 
>>>>> On Fri, Oct 4, 2013 at 6:11 AM, Imanuel Costigan <i.costigan at me.com> wrote:
>>>>>> Wanted to raise two questions:
>>>>>> 
>>>>>> 1. Is bugs.r-project.org down? I haven't been able to reach it for two or three days:
>>>>> 
>>>>> Yes. Quote from Duncan:
>>>>> 
>>>>>  ... the server is currently down. The volunteer who runs the server is
>>>>>  currently away from his office, so I expect it won't get fixed until he
>>>>>  gets back in a few days.
>>>>> 
>>>>> https://stat.ethz.ch/pipermail/r-help/2013-October/360958.html
>>>>> 
>>>>> Scott
>>>>> 
>>>>>> 
>>>>>> ```
>>>>>> ping bugs.r-project.org
>>>>>> PING rbugs.research.att.com (207.140.168.137): 56 data bytes
>>>>>> Request timeout for icmp_seq 0
>>>>>> Request timeout for icmp_seq 1
>>>>>> Request timeout for icmp_seq 2
>>>>>> Request timeout for icmp_seq 3
>>>>>> Request timeout for icmp_seq 4
>>>>>> Request timeout for icmp_seq 5
>>>>>> Request timeout for icmp_seq 6
>>>>>> ```
>>>>>> 
>>>>>> 2. Is wday element of POSIXlt meant to be timezone invariant? You would expect the wday element to be invariant to the timezone of a date. That is, the same date/time instant of 5th October 2013 in both Australia/Sydney and UTC should be a Saturday (i.e. wday = 6). And indeed that is the case with 1 min past midnight on 5 October 2013:
>>>>>> 
>>>>>> ```
>>>>>> library(lubridate)
>>>>>> d_utc <- ymd_hms(20131005000001, tz='UTC')
>>>>>> d_local <- ymd_hms(20131005000001, tz='Australia/Sydney')
>>>>>> as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6
>>>>>> as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 6
>>>>>> ```
>>>>>> 
>>>>>> But this isn't always the case. For example,
>>>>>> 
>>>>>> ```
>>>>>> d_utc <- ymd_hms(20381002000001, tz='UTC')
>>>>>> d_local <- ymd_hms(20381002000001, tz='Australia/Sydney')
>>>>>> as.POSIXlt(x=d_utc, tz=tz(d_utc))$wday # 6
>>>>>> as.POSIXlt(x=d_local, tz=tz(d_local))$wday # 5
>>>>>> ```
>>>>>> 
>>>>>> Is this expected behaviour? I would have expected a properly encoded date/time of 2 Oct 2038 to be a Saturday irrespective of its time zone.
>>>>>> 
>>>>>> Obligatory system dump:
>>>>>> 
>>>>>> ```
>>>>>>> sessionInfo()
>>>>>> R version 3.0.1 (2013-05-16)
>>>>>> Platform: x86_64-apple-darwin12.4.0 (64-bit)
>>>>>> 
>>>>>> locale:
>>>>>> [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8
>>>>>> 
>>>>>> attached base packages:
>>>>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>>>> 
>>>>>> other attached packages:
>>>>>> [1] lubridate_1.3.0 testthat_0.7.1  devtools_1.3
>>>>>> 
>>>>>> loaded via a namespace (and not attached):
>>>>>> [1] colorspace_1.2-4   dichromat_2.0-0    digest_0.6.3       evaluate_0.5.1
>>>>>> [5] ggplot2_0.9.3.1    grid_3.0.1         gtable_0.1.2       httr_0.2
>>>>>> [9] labeling_0.2       MASS_7.3-29        memoise_0.1        munsell_0.4.2
>>>>>> [13] parallel_3.0.1     plyr_1.8           proto_0.3-10       RColorBrewer_1.0-5
>>>>>> [17] RCurl_1.95-4.1     reshape2_1.2.2     scales_0.2.3       stringr_0.6.2
>>>>>> [21] tools_3.0.1        whisker_0.3-2
>>>>>> 
>>>>>> ```
>>>>>> 
>>>>>> Using R compiled by homebrew [1]. But also experiencing the same bug using R installed on Windows 7 from the CRAN binaries.
>>>>>> 
>>>>>> For those interested, I've also noted this on the `lubridate` Github issues page [2], even though this doesn't appear to be a lubridate issue.
>>>>>> 
>>>>>> Thanks for any help.
>>>>>> 
>>>>>> [1] http://brew.sh
>>>>>> [2] https://github.com/hadley/lubridate/issues/209
>>>>>> 
>>>>>> ______________________________________________
>>>>>> R-devel at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>> 
>>>>> 
>>>>> --
>>>>> Scott Kostyshak
>>>>> Economics PhD Candidate
>>>>> Princeton University
>>>> 
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>> 
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 
> 



[Rd] Possible POSIXlt / wday glitch & bugs.r-project.org status
Imanuel Costigan