comments and clarification on toTz about rcppcctz

[eddelbuettel]

or do I miss something?

I think so.

R> library(RcppCCTZ)
R> example(toTz)

toTzR> toTz(Sys.time(), "America/New_York", "Europe/London")
[1] "2017-05-02 20:37:49.37022 CDT"

toTzR> # this redoes the 'Armstrong on the moon in NYC and Sydney' example
toTzR> # note that the default print method will print the return object in _your local time_
toTzR> toTz(ISOdatetime(1969,7,20,22,56,0,tz="UTC"), "America/New_York", "Australia/Sydney", verbose=TRUE)
1969-07-20 22:56:00 -0400
1969-07-21 12:56:00 +1000
[1] "1969-07-20 21:56:00 CDT"

toTzR> # whereas explicitly formating for Sydney time does the right thing
toTzR> format(toTz(ISOdatetime(1969,7,20,22,56,0,tz="UTC"), 
toTz+             "America/New_York", "Australia/Sydney", verbose=TRUE), 
toTz+        tz="Australia/Sydney")
1969-07-20 22:56:00 -0400
1969-07-21 12:56:00 +1000
[1] "1969-07-21 12:56:00"
R> 

There is a particularly gotcha with POSIXct formatting. This example tries to get at that.

toTz() tries to do something different from just swapping TZ attributes in a POSIXct.

[eddelbuettel]

Another example.

R> tt <- ISOdate(2017, 05, 01, 0, tz = "America/New_York")
R> uu <- RcppCCTZ::toTz(tt,  "America/New_York", "Europe/Amsterdam", T)
2017-05-01 04:00:00 -0400
2017-05-01 10:00:00 +0200
R> difftime(uu,tt)                        ## so toTz() really changes
Time difference of 4 hours
R> vv <- ISOdate(2017, 05, 01, 0, tz = "America/New_York")
R> attr(vv, "tzone") <- "Europe/Amsterdam"
R> difftime(vv,tt)                        ## whereas putzing with the attribute does not
Time difference of 0 secs                 ## and by design!
R> 

[vspinu]

R> difftime(uu,tt) ## so toTz() really changes
Time difference of 4 hours

I see that, but it's 4 hours. What is the meaning of it? Difference between New_york and Amsterdam is 6 hours.

My guess is what you were trying to achieve is what lubridate::force_tz does - change time zone but preserve civil time in another time zone. Thus, 2017-05-01 in EDT becomes 2017-05-01 in CET. Is this correct?

In any case toTz is a stub, it doesn't work with vectors and whatever it does is not very efficient because it converts POSIXct to civil time and then back to tp here. I think this is not necessary as POSIXct already represents a time point since the UNIX epoch.

I would be happy to provide fixes and some extra functionality, but I guess it depends if you really want to push this library beyond "headers-only" state.

[eddelbuettel]

I see that, but it's 4 hours. What is the meaning of it? Difference between New_york and Amsterdam is 6 hours.

I think POSIXct still gets in the way. As per the second examples() example, it does its job.

I am happy with RcppCCTZ to be a wrapper to the CCTZ library as it is, and even happier if lubridate does what you want. Then we can keep this close and move on.

[vspinu]

As per the second examples() example, it does its job.

That is a coincidence because the original is in UTC.

I am happy with RcppCCTZ to be a wrapper to the CCTZ library as it is, and even happier if lubridate does what you want. Then we can keep this close and move on.

Thanks for being honest.

One last question before I move on. Would you ever consider adding RcppParallel dependency for this package for some more advance vectorized operations?

[eddelbuettel]

That sounds like feature creep to me. Demonstrate the timing need. How often to you parse one million rows?

In any event, such wrappers are trivial to do locally.

[vspinu]

Huge bottlenecks in lubridate are update and force_tz methods and those are used pretty much on all operations with periods. So they should be really fast. In my experiments so far, with CCTZ, I can increase 4x the computation speed of force_tz (update should be similar) but that's still not good enough, this is why Rcpp. In any case I will figure it out locally for now and ping you back if the need occurs for these functions in other packages.

[eddelbuettel]

RcppCCTZ gets pulled in by other packages, eg from nanotime. I am pretty solidly against the type of 'fat dependency' tails so pervasive in the whateververse (example: littler uses docopt, that uses stringr and BAM now I have to pay for stringi -- very inelegant).

I'd be up for really fast parsing but I would not want to impose RcppParallel on everybody. Maybe we can find other ways.

Also note that we now have RApiDatetime without any depends. Of course it uses SEXP types so we get the gc() problem with OpenMP etc.

[vspinu]

Also note that we now have RApiDatetime without any depends. Of course it uses SEXP types so we get the gc() problem with OpenMP etc.

I think strptime is a dead end. I was surprised that even CCTZ relies on strptime for non-trivial parsing. I am extracting the parser from lubridate and plan to stay away from strptime entirely. We better be joining forces on one project rather than attempting piles of parsers in different places.

tt <-  rep.int("2016-12-07 10:11:12",  1e6)

microbenchmark(strptime = strptime(tt, "%Y-%m-%d %H:%M:%S"),
               rapistrptime = RApiDatetime::rapistrptime(tt, "%Y-%m-%d %H:%M:%S"),
               cctz = RcppCCTZ::parseDatetime(tt,  fmt = "%Y-%m-%d %H:%M:%S"),
               fast_strptime = lubridate::fast_strptime(tt, "%Y-%m-%d %H:%M:%S"),
               parse_date_time = lubridate::parse_date_time2(tt, "YmdHMS"),
               times = 5)

Unit: milliseconds
            expr        min         lq       mean     median         uq        max neval  cld
        strptime 1256.47199 1267.98074 1271.39243 1269.53826 1274.32866 1288.64250     5    d
    rapistrptime 1103.70969 1106.00239 1113.80845 1119.37908 1119.92481 1120.02629     5   c 
            cctz  388.26808  390.97900  419.07621  400.85430  403.51490  511.76476     5  b  
   fast_strptime   77.35659   79.29205   81.57690   79.80658   84.56250   86.86677     5 a   
 parse_date_time   70.19217   70.62273   74.68908   71.47477   73.16838   87.98734     5 a   
> 

[eddelbuettel]

CCTZ relies on strptime

It doesn't (apart from something simple at the edge, maybe the TZ offset) . It uses C++11 which is why @dcdillon had to backport the part missing under g++-4.9.