hrbrmstr / tdigest Goto Github PK

👋🏽 I maintain the cran checks badges. Please change to the new cran checks badge URL (e.g., https://badges.cranchecks.info/worst/dplyr.svg). Old badges at (e.g. https://cranchecks.info/badges/worst/dplyr) will be unavailable as of Jan 1st 2023.

• Blog post from June 2022 announcing the change: https://recology.info/2022/06/cran-checks-badges/
• New badges repo: https://github.com/r-hub/cchecksbadges
• Old badges repo: https://github.com/sckott/cchecksapi

Hi,

Hopefully the below makes the issue clear and easy to reproduce. From what I can tell there is weird behaviour when using the current implementation of td_add. Apologies if this is me misunderstanding what behaviour I should expect, but from the references I think there is something wrong.

x=c(rep(3,10),rep(5,10))
x
[1] 3 3 3 3 3 3 3 3 3 3 5 5 5 5 5 5 5 5 5 5
td <- tdigest(x)
td
<tdigest; size=20; compression=100; cap=610>
tquantile(td, c(0, .01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99, 1))
[1] 3 3 3 3 3 3 4 5 5 5 5 5 5
quantile(x, c(0, .01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99, 1))
0% 1% 10% 20% 30% 40% 50% 60% 70% 80% 90% 99% 100%
3 3 3 3 3 3 4 5 5 5 5 5 5

Everything makes sense up to hear. But then:

td_add(td, 8, 10)
<tdigest; size=30; compression=100; cap=610>
x=c(x,rep(8,10))
quantile(x, c(0, .01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99, 1))
0% 1% 10% 20% 30% 40% 50% 60% 70% 80% 90% 99% 100%
3 3 3 3 3 5 5 5 8 8 8 8 8
tquantile(td, c(0, .01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99, 1))
[1] 3.000000 3.000000 3.000000 3.000000 3.000000 5.000000 5.000000 5.000000 8.272727 9.909091 8.000000 8.000000 8.000000

Something odd has happened, with tquantile solutions that are larger than the largest replicated value (8.27 and 9.91 where the largest value is clearly 8). This behaviour does not occur if we remake the tdigest object:

td <- tdigest(x)
tquantile(td, c(0, .01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99, 1))
[1] 3 3 3 3 3 5 5 5 8 8 8 8 8

This agrees again.

Hopefully the above is not misleading and down to me misunderstanding.

Cheers,

Aaron