[BUG] Can't plot INT64 values about bokeh

[bryevdv]

It's a design decision of JavaScript @jmakov, that we have to live with.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number

[bryevdv]

@jmakov please provide a pure-Bokeh Minimal Reproducible example.

[mattpap]

The largest integer bokeh will accept is 2**53-1 (JS' Number.MAX_SAFE_INTEGER), because that's the largest integer that can be exactly represented by a float64. If your integer data is larger than that, you will get a warning and bokeh will convert it to float64 with a possible loss of precision. If you don't mind working with float64 and you don't want to be warned by bokeh, then either perform the conversion yourself or use fp values from the beginning and let the type system of polars and thus bokeh know what your intentions are.

[jmakov]

I'm working with nanosecond timestamps so it's a bit inconvenient that they need to be converted just to get a plot. But if that's indented, then this issue can be closed since it's not a bug but a design decision.

[jbednar]

Is there a design document somewhere for Bokeh that talks about JavaScript BigInt vs Number? Obviously in most cases Number is more suited to plotting arbitrary quantities, and it looks to me like BigInt objects aren't necessarily easily interoperable with Number objects, but it would be nice to specify the design tradeoffs and issues somewhere. I don't see BigInt mentioned anywhere in the issues or docs.

[bryevdv]

For reference, Bokeh began in 2012, but BigInt did not start to exist in JavaScript implementations until 2018 or so. There was no decision to even consider in 2012, and since then there is the inertia of years of status quo to overcome. This issue is not unknown but it is one that affects a relatively small segment of use-cases, whose fixing will carry non-negligible risk and effort due to its low-level nature, and no-one has simultaneously had the time, money, and personal motivation to dive into it. Frankly I would not expect any movement on this unless or until the work is explicitly funded.

Edit: it's worth noting that even three years ago when I started the discussion linked above, that BigInt still did not exist in some major browsers (e.g. Safari) which is why it was not mentioned, even that recently. It might be a viable option to consider these days, I don't know.

[mattpap]

We don't have any design documentation for this, because only recently BigInt (and BigInt64Arrray and BigUint64Array) became widely available (circa late 2020 and 2021 respectively; Safari significantly delayed adoption) and us dropping legacy browsers in bokeh 3.0 allowed us to finally pursue modern APIs.

The biggest issue with BigInt is that it doesn't work with JS' number type at all, i.e. only BigInt op BigInt are permitted and anything else is a TypeError. Thus even taking a half of a big integer requires a special code path x/2n vs. x/2 for number type (n is a suffix indicating BigInt literals). Only a fraction of bokehjs' code does computations on raw data, but it's not an insignificant amount to deal with, both on the input side of the data pipeline and its output side (tooltips, etc.). Given the amount of work involved, I figured it would be easier to employ WASM and get 64-bit support essentially for free. That work didn't get anywhere far for now, due to WASM's inherent issues and the scale of that work.

On top of that, JavaScript doesn't offer operator overloading, so it isn't possible to create generic, readable/idiomatic and performant code that would work with both number types. Another issue is performance. Based on some on-line benchmarks, BigInt introduces a 4x slowdown compared to number type, though it's not a fair comparison, because we would have to compare it with manual 64-bit arithmetics and not floating-point arithmetics.

[jbednar]

That's all precisely the design considerations I was looking for, thanks! Not sure if this issue is the best place for it, but certainly better to have it here than not! Thanks so much.