kodraus / decstr

IEEE 754 decimal floating point bitstrings

Hi there 👋

Finding my examples

https://github.com/search?q=user%3AKodrAus+topic%3Aexample

Any example repositories I put together should turn up with a user:KodrAus topic:example GitHub search query.

decstr's Issues

wrong max & min exponents

There seems to be a displacement between the accepted min max exponents in the spec and the ones implemented.

I've made an example to showcase it and compare it with dec:

use {core::str::FromStr, decstr::*};

fn main() {
    /* 32-bit */

    // this is the epsilon indicated in:
    // https://en.wikipedia.org/wiki/Machine_epsilon#Values_for_standard_hardware_arithmetics
    let str_10en6 = "0.00001"; // 10e-6
    // this is the maximum accepted by the type as a string of digits:
    let str_10en7 = "0.000001"; // 10e-7
    print_decimal![Bitstring32::from_str(str_10en6).unwrap(), as_le_bytes];
    print_decimal![Bitstring32::from_str(str_10en7).unwrap(), as_le_bytes];

    // one more zero overflows:
    // let str_10en8 = "0.0000001"; // 10e-8
    // print_decimal![Bitstring32::from_str("0.0000001").unwrap(), as_le_bytes];

    // converting back to string returns the same number OK
    // const PI32: &str = "3.14159"; // 6 digits
    const PI32: &str = "3.141592"; // 7 digits
    assert_eq![
        Bitstring32::from_str(PI32).unwrap().to_string(),
        String::from(PI32)
    ];

    // Using scientific notation accepts much more.
    // The limit should be between -95 +96 exp range according to the spec,
    // FIXME: but it seems displaced -6, +6 integral units.
    println!("Bitstring32 boundary exponents:");
    print_decimal![Bitstring32::from_str("10e-101").unwrap(), as_le_bytes]; // min exp accepted
    print_decimal![Bitstring32::from_str("10e90").unwrap(), as_le_bytes]; // max exp accepted
    println!("dec:Decimal32 same exponents:");
    print_decimal![dec::Decimal32::from_str("10e-101").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal32::from_str("10e90").unwrap(), to_le_bytes]; //
    println!("dec:Decimal32 correct exponents:");
    print_decimal![dec::Decimal32::from_str("10e-95").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal32::from_str("10e96").unwrap(), to_le_bytes]; //
    println!();

    /* 64-bit */

    // epsilon from wikipedia:
    let str_10en15 = "0.00000000000001"; // 10e-15
    // this is the maximum accepted by the type as a string of digits:
    let str_10en16 = "0.000000000000001"; // 10e-16
    print_decimal![Bitstring64::from_str(str_10en15).unwrap(), as_le_bytes];
    print_decimal![Bitstring64::from_str(str_10en16).unwrap(), as_le_bytes];

    // one more zero overflows:
    // let str_10en17 = "0.0000000000000a01"; // 10e-17
    // print_decimal![Bitstring64::from_str(str_10en17).unwrap(), as_le_bytes];

    // converting back to string returns the same number OK
    // const PI64: &str = "3.14159265358979"; // 15 digits
    const PI64: &str = "3.141592653589793"; // 16 digits
    assert_eq![
        Bitstring64::from_str(PI64).unwrap().to_string(),
        String::from(PI64)
    ];

    // Using scientific notation accepts much more.
    // The limit should be between -383 +384 exp range according to the spec,
    // FIXME: but it seems displaced -15, +15 integral units.
    println!("Bitstring64 boundary exponents:");
    print_decimal![Bitstring64::from_str("10e-398").unwrap(), as_le_bytes]; // min exp accepted
    print_decimal![Bitstring64::from_str("10e369").unwrap(), as_le_bytes]; // max exp accepted
    println!("dec:Decimal64 same exponents:");
    print_decimal![dec::Decimal64::from_str("10e-398").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal64::from_str("10e369").unwrap(), to_le_bytes]; //
    println!("dec:Decimal64 correct exponents:");
    print_decimal![dec::Decimal64::from_str("10e-383").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal64::from_str("10e384").unwrap(), to_le_bytes]; //
    println!();

    /* 128-bit */

    // epsilon from wikipedia:
    let str_10en33 = "0.00000000000000000000000000000001"; // 10e-33
    // this is the maximum accepted by the type as a string of digits:
    let str_10en34 = "0.000000000000000000000000000000001"; // 10e-34
    print_decimal![Bitstring128::from_str(str_10en33).unwrap(), as_le_bytes];
    print_decimal![Bitstring128::from_str(str_10en34).unwrap(), as_le_bytes];

    // one more zero overflows:
    // let str_10en35 = "0.0000000000000000000000000000000001"; // 10e-34
    // print_decimal![Bitstring128::from_str(str_10en17).unwrap(), as_le_bytes];

    // converting back to string returns the same number OK
    // const PI128: &str = "3.14159265358979323846264338327950"; // 33 digits
    const PI128: &str = "3.141592653589793238462643383279502"; // 34 digits, 35 char
    assert_eq![
        Bitstring128::from_str(PI128).unwrap().to_string(),
        String::from(PI128)
    ];

    // Using scientific notation accepts much more.
    // The limit should be between -6144 +6145 exp range according to the spec,
    // FIXME: but it seems displaced -32, +34 integral units 
    println!("Bitstring128 boundary exponents:");
    print_decimal![Bitstring128::from_str("10e-6176").unwrap(), as_le_bytes]; // min exp accepted
    print_decimal![Bitstring128::from_str("10e6111").unwrap(), as_le_bytes]; // max exp accepted
    println!("dec:Decimal128 same exponents:");
    print_decimal![dec::Decimal128::from_str("10e-6176").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal128::from_str("10e6111").unwrap(), to_le_bytes]; //
    println!("dec:Decimal128 correct exponents:");
    print_decimal![dec::Decimal128::from_str("10e-6144").unwrap(), to_le_bytes]; //
    print_decimal![dec::Decimal128::from_str("10e6145").unwrap(), to_le_bytes]; //
    println!();
}

macro_rules! print_decimal {
    // $dec: the decimal number,
    // $tobytes: the fn to convert to bytes
    ($dec:expr, $tobytes:ident) => {
        let d = $dec;
        print!("{:?} ", d.$tobytes());
        println!(": {}", stringify![$dec]);

        // bits in original le order:
        // for b in d.$tobytes().iter() { print!["{b:08b} "]; }
        // bits in be order:
        // for b in d.$tobytes().iter().rev() { print!["{b:08b} "]; }
    };
}
pub(crate) use print_decimal;

its output:

[1, 0, 0, 34] : Bitstring32::from_str(str_10en6).unwrap()
[1, 0, 240, 33] : Bitstring32::from_str(str_10en7).unwrap()
Bitstring32 boundary exponents:
[16, 0, 0, 0] : Bitstring32::from_str("10e-101").unwrap()
[16, 0, 240, 67] : Bitstring32::from_str("10e90").unwrap()
dec:Decimal32 same exponents:
[16, 0, 0, 0] : dec::Decimal32::from_str("10e-101").unwrap()
[16, 0, 240, 67] : dec::Decimal32::from_str("10e90").unwrap()
dec:Decimal32 correct exponents:
[16, 0, 96, 0] : dec::Decimal32::from_str("10e-95").unwrap()
[0, 0, 0, 120] : dec::Decimal32::from_str("10e96").unwrap()

[1, 0, 0, 0, 0, 0, 0, 34] : Bitstring64::from_str(str_10en15).unwrap()
[1, 0, 0, 0, 0, 0, 252, 33] : Bitstring64::from_str(str_10en16).unwrap()
Bitstring64 boundary exponents:
[16, 0, 0, 0, 0, 0, 0, 0] : Bitstring64::from_str("10e-398").unwrap()
[16, 0, 0, 0, 0, 0, 252, 67] : Bitstring64::from_str("10e369").unwrap()
dec:Decimal64 same exponents:
[16, 0, 0, 0, 0, 0, 0, 0] : dec::Decimal64::from_str("10e-398").unwrap()
[16, 0, 0, 0, 0, 0, 252, 67] : dec::Decimal64::from_str("10e369").unwrap()
dec:Decimal64 correct exponents:
[16, 0, 0, 0, 0, 0, 60, 0] : dec::Decimal64::from_str("10e-383").unwrap()
[0, 0, 0, 0, 0, 0, 0, 120] : dec::Decimal64::from_str("10e384").unwrap()

[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 34] : Bitstring128::from_str(str_10en33).unwrap()
[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 192, 255, 33] : Bitstring128::from_str(str_10en34).unwrap()
Bitstring128 boundary exponents:
[16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] : Bitstring128::from_str("10e-6176").unwrap()
[16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 192, 255, 67] : Bitstring128::from_str("10e6111").unwrap()
dec:Decimal128 same exponents:
[16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] : dec::Decimal128::from_str("10e-6176").unwrap()
[16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 192, 255, 67] : dec::Decimal128::from_str("10e6111").unwrap()
dec:Decimal128 correct exponents:
[16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0] : dec::Decimal128::from_str("10e-6144").unwrap()
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 120] : dec::Decimal128::from_str("10e6145").unwrap()

0f64 encodes as 0.0

This library uses ryu to convert between binary and decimal floating points. The decimal string chosen by ryu is faithfully encoded in the decimal. Since ryu encodes 0 as 0.0, this means we encode Bitstring::from_f64(0) differently to Bitstring::from_i32(0).

This may be something we want to fix. It probably also applies to integers like 1f64, which ryu will probably represent as 1.0.

Parsing/formatting performance

cc @joseluis

I thought I should call out that I haven't really spent time optimizing parsing or formatting yet, so while it's not really bad, it's also not great. There are some very simply benches in the library you can run, but on my M1 MacBook they look a bit like this:

// This library
test decimal_from_str_finite       ... bench:         553 ns/iter (+/- 10)
test decimal_to_str_finite         ... bench:         514 ns/iter (+/- 12)

// dec
test libdecimal128_from_str_finite ... bench:         455 ns/iter (+/- 9)
test libdecimal128_to_str_finite   ... bench:         254 ns/iter (+/- 6)

seeking clarification on collaboration guidelines for project extension

Hi! First of all thank you this crate. I was searching for a ieee-754 decimal implementation and this is very good.

I'd like to use it as a dependency but I'd need to implement additional things like common constants, more traits like Default, Binary, also ideally PartialEq, PartialOrd...

I was wondering if you're open to collaborations for extending the library, or do you prefer if people would use it as a base for their own implementations? It's not clear to me from the readme.

BTW I found the fn zero() that calls from(0_u8) returns an all-zeroed byte array which is different from what would be expected, from the docs, the from_str("0") result and other implementations which all coincide. Is this unintended?

from_f32/from_f64 with NaN or Infinity returns None

print_decimal![Bitstring32::from_f32(f32::NAN).unwrap(), as_le_bytes]; // FIX None

This should return Some, with the payload encoded as an integer (probably zero, but may be something else). If the payload doesn't fit, then it should return None.