Coder Social home page Coder Social logo

Comments (4)

DrewMcArthur avatar DrewMcArthur commented on May 29, 2024 1

in decoding/int.bal:142, we handle this with an explicit error:


    if nBytes == 8 {
        // since 1 << 64 -> 1 bc 64-bit ints
        return error IntDecodingError("decoding signed Int64 types is not currently supported, sorry!", first_byte = data[0]);
    }

the next line is return [out - (1 << (8 * nBytes)), newdata];, which if nBytes == 8, then that attempts to shift 1 to the left 64 times, which gives us either 0 or 1, since ints are 64 bits, and 1 << 64 > MAX_INT (bc MAX_INT = 2 ** 64 - 1), leading to an incorrect decoded response.

we could probably just use an edge case, since we're trying to do X - 2^64, we can do some algebra -

x - 2 ** 64 == x - (2 * 2 **63)
                   == x - (2 ** 63 + 2 ** 63)
                   == x - 2 ** 63 - 2 ** 63

and that should work within the 64-bit architecture constraints

from msgpack-bal.

DrewMcArthur avatar DrewMcArthur commented on May 29, 2024

solution wasn't quite as simple as the idea above - looks like i need to spend some time understanding how ballerina handles these large integer types more.

from msgpack-bal.

DrewMcArthur avatar DrewMcArthur commented on May 29, 2024

am currently getting

 [fail data provider for the function testDecodeCase]
       int range overflow

for decoding these two test cases

TestCase[][] SignedInt64TestCases = [
    [{"input": -11111111111, "output": [0xd3, 0xff, 0xff, 0xff, 0xfd, 0x69, 0xb9, 0xe6, 0x39]}],
    [{"input": -9223372036854775807, "output": [0xd3, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01]}]
];

from msgpack-bal.

DrewMcArthur avatar DrewMcArthur commented on May 29, 2024

this occurs in handleSignedInt

function handleSignedInt(byte[] data, int nBytes) returns [int, byte[]]|IntDecodingError {
    int out = 0;
    int i = 0;
    byte[] newdata = data;
    while i < nBytes {
        i = i + 1;
        byte next;
        [next, newdata] = core:shift(newdata);
        out = (out << 8) | next;
    }

    if nBytes == 8 {
        // edge case, bc 64 bit ints cannot store 1 << 64
        int QUARTER_MAX_INT = 1 << 62;
        return [out - QUARTER_MAX_INT - QUARTER_MAX_INT - QUARTER_MAX_INT - QUARTER_MAX_INT, newdata];
    }

    return [out - (1 << (8 * nBytes)), newdata];
}

from msgpack-bal.

Related Issues (3)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.