Coder Social home page Coder Social logo

Comments (2)

mishamsk avatar mishamsk commented on July 1, 2024 1

@Fatal1ty read the code, indeed adding an index seems like:

  • (a) a lot of refactoring across a number of types
  • (b) should be slower indeed

for (b), maybe this pattern would be less degrading:

def sequence_unpacker(seq):
    result = [] 
    try:
        for idx, el in enumerate(seq):
            result.append(int(el))
    except Exception:
        raise SomeException(seq, pos=idx)

but still a real impact.

I'll draft something on one of the upcoming weekends

from mashumaro.

Fatal1ty avatar Fatal1ty commented on July 1, 2024

So when something breaks during deserialization, the current basic functionality is unusable. Even if I could have inspected the 300MB worth of text, finding what caused an error is impossible. The outer context is not helpful, on the contrary it is detrimental. A typical example - I've changed some model and trying to code a migration from the old one.

This is a very good example of why this issue should be addressed. To be honest, I've come across this problem myself.

They way I've fixed it is by using an unwrapped helper: code and doc

I really like this helper. Good job!

It is almost ideal, but there is one thing that can't be done outside of mashumaro - namely, I can't get the index of child if it is part of a sequence.

I don't see any another solution other that changing the generated sequence unpacker code to something like this:

def sequence_unpacker(seq):
    result = []
    for idx, el in enumerate(seq):
        try:
            result.append(int(el))
        except Exception:
            raise SomeException(seq, pos=idx)

But it will cost us a speed reduction because in the current implementation we are getting [int(el) for el in seq]. We might introduce a new "debugability" config option for those who want sacrifice performance in favour of improved error handling.

And then either:

make what my helper does the new behavior
make it a config option to let InvalidFieldValue behave in such a way for backward compatibility

I think we should start from a config option and then see how it goes. If we manage to keep sequence debugability without relying on stack trace and avoid notable impact on performance, we can make this behavior default.

I can try my hand at this if the concept seems right. Didn't want to do a PR before we discuss this

The concept is definitely right and this is a very important issue. If you need any help, let me know.

from mashumaro.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.