Comments (2)
@Fatal1ty read the code, indeed adding an index seems like:
- (a) a lot of refactoring across a number of types
- (b) should be slower indeed
for (b), maybe this pattern would be less degrading:
def sequence_unpacker(seq):
result = []
try:
for idx, el in enumerate(seq):
result.append(int(el))
except Exception:
raise SomeException(seq, pos=idx)
but still a real impact.
I'll draft something on one of the upcoming weekends
from mashumaro.
So when something breaks during deserialization, the current basic functionality is unusable. Even if I could have inspected the 300MB worth of text, finding what caused an error is impossible. The outer context is not helpful, on the contrary it is detrimental. A typical example - I've changed some model and trying to code a migration from the old one.
This is a very good example of why this issue should be addressed. To be honest, I've come across this problem myself.
They way I've fixed it is by using an unwrapped helper: code and doc
I really like this helper. Good job!
It is almost ideal, but there is one thing that can't be done outside of mashumaro - namely, I can't get the index of child if it is part of a sequence.
I don't see any another solution other that changing the generated sequence unpacker code to something like this:
def sequence_unpacker(seq):
result = []
for idx, el in enumerate(seq):
try:
result.append(int(el))
except Exception:
raise SomeException(seq, pos=idx)
But it will cost us a speed reduction because in the current implementation we are getting [int(el) for el in seq]
. We might introduce a new "debugability" config option for those who want sacrifice performance in favour of improved error handling.
And then either:
make what my helper does the new behavior
make it a config option to let InvalidFieldValue behave in such a way for backward compatibility
I think we should start from a config option and then see how it goes. If we manage to keep sequence debugability without relying on stack trace and avoid notable impact on performance, we can make this behavior default.
I can try my hand at this if the concept seems right. Didn't want to do a PR before we discuss this
The concept is definitely right and this is a very important issue. If you need any help, let me know.
from mashumaro.
Related Issues (20)
- Using Union with int/float casts to whichever appears first HOT 3
- Not parsing Generics correctly HOT 2
- Unserializable field in 3.12 if defined as a Generic TypeVar with mixin bounds HOT 6
- Allow propagation of class based discriminator settings to subclasses HOT 3
- Reject extra keys on deserialization HOT 7
- Investigate support for recursive Union types HOT 2
- Supports `numpy.ndarray` type for `orjson` HOT 2
- Add support for PEP 695
- Union type of [int | float] not serialized correctly HOT 2
- Add an alternative way to assign a field alias with annotations
- Add support for PEP 696
- Annotated SerializationStrategy used as a field serialization strategy leads to RecursionError
- Suppress warning about Union type HOT 4
- Take description from docstring HOT 4
- Incorrect type annotation for the result of `to_json` in `DataClassORJSONMixin`
- Different results between standard library json and orjson HOT 3
- Python 3.12.4 Compatibility
- Some tests error out with typing_extensions>=4.12
- More issues on Python 3.12.4 HOT 5
- [BUG] to_msgpack doesn't behave properly with Discriminator(include_subtypes=True) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mashumaro.