Coder Social home page Coder Social logo

Very large files fail to parse about yamldotnet HOT 6 OPEN

apkrieg avatar apkrieg commented on June 2, 2024
Very large files fail to parse

from yamldotnet.

Comments (6)

apkrieg avatar apkrieg commented on June 2, 2024

If this is something that would be accepted in a PR, I'd love to contribute, but I believe it would break things like Mark.

from yamldotnet.

apkrieg avatar apkrieg commented on June 2, 2024

I plan to make a PR for this sometime next week. If this change would not be accepted because it could be breaking, please close this so I don't waste the time. Cheers!

from yamldotnet.

EdwardCooke avatar EdwardCooke commented on June 2, 2024

Initially I think converting ints to longs where needed would be fine. I can’t really see much of a downside other than the potential for slightly increased memory footprint (4 bytes vs 8). Would be interesting to compare the numbers in your use case of your massive file.

from yamldotnet.

EdwardCooke avatar EdwardCooke commented on June 2, 2024

If you don't have time to work on this, then I may be able to find time. Do you have a download link for that yaml file you could share, I'd like to compare apples to apples when doing this work.

from yamldotnet.

lahma avatar lahma commented on June 2, 2024

Sounds quite natural that int won't fit a index large enough when parsing such a huge file. Unfortunately changing ints to longs will have negative performance impact for smaller files.. I'm not entirely sure if such big files ought to be supported without them being split to logical parts. Many parsers do not expect input files to be that large.

from yamldotnet.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.