Comments (3)
Hey @Goose97!
Thanks for the very clear description of the problem you're trying to solve, and the links to the specific parts of the code!
If I'm understanding the problem correctly, I think there's a simpler solution that's probably more performant.
The splitBlocks
function doesn't really need to exist at all. It was just a simple way to reason about the problem.
Instead, I think we can change the importer to just operate on the output of splitLines()
directly without using \n\n
delimited blocks as an intermediate. This also has the benefit of not doubling the memory requirements.
If we do your proposed change, I think we end up with the 1.3GB in memory three times: once as a buffer, once as the return value of .splitLines()
and once as the TextFileContent[]
. If we skip the splitBlocks
, then we just have the first two.
Really, ideally, we'd have only the one in the buffer and splitLines()
should return an iterator instead, then we'd only have the 1.3GB in memory once. That's a separate issue than the one you're trying to solve though.
from speedscope.
Hi @jlfwong,
That makes a lot of sense. I'll try an PR, going the way you suggested.
I guess I'll go ahead and convert the splitLines()
to return an iterator first (neat idea btw), then come back to this issue. Is that cool?
from speedscope.
I guess I'll go ahead and convert the splitLines() to return an iterator first (neat idea btw), then come back to this issue. Is that cool?
Yeah, if you'd like to tackle that, then please go ahead!
Please make it a separate PR from the addressing the issue described here (so one of the iterator, one for the perf file fix)
If, in the process of doing that, you realize it's kind of a pain to do, it's also fine to address the linux perf-file specific issue without solving the iterator.
from speedscope.
Related Issues (20)
- when i use this, how can i get release when i change it?
- output for speedscope format is not loaded in Firefox in Ubuntu unless it has .json extension HOT 12
- Tooltip partially hidden under mouse HOT 2
- Add A/B comparison support HOT 6
- souremaps arent being applied HOT 5
- Could not import the json example into www.speedscope.app HOT 1
- Not support non-systemwide perf result HOT 3
- Ability to pass data encoded
- Callgrind inputs: Some called functions along with their callees are detached from the rest of execution at the root level HOT 1
- Incorrect rendering with long traces (thousands of seconds)
- Allow search by filename HOT 1
- xcode 15 instrument data load fail
- pid/tid string format support HOT 4
- unable to load the json file and getting error for files size above 172 KB HOT 1
- [Feature Request] View Multiple Process Flamegraphs at Once HOT 1
- Add support for Rust v0 symbol mangling scheme
- Use speedscope programmatically HOT 1
- It's non-obvious when speedscope imports a multi-threaded profile
- Native support for cProfile dump HOT 1
- Cannot open large size JSON HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from speedscope.