tomc603 / scriba Goto Github PK
View Code? Open in Web Editor NEWProcess to write files to disk
License: MIT License
Process to write files to disk
License: MIT License
Allow the user to specify file should be pre-filled. This will prevent the OS from short-cutting the read path when a file is created with fallocate()
but has not been written to fully.
Ignore the input of the same path multiple times. The user should specify file numbers, or create sub directories in a path if their goal is actually to execute N times the operations against a single path.
This will change the output file names.
Output for read/write performance data is only displaying a single reader/writer for each file rather than all of them. The summing logic for each path appears to be correct though. But the summing logic for grand totals is suspect.
Output should list values for each reader/writer routine for a given file path. Then a sum for the parent path specified on the scriba CLI. Finally a grand total for all parent paths should be given.
Theoretical example
Reader Performance:
/test/path/one/scriba.0.data:
12.99 MiB/sec
11.87 MiB/sec
13.72 MiB/sec
/test/path/one/scriba.1.data:
17.01 MiB/sec
12.82 MiB/sec
15.75 MiB/sec
...
/test/path/one: 38.58 MiB/sec
/test/path/two: 45.58 MiB/sec
Read Total: 225.48 MiB/sec
Writer Performance:
...
Output total figures summed by base path, summing all activity from each file within a path.
IO stat collection sucks right now, collecting latency values for each read and write, and storing all values in a struct. At no point during operation is this data written to disk, so very high IO/s systems with limited RAM resources can run into an OOM.
Ideas for addressing this:
Bandwidth throughput data also isn't necessary to collect any more, given the data is gathered in main()
now.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.