Comments (17)
For this task you may want my termrec — it has tools to record and replay a terminal stream with timing information. There's a library with two parts: for storing/rewinding/etc the raw or frame-by-frame stream, and libtty to keep the state of the terminal. You could instrument the latter to get performance data.
from hexyl.
Another option is to use hyperfine
s --show-output
flag. It will simply loop through all the output of the benchmarked commands instead of piping to /dev/null
.
Without a TTY:
Command | Mean [ms] | Min…Max [ms] |
---|---|---|
hexyl $(which hexyl) |
173.2 ± 5.4 | 166.9…186.1 |
hexdump -C $(which hexyl) |
194.3 ± 6.4 | 188.0…215.4 |
xxd $(which hexyl) |
74.0 ± 2.1 | 70.9…83.4 |
With alacritty
:
Command | Mean [ms] | Min…Max [ms] |
---|---|---|
hexyl $(which hexyl) |
510.9 ± 16.3 | 493.5…544.9 |
hexdump -C $(which hexyl) |
374.7 ± 25.0 | 347.7…431.4 |
xxd $(which hexyl) |
227.5 ± 14.1 | 205.8…244.8 |
With terminator
:
Command | Mean [s] | Min…Max [s] |
---|---|---|
hexyl $(which hexyl) |
1.730 ± 0.047 | 1.659…1.807 |
hexdump -C $(which hexyl) |
0.632 ± 0.019 | 0.598…0.661 |
xxd $(which hexyl) |
0.465 ± 0.024 | 0.427…0.502 |
We can observe:
- Yes, the terminal emulator rendering time is definitely significant.
- Both terminals take much more time to render
hexyl
s output (relatively speaking), as expected - due to the colors. alacritty
is freaking fast 😄
Specifically, I'm thinking about how we turn colors on and off again for every single hex pair and textual character. Ideally we wouldn't turn colors off if the next printed hex/char uses the same color.
Optimizing our color usages would be more overhead on our side and therefore slow down our benchmark (though perhaps not significantly) but if it produces faster rendering it might be worth it.
I wouldn't really bother doing this. I don't think that the current performance is problematic in any way. A more interesting benchmark could be to measure the execution time for rather small files. There might be a startup latency for hexyl
due to its larger binary size, as is typical for Rust programs.
At the very least, we could investigate not printing the style suffix for each character, under the assumption that the style prefix for the next character will suffice (and then just printing the suffix prior to printing a frame character).
Yes, we could probably do this to also save on the bandwidth.
from hexyl.
Don't get me wrong. I love fast programs. I just don't think that hexyl
really has a performance problem. Specifically after your recent PR which made it several times faster. I'm never going to output binary blobs of 1MB or larger to the terminal. And if I am, I don't really care if it takes the hexyl
500 ms to print the 60,000 lines of output to the terminal.
from hexyl.
I on the other hand very often run hd|less
on very large files (seeking to an interesting part, of course). With hexyl, this would be less -R
goodness. So this request isn't completely without point.
from hexyl.
Valid point, but the usage of a pager will help you with the rendering speed because only the current page has to be printed.
from hexyl.
What might be interesting to measure is something like hexyl | less -R
and then immediately trying to view the final page.
from hexyl.
What might be interesting to measure is something like
hexyl | less -R
and then immediately trying to view the final page.
My hope would be that this would be pretty much the time that we get without a TTY.
hexyl $(which hexyl) | less -R
and subsequent Shit+G is definitely much faster for me than waiting for hexyl $(which hexyl)
to be finished.
Is there anything more we want to do here or can this be closed?
from hexyl.
Personally, I am still interested in the performance when just printing directly to Terminal.app. I'd like to do some investigation of this on my own and see if there are some easy wins, so if you don't mind I want to keep the ticket open for at least a little while.
from hexyl.
In a quick test, removing the suffixes and inserting reset sequences before the frame chars results in an approximately 14% slowdown on the benchmark, but a 30% speedup when actually rendering to Terminal.app.
from hexyl.
What might be interesting to measure is something like
hexyl | less -R
and then immediately trying to view the final page.
Wouldn't that be equivalent to something like hexyl | tail -n <height of terminal>
if you don't want to take overhead from less into account?
from hexyl.
@kitlith tail -n 30
would give you the last 30 unwrapped lines of output. less
performs wrapping. That said, less
seems to be pretty smart about jumping to end given how fast it can do it, so it's clearly not calculating wrap points for any non-displayed lines.
from hexyl.
Well, less skips to the end then explicitly says "Calculating line numbers..." while you already see the final screen.
from hexyl.
Point is, less calculating line numbers or doing line-wrapping isn't the focus of this issue? It's the rendering performance. less (shift-G) is not as benchmarkable as just showing the last few lines of a dump with something like tail, or copying the output and displaying it to the screen directly.
from hexyl.
Rendering performance is mostly about how fast the terminal emulator state machine can process the escape codes and text. less
isn't a great measure here because its line number calculation is hard-wrapped lines (and therefore just needs to scan for newlines rather than running the full state machine for all non-displayed lines), but given that piping to less
is expected to be a common use-case it's possibly more important than the time it takes for the terminal to render the actual full output of Hexyl.
from hexyl.
Is the question just about the trade-off between optimizing for one case at the cost of the other?
@lilyball I don't know how involved the changes you made were (in terms of LOC) but perhaps you can just gate them based off of whether or not hexyl is outputting directly to a tty?
from hexyl.
No, it's about adding some complexity to optimize a case that some dismiss as unimportant. More code = maintenance cost.
On the other hand, performance cost of comparing a few variables is so small that I'd guess even shaving some work from printf-equivalent and sending the data via pipe would already be a win — much less going into rendering in the terminal.
from hexyl.
I'm going to close this. If anybody feels that hexyl
is (still) too slow when writing to a terminal, please let me know.
from hexyl.
Related Issues (20)
- Add option to disable printing 0 bytes as ascii 0. HOT 1
- `--no-characters` causes output artifacts when squeezing lines HOT 5
- Is there a cheat sheet anywhere telling what color correlates to category of byte? HOT 3
- Add support for the NO_COLOR environment variable HOT 4
- Regression: broken pipe error
- Make --panels=auto the new default? HOT 3
- hexyl should not fail if `--panels=auto` is used and output is not a TTY
- Identical lines not squeezed HOT 3
- hexyl -C does the inverse of hexdump -C HOT 1
- Request: publish source tar with generated manpage HOT 3
- Support for different code pages / character tables
- Incorrect output when reading from terminal HOT 2
- Feature request: Ship as a crate HOT 3
- `hexyl /dev/zero` hangs HOT 6
- Display some non-printable characters like `bat --show-all` HOT 5
- Feature request: Comodore prg mode HOT 1
- CI artifact names conflict HOT 2
- Provide Statically Compiled Binaries for (aarch64|arm64) Linux
- Support raw output HOT 2
- Customize colors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hexyl.