More compact output about timg

[hzeller]

The reason for that is a fixed memory layout for fast updates.

[jdebp]

"Fast updates" ends up being slower in what is presented to the human beings, though. It is optimizing the internal storage mechanism, which is (loosely speaking) a CPU writing to and reading from RAM, at the expense of the output mechanism, which is orders of magnitude slower and by far the greater bottleneck. The amount of redundant data output is a significant concern when running this program over a SSH connection or over a serial-attached terminal.

The reasons that the ncurses library and programs like vim, screen, and tmux optimize their output using techniques like double-buffered redraw suppression and optimized cursor motions and attribute changes, are still around.

Take a look at pxl. It is based upon termbox, which does do optimized attribute changes and redraw suppression (albeit that it could do a lot more in the way of cursor motion optimization).

[hzeller]

But how much would we expect to save just with eliminating leading zeroes alone ?

There are only unnecessary bytes if the pixel value is below 100, because only then, the \033[38;2;000;000;000m sequence contains leading zeros. This sequence is 19 characters long, so per RGB value we use on average 19/3 ~ 6.33 bytes.
Assuming an even distribution of RGB values, we expect to save 1 bytes for values < 100 (because there would be one leading zero) and 2 bytes for values < 10.

So under the simplifed assumption of an even distribution of RGB values, even if we double-count the 10/256 (because it is already included in the 100/256 range), we expect, on average, to compress down to

  ((19/3 - 1) * (100/256) + (19/3 - 2) * (10/256) + (1 - 100/256 - 10/256) * 19/3) / (19/3) ≈ 92.6%

So we save about 7.4%.

So just eliminating leading zeroes might not save, on average, as much in a noticeable way.

So when working on more compressed output and the concept of a simple in-memory layout is thrown out, additional ideas should also be considered to make it more worthwhile, for instance not repeating the same color value for consecutive pixels of the same color (I expect a lot more average saving there).

[dolmen]

Removing leading zeroes is the first obvious optimisation of the output. For pure black, using \033[30m would be more efficient.
There are many other possible optimisations. For example when background and foregroud are the same color (use a space colored with only background color) or for contiguous pixels that have the same color. But that's some work for a dedicated library (or filter command).

[dolmen]

Here is an improved compressor:

#!/bin/sed -f
s/\([;[]\)00*/\1/g
s/\(\[[^0-9;:]*\)m.\[\([^0-9;:]*m\)/\1;\2/g

[dolmen]

This issue is still valid: output could be reduced by 50% (but not just by removing leading zeroes).

[hzeller]

Have a look at the last commit: the leading zeroes are now suppressed and repeated mention of colors that didn't change are not emitted.