background is not completely black about libschrift

[tomolt]

Hi maxstepin.
This is a problem I have known of pretty much from the start. It's not really a bug, and more a limitation of the underlying algorithm. These rows of not-quite-black pixels are the result of a combination of quantization errors and general numerical instability.
It is very likely that you can find the same behaviour with other font renderers that use a similiar algorithm such as stb_truetype.
Is this only an annoyance to you, or does this actually pose a potential problem?

[maxstepin]

Yeah, it's not really a problem.
Thanks!

[maxstepin]

I tried switching to floating point here:
struct cell { double area, cover; };
and invoke quantize() at the last possible moment in post_process()
and artifacts seems to disappear completely.

Just thought you might consider it.

I looked into stb_truetype and they also seems to be doing float->int conversion at the last possible moment.

[tomolt]

Hi maxstepin,

So I just tested this for myself. And you're right, the background is now completely black!

I would've thought the error caused by numerical instability would be much greater.
Actually, last time I measured it it was in fact bigger than the quantization error!
But since then, the algorithm has been changed dramatically, so I really should have updated my measurements.

I was a bit worried about the performance implications, but apparently this is pretty much exactly as fast as before. This is pretty shocking, considering we're now reading and writing 4x the amount of data to the buffer! Just in the last major version release, I got a 15% speedup overall by reducing the data size of the outline struct - I would have expected a comparable slowdown here. I guess draw_lines and post_process just aren't that memory-bound in the first place. Oh well, at least this turns out to be an easy fix now!

I'll probably put this in the next minor version (v7.1).
Thanks!

SideNote: I also tested this with float instead of double, but the slowdown from double-to-float conversion makes this much slower, so it's currently not worth it.

[maxstepin]

You might simplify quantize() now, considering it's only used in one place, and in that place (x >= 0.0) condition is always true.