line wrap break Chinese characters for long lines. about icdiff

[jeffkaufman]

I've replicated the behavior you describe with python2. That's because we treat the input as a sequence of bytes, and wrapping a unicode character in the middle of a byte doesn't work well.

With python3 we have a different problem, where text goes off the screen. That's because we expect all characters to be the same width and that we can fit $COLUMNS of them on the screen at once.

[jeffkaufman]

I've fixed the character width issue with 908af2f (1.2.2), where fullwidth characters were being treated as single width.

For now at least, icdiff works on bytes if run under python2 and utf8 characters if run under python3, so support for fullwidth characters is python3 only.

[xiaket]

Hi, @jeffkaufman , this issue not completely fixed.

I switch to python3 and used the same test case as I've provided. Indeed there is no breaking of Chinese characters now, but the text rendering is faulty. Chinese characters are double-width characters, and they should take the space of two ascii characters instead of one. when the current version of icdiff renders the diff output, it would be like(for a console width of 124 characters):

which is not desirable, for the content is not properly wrapped.

For the icdiff version, I'm using 1.3.1.

[jeffkaufman]

I'm testing with:

CHARS='测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试测试行abc测试'
icdiff <(echo "$CHARS") <(echo "$CHARS" | tr c d)

And you're right, this isn't working.

[jeffkaufman]

Fixed in a9b26aa.

[xiaket]

great job, thanks.

[jeffkaufman]

with 93cffff this should work on python2 as well