Inconsistency between linux and windows on getch about pancurses

[ihalila]

Here's an attempt: b5480c6

It assumes that it's UTF-16 on Windows, this works on Windows 10 for me atleast.
Similarly it assumes UTF-8 on Linux which has also worked in my limited testing. For future work it would probably make sense to detect the locale and not try and parse non-utf8 data as utf-8, I need to see if I can setup an environment where I can test this easily.

[ihalila]

What kind of behaviour are you looking for exactly? I fixed the panicing on unexpected values, pancurses should now return larger input values as Input::Unknown's as it should in commit 8f2b779

[hellow554]

I don't know which behaviour should be favored, but all platforms should behave uniform.

On linux, pancurses doesn't parse anything, and directly returns the raw utf8 bytes one by one.

On Windows on the other hand it will return the char directly as one char instead of two separate ones.

I hope I could make clear what my concern is. I can test the code tomorrow.

[ihalila]

It's not so much that pancurses changes things, the logic is actually the same on both platforms. If the output differs it's because there's a difference between PDCurses and ncurses. It would be good for pancurses to correctly parse characters though.

[gyscos]

On linux, ncurses returns individual bytes. The actual encoding depends on the terminal configuration, but it's often utf8.
I don't have a windows machine to test, but I assume pdcurses returns utf16 values (is it configurable? see https://github.com/wmcbrine/PDCurses/blob/cdddccd6bbab39316a8c9e32948c3ad5d5bbb091/wincon/README.md).

Returning input_byte as u8 as char is a bit weird:

• For codepoints values under 128 (ASCII), in both utf8 and utf16, the input value equals the unicode number, so this conversion works.
• For codepoints between 128 and 255, utf8 will produce two bytes, and each will be wrongly treated as a char. UTF16 still coincides with codepoints, so on windows these values are still properly parsed.
• For codepoints above 255, the value will no longer fit in a u8, so it will be returned as "unknown" on windows. On linux, it will still return multiple weird characters.
• And that's not even considering different locales/terminal configurations.

I see two possibilities:

• Not parse anything, but be honest about it, and return either a u8 on linux or a u16 on windows. Problem: this means the datatype itself changes between platforms (we could return a u16 in both cases but it's not as clean).
• Parse the input correctly for each platform. This is a bit tricky, as it means:
  • On windows, parse as utf-16 (is it even actually utf-16?)
  • On linux, parse the bytes depending on the terminal configuration. Should this depend on the locale? Or should we assume it's utf8?

[ihalila]

Closing this with the release of v0.15 which includes the improvements to input handling. I didn't add support for anything other than UTF-8 on Linux, that's left for some time in the future.