Words typed in Lulua are not identical to words typed in other default layouts about lulua

[PromyLOPh]

Yes, you are right, the hamza mechanic chosen (i.e. composition-based) is different from most layouts currently in use. But Unicode strings cannot be compared on the character or even byte level. UTS #10 defines the Unicode Collation Algorithm, which should be used to correctly sort (and compare, which is the same thing) Unicode strings. Step 1.1 explicitly mentions that before doing anything else both strings must be normalized into form D (which stands for decomposed). Thus

>>> 'أحمد' == 'أحمد'
False
>>> unicodedata.normalize('NFD', 'أحمد') == unicodedata.normalize('NFD', 'أحمد')
True

I’m sure most applications don’t do that, because they are only developed with Latin character sets in mind.

Is this explanation clear? Should I add a note about this to the website?

[PAHXO]

Thanks for the thorough explanation. Yes, a note added to the website would be appreciated.

I wonder if there's a workaround that could be implemented to combine those chars into a decomposed form to add support for those legacy applications, but even that would be difficult to be supported on multiple platforms.

[PromyLOPh]

I’ll add a note.

Most platforms know a concept called dead keys. That means multiple key presses produce one output character. It’s used on European keyboards for example for the French acute accent. It works by pressing the acute first and then a letter character, thus ´+a=á. In theory this should also work for combining ٔ and ا into one character.

I’m not a native Arabic speaker/writer, but using this concept would feel very weird to me, because
a) ٔ and ٕ often feel like an optional diacritic, just like short vowels/shadda/…, which are added after the consonant and
b) the layout would have two different concepts for character composition: The Hamza+Alef deadkey variant, which is a “true” composition into one character, plus – the other way around – consonant+short vowel which do not combine into a single character.

What do you think about dead keys? Is it worth it?

[PAHXO]

Sadly, the dead key solution is probably not good enough because of the order. The hamza being typed first is just not intuitional.
But maybe with some experimenting, it would not be that difficult to adjust to.

The best solution would be to edit the previous character and keeping the input order the same, but it has technical limitations

[PromyLOPh]

This is certainly possible with a custom IME, similar to what is used for Asian languages (Korean, Chinese, …). It does not “edit” the previous character though, instead it “buffers” characters and “commits” them (or combined versions thereof) based on certain rules. But I feel it’s total overkill and typing would feel very weird, because we’d have to buffer (that means: defer writing!) every alef, waw and yah, so they can be combined with the following hamza.

In Korean for example every “sound” is combining like this ㅇ+ㅏ+ㅇ=앙 and thus it’s very natural to buffer a character until it’s complete and then write it. (But it has the downside that you cannot modify this combined character afterwards. You need to retype all parts again, if you delete it.)

[PAHXO]

Yeah, even the IME solution doesn't seem very sound. The current system appears to be the best despite the Unicode comparison issue.