on FreeBSD 13.3 issues with extended chars

[rmottola]

Small sum up of email thread.

After FreeBSD 13.3 update, extended character typing (e.g. àè or ° and €) started not working and outputting multiple characters.
It appears to be linked to these env. variables set:
LANG=C.UTF-8
XMODIFIERS="@im=kinput2"

[rmottola]

Even without setting XMODIFIERS (when things appear to "work" for the user) I noticed that KeyPress and KeyRelease characters are different for extended characters:

2024-09-11 23:19:51.042 Ink[2352:100713] XKeyEvent keyCode 22
2024-09-11 23:19:51.042 Ink[2352:100713] unicode modifiers: 0
2024-09-11 23:19:51.042 Ink[2352:100713] unicode no modifiers: 0
2024-09-11 23:19:51.042 Ink[2352:100713] process_key_event: keys:è ukeys:è
2024-09-11 23:19:51.125 Ink[2352:100713] XKeyEvent keyCode 22
2024-09-11 23:19:51.125 Ink[2352:100713] unicode modifiers: 0
2024-09-11 23:19:51.125 Ink[2352:100713] unicode no modifiers: 0
2024-09-11 23:19:51.125 Ink[2352:100713] process_key_event: keys:è ukeys:è 

It is partially expected: On KeyPress the character conversion is done by using Xutf8LookupString and initialising NSString with NSUTF8StringEncoding, while on KeyRelease Xutf8LookupString is not available per X11 spec, then XLookupString is used and string is initialised using NSISOLatin1StringEncoding. One would expect that at least for Latin1 characters it should be equivalent, but not and the string contains "garbage".

[rmottola]

I found out that when the Latin1 is not interpreted correctly, count my XLookupString is > 1, it means it cannot be a single char of a key. It may be 2 long for e.g. à, but 3 for €.

I tried some smartness and if I interpret it as UTF8, all appears to work!!