XKB groups and layout changes are not supported on Linux

[nimble0]

Classification: Bug

Reproducibility: Always

Summary

Emulated keystrokes are affected by KDE keyboard layout remapping.

Steps to Reproduce

1. Set KDE System Settings keyboard layout to Colemak.
2. Launch and activate Plover.
3. Press -T

Expected Results

Output string "the".

Actual Results

Output string "ghf".

Version

Plover version 4.0.0.dev0

First appeared in commit 2419313.

Installed via: Git

Notes

The cause seems to be in _send_keycode, specifically the change from target_window.send_event to xtest.fake_input.

Issue #298 is similiar but this bug appeared later.

Standard Qwerty layout behaves correctly.

Configuration

OS: Linux Mint 17, KDE 4.13.3

[benoit-pierre]

Plover does not handle keyboard layout changes on Linux (and never has). The layout is parsed one time at initialization. The XKB code provided by python-xlib is also a bit of a misnomer, because it does not use the XKB extension, so Plover also does not know anything about XKB groups (and group changes).

[nimble0]

The layout isn't changed while Plover is running (I've changed the steps so that's clearer).

Looking at this more closely, there seems to be a global keyboard layout and some more local layout. Prior to that commit, the local layout didn't have any affect on Plover's output. After that commit though, it does affect the Plover's output but Plover is only aware of the global keyboard layout, so whenever the layouts are different, there's a problem.

The global keyboard layout can be changed with setxkbmap. The local one can be changed with KDE System Settings, but setxkbmap also changes it.

I suppose injecting the events into the window skips that local layout remapping.

[benoit-pierre]

FWIU, there's a global server layout. You can also have per device layouts (including one for the fake XTest keyboard device). And then a layout can have XKB groups, which can be switched per application. Usually, DEs will configure a global layout with multiple XKB groups (one per language configured), and change group per application. Plover only has access to the equivalent of xmodmap -pke, and so is not group aware and behave as if the first group was always selected. It also does handle the fact that different devices can have different layouts, so if for example capslock/backspace are swapped on one keyboard and not on the other, things will get weird when you switch keyboard.

[nimble0]

That makes sense and is quite helpful to know. I do find it a bit strange there was no problem before.

[HenryMarshall]

@benoit-pierre I have a keyboard with a non-standard arrangement of keys so the keycodes don't map to the expected physical keys. I don't want to alter the global XKB layout as that would mess with my normal ability to type. If I wanted plover to use a different mapping, would my best bet be to directly alter xkeyboardcontrol's KEYCODE_TO_KEY dictionary?

[nimble0]

Key bindings can be changed within the program's options; you don't need to touch the source.

[HenryMarshall]

@nimble0 The configurator seems unwilling to accept modifier keys (namely: shift and control) as bindings. Are you aware of a workaround for that?

[nimble0]

I did make a fix for that, unfortunately I won't have access to it for a few weeks. There are a few issues that held me off from making a pull request for it: - I didn't have an OS X or Windows environment for testing. - A modifier key like control will cause issues if used in the strokes that activate plover (because shortcuts will activate before plover knows to block it). Also can be problematic if you try to use a stroke while not realising that plover is currently inactive. - Occassionally, I have noticed shortcuts activating when they shouldn't which could be caused by the fix. This is rare enough and not distrupting enough that I haven't been particularly motivated to look into it more (I use this fix full time). I think the fix roughly involves this: -  Add appropriate entries to KEYCODE_TO_KEY in this file https://github.com/openstenoproject/plover/blob/master/plover/oslayer/xkeyboardcontrol.py - Add appropriate entries to SUPPORTED_KEYS_LAYOUT (must match name used in KEYCODE_TO_KEY) in this file https://github.com/openstenoproject/plover/blob/master/plover/oslayer/keyboardcontrol.py - Remove "if modifiers == 0:" on line 240 in file https://github.com/openstenoproject/plover/blob/master/plover/oslayer/xkeyboardcontrol.py . This line prevents Plover from doing anything while a modifier is active.

[HenryMarshall]

@nimble0 Thanks muchly, I've got my layout working perfectly. One thing of note though, I was unable to get mapping changes to stick in the master (v4.0.0.dev0) branch. Fortunately, as v3 shares the ~/.local/share/plover/plover.cfg file with v4, I was able to edit it in the stable branch and have the changes reflected in v4 when I restarted. Should I open a separate issue for this?

[nimble0]

Yes, that should be a separate issue. That bug appeared with the new Qt interface so I think it's just that no-one has gotten round to adding the UI bindings to save those settings.

[HenryMarshall]

Logged in [a new issue]. Thanks for all your help.

[SimonWoodburyForget]

I might be wrong, but Xlib seems to kind of know what the layout currently is, while xtest just assumes completely differently for one reason or another.

When I do display.keysym_to_keycode(Xlib.XK.string_to_keysym("f")) I'm getting the expected Dvorak keycode 29, but when I use that same keycode through xtest.fake_input(display, X.KeyPress, 29) I get the Qwerty character "y".

Looking around for some temporary solution, I've replaced xtest with python-uinput and it seems to act as expected when using it on a gutted down backend. (taking grabbed keycode events with xlib and shoving them back through uinput)

[benoit-pierre]

A keysym is not a keycode.

[SimonWoodburyForget]

Sorry I understand that. I fixed a typo, I miss-typed display.keysym_to_keycode to convert the keysym into a keycode, which returns 29 (the Dvorak layout "f" keycode), as supposed to 41 (on Qwerty), but then xtest goes ahead and types a "y" which is different to what display.keysym_to_keycode thought.

[linduxed]

I have two questions, for those that might know:

1. Is Update keymap on keymap change #1164 still the only PR/branch that might be addressing the this issue?
2. Have the Python libraries changed since this issue was opened, so that one can use something else than python-xlib? Is python-xlib still the main library being used?

[user202729]

1. As far as I know, yes. (but as I mentioned there, it does not work too well if the input machine is also keyboard, for it causes a key map refresh every stroke.
2. Yes, python-xlib is still the main library used in the code base.