Solving a Firefox bugreport (xmodmap replacement(?))

[johnp]

I've been investigating a Firefox bug over the last week and have gotten stuck on one final issue: We need to know what the real mapping of the LevelThree / LevelFive vmods is, because GDK only sends us a GdkModifierType, which does not support those vmods. The legacy X11 code contains a setup routine utilizing the xmodmap to (afaics) solve this issue, but it seems to be impossible to do the same using the libxkbcommon API (or the GDK API respectively). Am I overlooking something or is it really not possible to somehow get this information?

[wismill]

In order to find the vmod → rmod mapping, you may use the following:

// Please use these named constants whenever possible
#include <xkbcommon/xkbcommon-names.h>
#ifndef XKB_VMOD_NAME_LEVEL3
#define XKB_VMOD_NAME_LEVEL3 "LevelThree"
#endif

const xkb_mod_index_t levelThree_idx = xkb_keymap_mod_get_index(keymap, XKB_VMOD_NAME_LEVEL3);
const xkb_mod_mask_t levelThree = UINT32_C(1) << levelThree_idx;
struct xkb_state *state = xkb_state_new(keymap);
assert(state); // TODO: handle error properly
xkb_state_update_mask(state, levelThree, 0, 0, 0, 0, 0);
const xkb_mod_mask_t levelThree_mapping = xkb_state_serialize_mods(state, XKB_STATE_MODS_EFFECTIVE);
xkb_state_unref(state);

This should work will older versions of xkbcommon.

There is no dedicated API because the vmod/rmod distinction is a X11 legacy and an implementation detail.

@whot @bluetech The current use case is valid while niche. I opened #735 to document it “officially”.

[wismill]

Reading the corresponding Firefox issue, the doc update I filed in #735 is indeed what gdk_wayland_keymap_add_virtual_modifiers (partially) does, as you pointed.

As for the virtual modifiers support, we did fix some functions in 1.8.0, but you mentioned that you need to support older versions of libxkbcommon.

Finally, in case you missed it: #734 fixed a typo in our doc about the usual vmod→rmod mappings.

[johnp]

Hmm, that confirms my fears/assumptions. Luckily, nothing we can't deal with on the FF side, even though it will be somewhat ugly.

If it does and exposes it, then there is nothing we can do here.

I don't expect xkbcommon to be able to do anything about that. From how I understand things, GDK built it's GdkModifierType on top of X11/xkb, but wanted that to be an implementation detail. The overlap < 256 makes rmods "just work" even if people mix things up, but their choice of bit allocation for the mouse button masks is somewhat problematic.

Maybe there is an issue in mixing libxkbcommon and GDK APIs here.
[..]

• use fully the GDK API if possible (do not mix libxkbcommon state/mask with GDK ones), or

There definitely is, but if we were to just use the GDK API, we would not be able to support LevelThree/LevelFive at all, since their GdkModifierType abstraction does not support those vmods – only the mapped rmods, for which we don't know the meaning without "reverse engineering" the used modmap (what this discussion is about after all).

• pass explicitly the keyboard state to GDK (so you get the raw keyboard state from source), or

AFAIK not possible, since GDK registers its own wl_keyboard listeners.

• drop the keyboard handling by GDK.

That should theoretically be possible, and, in fact, FF recently started directly using the wl_keyboard events for key repeat handling. But, due to the complexity involved, and other considerations, I don't expect this to happen soon [0][1][2].

[0] https://bugzilla.mozilla.org/show_bug.cgi?id=1861980
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1701123
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1827232

[...] the doc update I filed in #735 is indeed what gdk_wayland_keymap_add_virtual_modifiers (partially) does, as you pointed.

I saw the PRs and they look good. By now I can also see that this is what GDK does internally, but at the time that was hard to wrap my mind around.

we did fix some functions in 1.8.0, but you mentioned that you need to support older versions of libxkbcommon.

Our minimum version is 0.4.1 right now. We are probably able to bump that somewhat if necessary. I've been using 1.6.0 (Fedora 41) in my tests, and things look good so far.

[wismill]

• drop the keyboard handling by GDK.

That should theoretically be possible, and, in fact, FF recently started directly using the wl_keyboard events for key repeat handling. But, due to the complexity involved, and other considerations, I don't expect this to happen soon.

From your other comments this seems to be the only safe solution, long term. I assume there is no way to get the raw Wayland keyboard state via Gtk, is there? Another possibility would be to use Gtk and the pure Wayland API in parallel, but then you may have race conditions with both keyboard states.

FYI, we are in the process to enable pure virtual modifiers (see #450 and #745) for Wayland. At some point Gtk will have to stop to tamper with the keyboard state if it wants to support XKB properly.

Our minimum version is 0.4.1 right now. We are probably able to bump that somewhat if necessary.

0.4.1 is awfully old! Are there systems with this version that also use currently supported versions of Firefox? I think you should check e.g. Repology to see systems with the oldest currently supported version of Firefox and use the corresponding lower bound for libxkbcommon. I would say that 1.0+ seems a safe choice, but I have not check the details.

[johnp]

Hey wismill, sorry for dropping the ball on this thread, other things in life took priority. After all, the only reason I decided to work on this was because I was involved in fixing the same problem (different underlying bug) on X11 6 years prior, and I got a DM that the same problem had reappeared.

I see you've got some permissions on Bugzilla, so feel free to propose any improvements on the Firefox side. I had to rush things a bit because the soft freeze for Firefox 140 (the next ESR version) starts today (simple backports should still be easy even after), and because I had already lost a lot of time dealing with FF build problems (likely git migration or Fedora 42 upgrade related).

[wismill]

No worry, I understand. It just felt a bit bad that when I finally found time to review it was too late.

Still, I left a late review with some hope it can still be useful for the next iteration. I am currently too busy to send patches to Firefox, but I will do my best to review and answer questions, so do not hesitate to ping me, here or there.