Shifted keys register as unshifted on bluetooth programmable keyboards #893
Comments
|
I'd say your analysis doesn't make sense. The shift key and the modifier state is not handled on fcitx side, but display server side. Especially on X11, in the most case the key is forwarded the same way that the application is seen. Let's only consider it on X11 first. Can you post fcitx5-diagnose when some applications that you used to type text running? |
|
Hi! I wasn't aware of that, I filled the issue because it's only happening when fcitx is running. I first checked with someone at ZMK a few weeks ago and he suspected a kind of remapper causing the issue (karabiner and the likes). I have no such thing installed, the closest is fcitx5 IME addons that convert unicode characters that I send with my keyboard. And since the unshifted keys behavior doesn't happen when they are not running, I figure the issue somewhat linked to that? Here's the diagnose log from my Arco Plasma X11 (also have Wayland on the same machine): Since I had a session opened, I did some more testing on X11 and it's better than on Wayland. The main issues are with Kitty and Wezterm (which I use all day long!), most of other apps I tested were fine. Apart from those two, on Wayland there are also issues with Firefox (and variants), Libre Office, Thunar, Termite, Gnome Terminal. I could not reproduce with KDE defaults apps: Kate, Konsole, Dolphin. Seems to me Qt is working fine but GTK doesn't always? Thanks, it's very much appreciated! |
|
Can you start fcitx5 as and try to type something with shift in any qt application? When the group layout equals to the engine layout, fcitx does not do any keyboard layout conversion. Which seems to be the case from the profile in your diagnose. |
|
ah, sorry, I missed the part that Qt app works fine. So can you type in some app that doesn't work well? |
|
As for Gtk, can you try install fcitx5-gtk then try in gtk (under X11). |
|
I think there could be a race in what key state should be used for xim frontend. But in fcitx5, the key state in the key event is not used, instead, it's listening to the global xkb state notification. So if the latter is not arrived to fcitx earlier than the key event forward by the client, wrong key sym would be produced. I think that's why you see it with xim client. For wayland case, I don't think it's possible to trigger the same behavior, unless the bug is within hyprland, since the modifier update is being pushed by hyprland on the same wayland object. The order should be guaranteed and no mismatch. Can you fcitx5 --verbose=wayland=5,key_trace=5 and type some key in kitty or wezterm? Also kitty can only do ibus frontend on X11, I wonder if it has any problem on X11. |
…#893) In rare condition, race could be happen when client forward key while fcitx haven't update its internal state. This should make the behavior sync with fcitx4.
|
A few things to test: On X11
On Wayland
|
|
Hi!
Maybe I need to reboot? Here's the logs on Wayland: For the trace I started fcitx5 on Kitty. Then typed ")" once (registers as "0"), then double tapped the same key (registers as "0)" then exited. As I'm typing this on Firefox right now, I realize I cannot reproduce on Wayland since I installed Only WezTerm and Kitty didn't change as far as I can tell. I'll try to tweak their configs to see if something can be done. |
|
I confirm the behavior is the same on X11 and Wayland now, GTK apps all seem fine. Only WezTerm and Kitty are still having the issue. |
|
Ok, once that worked it was easy: Setting this is wezterm.lua solves the issue: {
use_ime = true,
xim_im_name = "fcitx",
}And I guess it's settled then, thank you so much for your help. 😄 |
|
Sorry, I don't really think so, kitty doesn't do GLFW_IM_MODULE=fcitx. GLFW_IM_MODULE=ibus is the only value that kitty accepts on X11. (no worries, fcitx5 also implements partial ibus for client). A few questions for kitty on X11:
A few questions for wezterm on X11: For wayland,
Then try to type in kitty/wezterm (Make sure Ctrl + ; works so you know key are flowing to fcitx), does the issue present? As for your wayland log, it's very interesting because
|
|
Hi! I've done some more testing on Wayland, unfortunately the issue is still there on Kitty and Wezterm. The "0" still happens instead of ")" if not preceded by another shifted key in a short time window. That is why the log showed the key as ")", I double tapped it so it registered as "0)". For the other keys I really have to get specific conditions. The left parenthesis is the opposite, if I tap another key really rapidly just before, I get "9" instead of "(". So tapping dash then left parenthesis, I consistently get "-9-9-9" instead of "-(-(-(". I don't get why "(" and ")" have the opposite behavior tough, that's weird... For the additional tests:
|
|
@JeffDess systemsettings is for kde plasma wayland. What I'm trying to ask you to do is to compare hyprland and kwin wayland. Hyprland simply permit any process to use the zwp_input_method, but on kwin, it must be launched by kwin to be able to use the wayland input method protocol, that's what the systemsettings part are for. If you did it correctly, you should be able to see frontend:wayland in fcitx5-diagnose when using kwin. The reason that I'm asking for this is because based on the log I can see fcitx is doing things right. I want to compare if using kwin make a difference. |
|
Got it, I'll try that and report back. |
|
@JeffDess btw can you also try to see if xterm has the same issue? you can try it with whatever environment since it's X only (use ctrl + ; to confirm key flows to fcitx). |
|
Ok, I've tested in a Plasma Wayland, fcitx5 virtual keyboard selected. No change in Wezterm or Kitty. But the KITTY_DISABLE_WAYLAND trick works, Kwin seems to be able to handle the window, the window was simply unmanageable in Hyprland (floating instead of tiling, unable to move it). I do have this message tough:
Xterm is working without issues. $ fcitx5-diagnose | grep frontend
# IC [5cd6a1411d2346e2aa745c29505e72bf] program:xterm frontend:xim cap:4000000000 focus:0
# IC [7c264a14c5974b2dab50be6c4c9847bc] program:firefoxdeveloperedition frontend:dbus cap:6000000032 focus:0
# IC [9bc3b47c5baf4c85a6ced1bbf35f201d] program:kitty frontend:wayland cap:100000052 focus:0
# IC [5750c61cfde3482fb3c3effd9bd9d79d] program:krunner frontend:dbus cap:e001820072 focus:0
# IC [c205f8d0d46246169e37ea039b915c96] program:firefoxdeveloperedition frontend:wayland cap:52 focus:0
# IC [feea18f1703d47ba8903372e5334288e] program:plasmashell frontend:dbus cap:1e001800072 focus:0
# IC [f2eaa889e9e54ad3a7e383be56aafee1] program:xterm frontend:wayland cap:100000052 focus:0
# IC [66b66dd08c57445eb9a705c20090ea02] program: frontend:ibus cap:12 focus:1
# IC [c8782604a1ff43aabd45a121113e5fcf] program:org.wezfurlong.wezterm frontend:wayland cap:52 focus:0
# IC [7ff74061692c44b8a79bf5c0e51cb014] program:org.kde.plasmashell frontend:wayland cap:52 focus:0
# IC [1dc56ab7357f4eb699eff9c87e88bd8f] program: frontend:wayland cap:100000052 focus:0
# Wayland Input method frontend 5.1.5 |
|
@JeffDess I wonder if gtk uses text-input-v3 will trigger this, can you start firefox with following command and can you also check if |
|
Getting closer... Firefox with the two variables reproduces the issue. Kwrite always reproduces the issue with the variable set or unset in Plasma Wayland. But retried Kwrite in Hyprland, no issues with the var set or unset. |
|
@JeffDess maybe let's take kwrite as an example and investigate a little bit on how kwrite receive keyboard event.
|
|
I closed and reopened the session in Plasma and was able to reproduce the issue. However, I rebooted and now it's working fine in Kwrite, I cannot reproduce. I guess I just played to much with the processes. As for this command: $ WAYLAND_DEBUG=1 kwrite
# Failed to create wl_display (Connexion refusée)
# qt.qpa.plugin: Could not load the Qt platform plugin "wayland" in "" even though it was found. |
|
how did you run that command though? If you're using wayland how could that happen?... |
|
I am not sure I quite understand the question, here's what I did:
I can suppress the warning by running |
|
@JeffDess You probably doesn't use sddm or any dm to start desktop? When you run kwrite with the command, the message you get means you have wrong WAYLAND_DISPLAY set or sth, maybe the value is even from previous session? |
|
I fixed it, I think replacing qt5ct with qt5ct-kde is what did it. Perhaps just uninstalling and reinstalling the same would have done it also? And yes I am using SDDM. Here's the output of Kwrite with Wayland debug: |
|
Regardless the "time" steam, the actual time being processed by the app is still different, and also, modifiers is always immediately followed by shift press. I can't really think of a reason what could trigger the behavior you saw. [ 818801.221] [email protected](6305, 848026, 42, 1) <- shift press I wonder if you could build a qtwayland and add some debugging log within it. Specifically: The return value of xkb_state_key_get_one_sym , You can just write |
|
Also if you could kindly provide me some link on where you got the keyboard since I'm in US, I could try to get one and debug myself. |
|
Alright I have built the Qt app with the added statement. I've done If that's of any use, I also tested WezTerm with WAYLAND_DEBUG=1. The text I typed was someting like "))))))))" but registered as "0000)000": As for the keyboard itself, I bought the parts for a Corne 5 columns from Typeractive and built it myself. I don't think they sell prebuilts, you'd have to find a used one or to build it to have that exact model. Some other stores have similar ready-to-use keyboards, but I beleive you'd be able to reproduce with a nice!nano snapped to any board. I believe it's the same footprint and pinout than ProMicro and Elite-C, so if you can find a nice!nano, a compatible PCB and a couple of switches that might do. |
|
wezterm's log looks pretty werid, [email protected](5561, 0, 0, 0, 0), this means set state to all depressed. Can you try again in kwin? I could tell from wl_registry that this is not log under kwin. |
|
There you go, I also found a GUI debug logger in WezTerm, so I include this new format as well: |
|
Ok, this also looks quite fishy [ 662699.893] [email protected](7134, 11421373, 42, 1) <- press But the kwrite log in this commet: #893 (comment) looks OK, I wonder when you produce that log, did you have seen any issue? |
|
No issue on Kwrite if I recall. |
|
...Ok, but I was also very confused by your previous comment.
Does kwrite has the issue, or not? |
|
My apologies if it wasn't clear, I mentioned afterwards that the issue with Kwrite was fixed and then submitted the log. Kwrite and other Qt apps are working fine now, GTK as well, the only places where the issue persists is on WezTerm and Kitty as far as I can tell. When I had the issue with Kwrite , I had fiddled with the virtual keyboard on Plasma, spawned/killed/restarted fcitx a bunch of times and set/unset environment variables. If I recall correctly:
I think it must have been linked to virtual keyboard or QT_IM_MODULE or QT_QPA_PLATFORM that were unset (they are set another way in Hyprland). |
|
I'm waiting for the keyboard to arrive. should arrive by this week. |
|
Excellent, it will be much more easier for you to debug. You could flash my latest firmware if you want to have the exact same thing. |
|
In the meantime, I've been testing some more on X11. Can't log back in Plasma since I installed the Wayland version for some reasons, but other WM gave me consistent results. Kitty works fine but Wezterm still has the issue. I've tested many combinations of environment variables about GTK and GLFW on Wayland but nothing works. Kitty and Wezterm still have the issue. My configs: /etc/environment (X11) XMODIFIERS=@im=fcitx
GTK_IM_MODULE=fcitx
QT_IM_MODULE=fcitx
GLFW_IM_MODULE=ibus
QT_QPA_PLATFORM=xcb
QT_QPA_PLATFORMTHEME=qt5cthyprland.conf (Wayland) |
|
Ok, so I'm testing with my keyboard now. I could reproduce it regardless whether bluetooth is used or not. As for X11, Also, I noticed that if you press "()" in a very fast speed and then hold ")", it will produce 0, which is quite understandable after I learned how this keyboard works. It send four key event sequence, but what if they are overlapped? it will certainly caused some confusion, but anyway, I don't think that's a "bug" in application. To conclude, the bug should be in wezterm's X11/xim implementation. Please report to wezterm. you can simply ask wezterm dev to run following script to reproduce the issue. Basically I think the point would be, the xcb key event, either from X server or input method, it already carries the state mask, and that is the always correct. Use key code + key mask from x server may cause event be handled with wrong mask. The fix should be in wezterm's process_key_event_impl. For wayland it's a different story, will investigate later. |
|
Oh! So this issue is per application, per display server. Wow, thank you so much for your work, I'll report it to Wezterm. If you happen to find more about the bug on Wayland, I'm all ears! |
|
So basically wayland case, it's basically a design flaw in the wayland input method protocol. Normally it shuold be like this: key(42, press) This order is guaranteed, since it comes from a single source: compositor. The problematic event sequence may look like this: modifiers(1, 0, 0, 0) <- shift mask set The key event arrives later than input method it because it have to go through compositor -> input method -> compositor -> application, even if input method does nothing by just by pass the key in your use case. The reason is that input method basically does a key grab on the keyboard with wayland input method protocol. When text-input is active on wayland, compositor will send key event to input method, instead of the X's (X -> app -> input method -> app). When input method send key event back to server, it can only send key code, not a modifier key mask to associate with it, while the xcb_key_press_event struct can carry a key mask with it. The reason that wezterm has the same issue under X is because its X implementation uses the similar approach like what wayland does (listen to mask from display server, instead of using the mask from the key event). The workaround for it that I can think of is to convert the event seen on input method as text commit directly. This means that input method will convert key press to text commit on input method side, instead of on the application side. This approach has one down side, if application does not handle the text commit the same way like key event (e.g. input method emit text "A" is not handled the same way like key "A"), it will produce inconsistent outcome. In all GUI toolkits, "handle key event" and "handle text from input method" are two different code path, which means they could certainly have different behavior and sometimes this difference is also legit. That's the reason that I don't really prefer this to be enabled by default, since in real life (a non-macro based keyboard world), the event order issue doesn't really happen (human don't really press and release shift and other key at "exact" same time, it will be around ms level, nor us level). When you use im module (GTK_IM_MODULE, QT_IM_MODULE), you are actually using a similar event handling approach like X, so bug doesn't happen there. Also, if there's a way to put a tiny longer delay between the key press and key release that the keyboard firmware produces, you can also try that. |
|
I pushed a commit that add an option to allow commit text instead of sending back key event. You can go to fcitx5's addon config "Wayland Input Method Frontend" and uncheck the new option: "Forward key event instead of committing text if it is not handled". I'll not make this false by default, but should be working as an easy workaround. |
|
Thank you so much for that fix. It seems to work fine on WezTerm Wayland, I am getting a lot of input lag for some reason, but I don't think it's related to this fix. On the other hand, Kitty have a similar behavior than before but instead of displaying the unshifted characters, it just doesn't output anything. Doesn't feel good to have every other keypress dropped! I will test it properly in a couple of days, I don't have access to homebase computer right now. I have also asked ZMK and there is no global setting to add delay. Given that it's by design, that the default behavior is correct on X and Wayland (without IME) and the specificity of this bug, I don't have high hopes seeing this implemented. Your explanation makes a lot of sense. It means even applications using IME on Wayland could not implement a fix, as they never see the correct input? |
|
I could check again later on kitty to see if there’s some issue there.
…On Sat, Nov 25, 2023 at 18:26 Jean-François Dessureault < ***@***.***> wrote:
Thank you so much for that fix. It seems to work fine on WezTerm Wayland,
I am getting a lot of input lag for some reason, but I don't think it's
related to this fix. On the other hand, Kitty have a similar behavior than
before but instead of displaying the unshifted characters, it just doesn't
output anything. Doesn't feel good to have every other keypress dropped! I
will test it properly in a couple of days, I don't have access to homebase
computer right now.
I have also asked ZMK and there is no global setting to add delay. Given
that it's by design, that the default behavior is correct on X and Wayland
(without IME) and the specificity of this bug, I don't have high hopes
seeing this implemented.
Your explanation makes a lot of sense. It means even applications using
IME on Wayland could not implement a fix, as they never see the correct
input?
—
Reply to this email directly, view it on GitHub
<#893 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB7MZBHXJCWY7SBU5OBOYLYGKSE5AVCNFSM6AAAAAA7HX5AZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRWGQ3TANJXGA>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
|
So far I can't reproduce anything like lag or not able to commit anything here with hyprland or kwin. But I kinda know wlroots's text-input-v3 may have some bug when switching between windows. can you try to capture the log of kitty (without any kitty relevant environment variable being set?) Also, just FYI, I don't usually set any environment variable to force anything under wayland. Especially, I don't use QT_QPA_PLATFORM, QT_QPA_PLATFORMTHEME, since it may break kde. Also, this might be interesting to read. |
|
Oh yes, I've read that page many times and tried setting/unsetting all of theses variables. Not quite sure if it makes any difference on Hyprland (for Kitty and WezTerm). QT_QPA_PLATFORMTHEME is needed, or QT apps default to a white theme. I unsetted the IM ones, and made sure everything was as expected: echo $QT_IM_MODULE $GTK_IM_MODULE $XMODIFIERS $SDL_IM_MODULE $GLFW_IM_MODULE
# (no output)I then reproduced the Kitty behavior in Hyprland while logging the events with:
Here's what I did afterwards:
You'll probably also see "GUI+T", "CTRL+SHIFT+W" in the logs as I opened and closed a terminal. |
|
@JeffDess as you can see [email protected]_string(")") appeared twice in the fcitx log, which means fcitx send two commit ")" to compositor. in kitty log, "text-input: commit_string event: text: )" also appeared twice Which means, compositor did forward two commit string to kitty. So the answer could only be, something is wrong on the kitty side. On my system I'm using this new option till now on kwin with my laptop keyboard, so far I haven't yet to notice any issue. |
|
@wengxt So I've been testing the new option on my main computer:
Is there some process running on Plasma Wayland that I could start manually on Hyprland? I don't quite understand why Kitty handles it differently. |
test scenario is: ``` bash -c "sleep 5; for((i=0;i<30;i++)); do xdotool keydown --delay 0 Shift_L keydown --delay 0 9 keyup --delay 0 Shift_L keyup --delay 0 9; done" ``` That should cause a series of `(` characters to be emitted, but prior to this commit is was usually mostly `9`'s. What's changing here is: * We copy the pertinent fields from the last xcb StateNotify event. That ostensibly has the current modifier and layout state, but because it comes from the X server, it doesn't factor in knowledge from the IME. * When processing an XCB key event, compute the current modifier mask and override the XKB state with it. * Now XKB will produce correct information about the key syms * Restore the modifier state from the saved StateNotify information. refs: #4151 refs: #4615 refs: fcitx/fcitx5#893 refs: ibus/ibus#2600 refs: #3840
Describe the bug
Hi! I am experiencing an issue for several months now with some keypresses on my ZMK keyboard over bluetooth when fcitx5 is running. If I send right parenthesis for instance (which is left shift + 0 on my keymap), the character being appearing on the screen is 0 (unshifted key). Mashing the same key repeatedly, I get the shift key randomly applied, something like "0))000)". Keys register perfectly as soon as I plug in the USB cable or if I end the fcitx5 process.
I have had this issue on X11 and Wayland, and I am able reproduced the same on a second computer with similar OS/DE. I've determined that this behavior is happening only when xcb addon (on X) and waylandim addon (on Wayland) are running. Any application will have the issue.
From my understanding, ZMK sends the key release two ms after the key press over USB, but over bluetooth it has the same timestamp for both because it's sent in the same packet for optimization. The keypresses order is always right tough,
but it seems to me fcitx5 is not registering the shift key if it is pressed and released at the same timestamp as the other key.
That might be something related to my specific configuration, but to be honest don't think it is a keyboard/configuration issue, as I can make the same keyboard to work flawlessly on X11 with ibus. I've tried to set/unset pretty much every documented environment variables with no luck.
Is it something that could be fixed?
Thanks!
To Reproduce
Steps to reproduce the behavior:
anya GTK application in which you can typeThat being said, I feel like keys with lots of other keybinds are more prone to have the behavior than others. On my keyboard, the RPAR is under the left index finger, which has homerow mods, and the reproducibility is 100%. Other keys are less consistent, but am able to reproduce with pretty much everything on the number row. Caps word will turn any letter to uppercase as expected, except the ones with homerow mods which remain lowercase (also 100% repro). I guess it's the hold-tap behavior that sends all the tap info in the same packet? If you'd like to have a look at the source used to build my firmware, here's my config.
Expected behavior
Desktop
Additional context
Technical details
Here's more details to back this up:
Keycodes
showkeycommand outputs identical keycodes and the same order when ")" or "0" is received :(first character on each line is what is registered)
Timings
xevis giving us timestamps of the events at the exact same time over bluetooth, but over USB there always a two ms offset between presses and releases.Pay attention to the
timeparts on the logs, the rest is pretty much identical. Same as before, Bluetooth reads "0" and USB reads ")" for left shift + 0.Bluetooth
xev:
wev:
USB
xev:
wev:
The text was updated successfully, but these errors were encountered: