Forward Alt and Super #5322
Forward Alt and Super #5322
Conversation
|
I tried scrcpy.exe -shortcut-mod=rctrl
|
|
Please run with Your system (window manager) may capture the shortcut for itself (so scrcpy might not even receive it). |
|
Where are the logs stored? |
|
I'm getting logs in the console, but I won't be able to see immediately what happens when trying to press alt+right or alt+left without capturing to output. Is there a --verbosity flag that saves the log output so I can post it here? |
|
The logs are only output to stdout/stderr. Redirect them to a file using: scrcpy --shortcut-mod=rctrl -Vverbose > file.txt |
|
scrcpy 2.7 https://github.com/Genymobile/scrcpy |
|
The screen only moves when pressing the right or left arrow key, however that isn't generally what Talkback expects from a keyboard user. It expects the actual alt+left, alt+right, alt+enter, etc. to perform the keyboard commands. For reference, the Talkback Keyboard Command reference will give you a full breakdown of the default keymap, and the keys TalkBack anticipates will be used. |
The screen is rotated on Alt+left/Alt+right? It is not expected if the shortcut modifier is not Alt 🤔 I cannot reproduce if I pass |
|
I should clarify. The screen is not rotating left or right, but the focus on a particular screen element, rather. The way Talkback works is you press alt+left or right arrow to move from one item to the other, alt+enter to select that item, etc. If you need a live demonstration via Zoom screenshare, please let me know here. |
But the logs say that the screen is rotating:
If you can just capture a video on your computer, to see what happens when you press Alt+left and Alt+right (and what should happen, if you can, I have never used tallback to be honest). |
|
Will keycode events be logged in hid mode? If so I will do a side-by-side comparison between mirror mode and hid mode. |
|
I can confirm that --shortcut-mod=rctrl is properly making the shortcut right control. However, what is causing alt+any other key to not register? |
On a Pixel 8, if I press Super from the home screen, it opens "Search your phone and more". I type some random text, then press Alt+backspace, it clears the text field. So Alt and Super are correctly forwarded. |
|
Well, seeing as I'm on windows - if I press the windows key (super key) it opens the start menu. I'm typing the following command to start scrcpy: |
👍
OK, they're probably captured by Windows without sending them to scrcpy. Check the events with |
|
logging.txt |
|
Your logging.txt is in UTF-16 for some reason. To convert (and print to stdout): iconv -f utf-16 logging.txtHere:
So it correctly injects Alt+→←↓↑→←→←. Android might not react to such events, but they are correctly injected. |
|
It looks like the screen rotating problem from last time is gone as well. So, is SCRCPY injecting alt+right arrow, alt+left arrow, etc. as a combined event, I.E. the equivalent of pressing and holding alt, pressing right arrow, and releasing them both? Or is it firing off alt and the arrow keys as separate events. If the ladder, why for the arrow keys and not for alt+backspace? Is there an sdk limitation that prevents these keys from firing the way they're suppose to? |
|
It works exactly the same way for Alt+backspace or Alt+arrows: it inject Alt pressed, then arrow pressed, then arrow released, then Alt released (it injects exactly the key events when you press/release, check your logs). |
|
I'm not seeing anything that could indicate there not functioning with TalkBack, though. For reference, alt+right arrow is supposed to move the the next element on the screen, alt+left arrow is previous, alt+enter clicks on the selected icon. When a regular keyboard is connected, or when SCRCPY is in hid mode, this works, however hid mode prevents the device's audio/video from being transmitted. Any thoughts? Is it possible that SCRCPY's shortcuts can/should instead be user-customizable to send the appropriate keyboard events to the phone? I.E. alt+enter in scrcpy sends an alt+enter keyboard event to the phone, similar to how TeamViewer and other programs have a send ctrl+alt+del menu item that sends that keyboard event to the remote machine, bypassing the controlling machine's interception of that command? |
Probably because talkback is listening at a lower level (uinput?), so it cannot capture events injected via the SDK). However, a normal Android app should receive these events. |
Not in UHID mode. Doesn't it work in UHID mode? |
|
Oh, is audio passthrough supposed to work in uhid mode? |
|
Ah, that does work. Had to manually assign --keyboard-mode=uhid, and also disable the lalt shortcut. Let's see if it works with -K normally now. |
|
Ok, looks like this is all fixed. Thanks! |
Alt and Super (also named Meta) modifier keys are captured for shortcuts by default (cf --shortcut-mod). However, when shortcut modifiers are changed, Alt and Super should be forwarded to the device. This is the case for AOA and UHID keyboards, but it was not the case for SDK keyboard. Fixes #5318 <#5318> PR #5322 <#5322>
rom1v
force-pushed
the
forward_alt_meta
branch
from
4d9dba2
to
ff9fb59
Compare
Alt and Super (also named Meta) modifier keys are captured for shortcuts by default (cf
--shortcut-mod).However, when shortcut modifiers are changed, Alt and Super should be forwarded to the device. This is the case for AOA and UHID keyboards, but it was not the case for SDK keyboard.
This PR fixes this.
In addition, mouse capture key for relative mouse mode (when
--mouse=aoaor--mouse=uhid) used hardcoded keys (Alt or Super), regardless of the configured shortcut modifier.Replace these hardcoded keys by the modifiers keys (configured by
--shortcut-mod).As a result, if you use for example
scrcpy --shortcut-mod=rctrl, then Alt and Super are forwarded to the device (in all modes).Fixes #5318
Here is a binary to test:
scrcpy-win64-pr5322.zipSHA-256: aa839e5e3ac34e1034287ff4dfbbcb342612fe0dcd32081d81d6cc50c496eb8