Scaling on Wayland breaks video streaming

[ultra-azu]

Describe the Bug

On sway, a wlroots based Wayland window manager, if scaling is done, sunshine detects incorrectly the screen resolution and infinitely tries to capture screen but failing. On client, either nothing or a garbled screen appears. I'll gladly provide any other nformation needed.
sunshine.log

Expected Behavior

No response

Additional Context

I have yet to try if it can be replicated on other Wayland compositors or even with scaling on X11. However I'll try this as soon as I can.

Also if I try to run sunshine as root, when it connects to client it segfaults, however I suspect it is a different bug since scaling does not affect it.

Sunshine Host Operating System and Version

Artix Linux

Architecture

64 bit

Sunshine Version

0.14.0

GPU Type

AMD

GPU Model

RX 580

GPU Driver/Mesa Version

22.1.2-3

Capture Method (Linux Only)

Wayland protocol(from what I can tell)

[ultra-azu]

Btw doing sudo setcap cap_sys_admin+p /usr/bin/sunshine-0.14.0 has no effect.

[Quackdoc]

Same issue on wayfire, it is also wlroots based.

[Quackdoc]

I was able to get it working by using wlr-randr to change the resolution while the stream was running and that seems to have "fixed" it, needs done every time, though Im wondering if the new fractional scaling stuff might wind up fixing this when it's all merged.

[LizardByte-bot]

This issue is stale because it has been open for 90 days with no activity. Comment or remove the stale label, otherwise this will be closed in 10 days.

[LizardByte-bot]

This issue was closed because it has been stalled for 10 days with no activity.

[puffnfresh]

I can reproduce this (with v0.19.1) please reopen.

[Stetsed]

I can reproduce this on 1.19.1 using Hyprland

Edit: Attempting to run with wlr capture method using a 1.5x scaling, just gives me a black screen\

Switching to KMS capture makes the capture work but the mouse doesn't show

[Stetsed]

@puffnfresh Just a quick FYI, you can fix it by switching to KMS capture and adding WLR_NO_HARDWARE_CURSORS to your config, this will make KMS have a mouse cursor and work. Why wlr capture doesn't work idrk

[MatthiasGrandl]

Same here. You can start the stream at 1.0 scaling and then change the factor and it keeps working, but initializing the stream with scaling doesn't work.

Edit: both wlr and kms capture behave this way.

[MatthiasGrandl]

If you are like me and are using sunshine for game streaming, you can use gamescope to start sunshine like this:

gamescope -e -w 3840 -W 3840 -H 2160 -h 2160 -r 120 -- bash -c 'sunshine & steam -steamos -tenfoot -fulldesktopres -nointro'

This totally works. Even my regular non-gamescope desktop is still getting captured just fine.

[mroi]

I have the same problem using the labwc compositor (another wlroots derivative).

I checked the sunshine code and it queries the xdg-output protocol for the resolution. According to the Wayland documentation, this protocol returns logical sizes. But with a scale factor applied, these values will differ from the physical size that is needed to capture the frames correctly.

I tried a ham-fisted patch by slapping my scale factor of two onto width and height in wayland.cpp lines 102 and 103.

--- a/src/platform/linux/wayland.cpp    1970-01-01 01:00:01.000000000 +0100
+++ b/src/platform/linux/wayland.cpp    2023-12-13 16:54:45.842332101 +0100
@@ -99,8 +99,8 @@
 
   void
   monitor_t::xdg_size(zxdg_output_v1 *, std::int32_t width, std::int32_t height
-    viewport.width = width;
-    viewport.height = height;
+    viewport.width = 2 * width;
+    viewport.height = 2 * height;
     BOOST_LOG(info) << "Resolution: "sv << width << 'x' << height;
   }
 

This works for me, but of course is not a proper fix.