[AMD/LINUX] Massive bitrate excursions when displaying mostly static content

[hanneshoch3]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

When I use sunlight to stream over a 56 mbit/s connection with ~35 mbit/s set in Moonlight, everything works fine as long as there is lots of movement. When I open a map ingame for example and the picture becomes mostly static I can observe massive packet loss on the client side and massive bitrate excursions host side (via mangohud overlay), up to 90 mbit/s.
I tried all H/W encoders (H265, HEVC, AV1) on my 7900XT and the all show the same issue. I set the strict bitrate for AMD GPU in vaapi settings, but there is no difference at all. I also tried to set a higher minimum FPS limit (which increased minimum observed bandwidth), but also did not help at all.
Using software encoding it all works just as expected.
I run Arch and tried both sunshine and sunshine-git, as well as mesa and mesa-git, no differences.

Expected Behavior

No bitrate excursions and thus no frame/packetloss.

Additional Context

No response

Host Operating System

Linux

Operating System Version

Arch

Architecture

amd64/x86_64

Sunshine commit or version

2̶0̶2̶4̶.̶1̶1̶0̶2̶.̶1̶9̶0̶0̶0̶4̶.̶r̶0̶.̶g̶3̶9̶b̶a̶b̶4̶5̶-̶1̶

2025.426.10137.r0.gc6f3647-1

Package

Linux - AUR (Third Party)

GPU Type

AMD

GPU Model

7900XT

GPU Driver/Mesa Version

25.2.0_devel.204732.47db31b5564.d41d8cd-1

Capture Method

None

Config

adapter_name = /dev/dri/renderD128
address_family = both
channels = 1
fec_percentage = 25
fps = [30,60,90,120,144]
lan_encryption_mode = 2
min_threads = 8
notify_pre_releases = enabled
qp = 29
resolutions = [
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    3840x2160,
    2560x1440,
    3120x1440
]
sw_preset = ultrafast
upnp = enabled
vaapi_strict_rc_buffer = enabled
wan_encryption_mode = 2

Apps

Relevant log output

[2025-04-27 15:44:35.271]: Info: Video encryption enabled
[2025-04-27 15:44:35.271]: Info: New streaming session started [active sessions: 1]
[2025-04-27 15:44:35.501]: Info: Found default monitor by name: bluez_output.F4_9D_8A_09_CD_47.1.monitor
[2025-04-27 15:44:35.504]: Info: CLIENT CONNECTED
[2025-04-27 15:44:35.508]: Info: /dev/dri/card1 -> amdgpu
[2025-04-27 15:44:35.509]: Info: /dev/dri/card0 -> amdgpu
[2025-04-27 15:44:35.509]: Info: Found display [wayland-0]
[2025-04-27 15:44:35.509]: Info: Found interface: zxdg_output_manager_v1(31) version 3
[2025-04-27 15:44:35.509]: Info: Found interface: zwp_linux_dmabuf_v1(52) version 4
[2025-04-27 15:44:35.509]: Info: Found interface: wl_output(59) version 4
[2025-04-27 15:44:35.509]: Info: Resolution: 3840x2160
[2025-04-27 15:44:35.509]: Info: Offset: 0x0
[2025-04-27 15:44:35.509]: Info: Logical size: 2560x1440
[2025-04-27 15:44:35.509]: Info: Name: DP-1
[2025-04-27 15:44:35.509]: Info: Found monitor: ASUSTek COMPUTER INC VG28UQL1A
[2025-04-27 15:44:35.509]: Info: -------- Start of KMS monitor list --------
[2025-04-27 15:44:35.509]: Info: --------- End of KMS monitor list ---------
[2025-04-27 15:44:35.509]: Info: Screencasting with KMS
[2025-04-27 15:44:35.510]: Info: /dev/dri/card1 -> amdgpu
[2025-04-27 15:44:35.510]: Info: /dev/dri/card0 -> amdgpu
[2025-04-27 15:44:35.510]: Info: Found monitor for DRM screencasting
[2025-04-27 15:44:35.510]: Info: Found connector ID [107]
[2025-04-27 15:44:35.510]: Info: Found cursor plane [90]
[2025-04-27 15:44:35.514]: Info: Creating encoder [av1_vaapi]
[2025-04-27 15:44:35.514]: Info: Color coding: SDR (Rec. 601)
[2025-04-27 15:44:35.514]: Info: Color depth: 8-bit
[2025-04-27 15:44:35.514]: Info: Color range: MPEG
[2025-04-27 15:44:35.515]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY
[2025-04-27 15:44:35.521]: Info: vaapi vendor: Mesa Gallium driver 25.2.0-devel for AMD Radeon RX 7900 XT (radeonsi, navi31, LLVM 19.1.7, DRM 3.61, 6.14.4-1-cachyos)
[2025-04-27 15:44:35.522]: Info: Streaming bitrate is 23363000
[2025-04-27 15:44:35.522]: Info: Using normal encoding mode
[2025-04-27 15:44:35.522]: Info: Using VBR with single frame VBV size
[2025-04-27 15:44:35.522]: Warning: [av1_vaapi @ 0x705bb0006bc0] Multiple slices were requested but this codec does not support controlling slices.
[2025-04-27 15:44:46.021]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:46.137]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:46.571]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:46.937]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:47.304]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:48.952]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-04-27 15:44:49.038]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-04-27 15:44:49.735]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:49.838]: Warning: Skipping FEC for abnormally large encoded frame (needed 7 FEC blocks)
[2025-04-27 15:44:50.402]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-04-27 15:44:51.302]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:51.452]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-04-27 15:44:52.121]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:52.418]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:56.985]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:57.072]: Warning: Skipping FEC for abnormally large encoded frame (needed 7 FEC blocks)
[2025-04-27 15:44:57.206]: Warning: Skipping FEC for abnormally large encoded frame (needed 7 FEC blocks)
[2025-04-27 15:44:57.702]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-04-27 15:44:57.789]: Warning: Skipping FEC for abnormally large encoded frame (needed 7 FEC blocks)

[ReenigneArcher]

2024.1102.190004.r0.g39bab45-1

Time to update, this is very outdated.

[hanneshoch3]

Whoops sorry, I must've copied the wrong text: 2025.426.10137.r0.gc6f3647-1

[Slashic]

Same issue on windows and v2025.423.24313 using 7900XTX on host and wired ethernet 1Gb/s.

I tried a lot of parametters vbr, cbr, HRD, different bitrates, framerates, this issues is the same described here and the stream ends up crashing.

update: this only happens at the beginning of a stream, after relaunching the stream it is stable for hours, it is reproduced 100% of the time.

[2025-05-01 17:19:10.245]: Info: Desktop resolution [3840x2160]
[2025-05-01 17:19:10.245]: Info: Desktop format [DXGI_FORMAT_R16G16B16A16_FLOAT]
[2025-05-01 17:19:10.245]: Info: Display refresh rate [60Hz]
[2025-05-01 17:19:10.245]: Info: Requested frame rate [60fps]
[2025-05-01 17:19:10.263]: Info: Capture format [DXGI_FORMAT_R16G16B16A16_FLOAT]
[2025-05-01 17:19:10.270]: Info: Creating encoder [hevc_amf]
[2025-05-01 17:19:10.270]: Info: Color coding: HDR (Rec. 2020 + SMPTE 2084 PQ)
[2025-05-01 17:19:10.270]: Info: Color depth: 10-bit
[2025-05-01 17:19:10.270]: Info: Color range: MPEG
[2025-05-01 17:19:10.296]: Info: Streaming bitrate is 117964000
[2025-05-01 17:19:11.013]: Info: 
Device Description : AMD Radeon RX 7900 XTX
Device Vendor ID   : 0x00001002
Device Device ID   : 0x0000744C
Device Video Mem   : 24490 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16299 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2025-05-01 17:19:11.014]: Info: Active GPU has HAGS disabled
[2025-05-01 17:19:11.014]: Info: Using realtime GPU priority
[2025-05-01 17:19:11.014]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020
Bits Per Color     : 10
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2025-05-01 17:19:11.014]: Info: Desktop resolution [3840x2160]
[2025-05-01 17:19:11.014]: Info: Desktop format [DXGI_FORMAT_R16G16B16A16_FLOAT]
[2025-05-01 17:19:11.014]: Info: Display refresh rate [60Hz]
[2025-05-01 17:19:11.014]: Info: Requested frame rate [60fps]
[2025-05-01 17:19:11.032]: Info: Capture format [DXGI_FORMAT_R16G16B16A16_FLOAT]
[2025-05-01 17:19:11.039]: Info: Creating encoder [hevc_amf]
[2025-05-01 17:19:11.039]: Info: Color coding: HDR (Rec. 2020 + SMPTE 2084 PQ)
[2025-05-01 17:19:11.039]: Info: Color depth: 10-bit
[2025-05-01 17:19:11.039]: Info: Color range: MPEG
[2025-05-01 17:19:11.065]: Info: Streaming bitrate is 117964000
[2025-05-01 17:19:36.752]: Warning: Skipping FEC for abnormally large encoded frame (needed 7 FEC blocks)
[2025-05-01 17:19:39.820]: Warning: Skipping FEC for abnormally large encoded frame (needed 9 FEC blocks)
[2025-05-01 17:21:45.736]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:22:12.635]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:22:49.682]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:23:05.211]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:23:27.877]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:24:26.098]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:27:09.437]: Warning: Skipping FEC for abnormally large encoded frame (needed 8 FEC blocks)
[2025-05-01 17:27:21.920]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:27:31.738]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:28:57.237]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-05-01 17:29:17.715]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:31:57.283]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:33:01.157]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-05-01 17:33:40.369]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:38:35.684]: Warning: Skipping FEC for abnormally large encoded frame (needed 8 FEC blocks)
[2025-05-01 17:39:09.601]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:14.115]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:19.120]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-05-01 17:39:20.087]: Warning: Skipping FEC for abnormally large encoded frame (needed 6 FEC blocks)
[2025-05-01 17:39:20.419]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:22.564]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:29.736]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:29.936]: Warning: Skipping FEC for abnormally large encoded frame (needed 5 FEC blocks)
[2025-05-01 17:39:37.085]: Info: CLIENT DISCONNECTED

[hanneshoch3]

So, thanks to your report on windows I got to thinking, maybe it was an AMD Firmware Issue. I still have some old firmware-linux packages, so I tried to just downgrade to linux-firmware-20241210.b00a7f7e-1, since I knew it was working around christmas time.

In a short test, it seems to be working again now. So maybe it is an AMD firmware issue? Can you try to install an older windows AMD driver, in the hopes it will also downgrade GPU firmware for you and try again? Might not be a sunshine bug at all, but I haven't tested too much yet.

[Slashic]

It may be an AMD driver issue indeed, but the fact that the Linux driver and Windows drivers behave the same way is quite strange, i'll try streaming from Fedora 41 on the same machine, but Sunshine on Linux isn't at Windows level yet in terms of performances.

Also, there's no "firmware downgrade" involved, everything is software based even on Linux, kernel drivers and Windows drivers never touches the VBIOS, it's often partly overrided by the driver itself, so just putting another version will suffice.

[hanneshoch3]

Nevermind my firmware comment, further testing showed no difference at all.

[Slashic]

Same behavior with v2025.509.184504:

Skipping FEC for abnormally large encoded frame

Then stream crashes with error -1 on Moonlight.

Configuration used:

[2025-05-11 20:27:04.385]: Info: Sunshine version: v2025.509.184504
[2025-05-11 20:27:04.385]: Info: Package Publisher: LizardByte
[2025-05-11 20:27:04.385]: Info: Publisher Website: https://app.lizardbyte.dev
[2025-05-11 20:27:04.385]: Info: Get support: https://app.lizardbyte.dev/support
[2025-05-11 20:27:04.385]: Info: config: 'output_name' = {da41159a-36a2-5bab-a18b-c52bbce27ee7}
[2025-05-11 20:27:04.385]: Info: config: 'min_log_level' = 2
[2025-05-11 20:27:04.386]: Info: config: 'sunshine_name' = Slash (Windows)
[2025-05-11 20:27:04.386]: Info: config: 'always_send_scancodes' = disabled
[2025-05-11 20:27:04.386]: Info: config: 'amd_enforce_hrd' = enabled
[2025-05-11 20:27:04.386]: Info: config: 'amd_quality' = speed
[2025-05-11 20:27:04.386]: Info: config: 'encoder' = amdvce
[2025-05-11 20:27:04.386]: Info: config: 'hevc_mode' = 3
[2025-05-11 20:27:04.386]: Info: config: 'av1_mode' = 3
[2025-05-11 20:27:04.386]: Info: config: 'amd_rc' = cbr
[2025-05-11 20:27:04.386]: Info: config: 'dd_hdr_option' = disabled
[2025-05-11 20:27:04.386]: Info: config: 'motion_as_ds4' = disabled
[2025-05-11 20:27:04.386]: Info: config: 'touchpad_as_ds4' = disabled
[2025-05-11 20:27:04.386]: Info: config: 'dd_configuration_option' = ensure_only_display
[2025-05-11 20:27:04.386]: Info: config: 'fec_percentage' = 0

HRD does not change this, nor setting CBR (+ HRD)

Also this issue was already reported but closed:

#1040 (comment)

#2788 (comment)

[Slashic]

After some extensive testing, i went through some old builds because I remembered this issue appeared around October 2024:

I wasn't able to reproduce the issue with version 0.23.1, nor v2024.1004.14216 (4th of October 2024 nightly)
With v2024.1107.45727 (07th of November 2024) i was able to instantly reproduce the issue.

After having a look at commits between these two builds, I heavily suspect FFmpeg update to 7.1 is the one causing this issue: 011a872

I may be wrong but this is the only major commit done on video encoding between these two builds.

[Slashic]

Update: I finally managed to reproduce the issue with v2024.1004.14216, although the issue seems less frequent for some reason, it's not the FFmpeg update or at least it's not entirely due to that, unfortunately the gap is too wide with 0.23.1 and a lot changed as well.

[gschintgen]

I think I can confirm that there's been a significant regression on AMD/Linux. I'm on Ubuntu 24.04 (with kisak-mesa, but otherwise stock) and here's what I observed when I came back to Sunshine after a longer break:

Current release (v2025.122.141614) has issues as described in the OP: in slower moving parts, e.g. in the menus before and after a race in a racing game, there's a lot of packet loss shown by moonlight-qt and the stream is stuttery and the "Slow connection" warning is shown frequently or even most of the time. During the races it's better but not perfect (some packet loss was present but not as catastrophically). I tried without and without the strict bandwidth limiting, h264 and h265, and was not really satisfied even though it was playable.

I then tried the latest pre-release but it's not satisfactory either (same symptoms). I tried 0.23.1 but it didn't work at all (at first there were missing dependencies and after having installed the missing libs it would still crash immediately at application startup). So no currently available public build for Ubuntu 24.04 is working correctly as far as I can tell.

Fortunately I still had an older build artifact lying around: commit 6c48342 of PR #3115.
That version is working great for me! In other words the regression seems to have been introduced between 1st September 2024 and 22nd January 2025.

[ReenigneArcher]

@gschintgen can you test out this PR? #3844

[gschintgen]

@gschintgen can you test out this PR? #3844

First I'll try to track down where the regression is even coming from. (I'm not sure all that minimum frame rate stuff has an impact here, the in-game menus are not static, there's just less motion.)

I thought it might be coming from ffmpeg 7.1, but no. Commit 1c21236 (October 20) is still perfectly fine for me. (It includes ffmpeg 7.1 and the related patches that came a few days later.) I'll do a git bisect, but the results won't be for today.

[gschintgen]

I did some bisecting and validating and here are my preliminary conclusions. It's getting late though... Any confirmation would be appreciated.

Up to commit 9662f05 (Nov. 1) the stream is rock solid. As an objective measure for "solid" I mean that "Frames dropped by your network connection" as shown by moonlight-qt's OSD is stuck at a clean 0.00%. (I also playtested various commits quite a bit to confirm both behaviours: "solid" and "erratic".)

When I move up to commit 9e52ac4 (also Nov. 1: "feat(vaapi): add option to enable strict enforcement of frame size") streaming performance becomes quite erratic, "Frames dropped" often fluctuates between 0 and ~40%(!) and the stream becomes stuttery and the red "Slow connection" warning appears.

(I hope I'm not misinterpreting commit dates and merge dates, etc.)

It is my understanding that in July 2024 there was a merge of a fix for bitrate overshoots (see #2821) and that this fixed rate control has been in place until the newer fix (?) was merged on 1st November. (PR #3332). As such builds between July and end of October are supposed to have proper bitrate handling at the price of possible image quality regressions (see #2864).

I therefore based my testing scenario on an "Enabled" strict bitrate control setting in the VAAPI tab in Sunshine's WebUI (for commits that have this option in place) in order to have a hopefully fair comparison between commits before and after the relevant changes.

I'm using a RX6650 (RDNA2) card as that is the only one at my disposal. I chose 1080p, 80MBit/s, h265 for my testing (over 1GBps-Ethernet).

I could do some more (re)testing (in particular with disabled strict rate control), post sunshine logs before and after, etc. but that's for another day. Any insight (or request for further testing) would be appreciated. In particular from @cgutman :-)

[ReenigneArcher]

Thanks for digging into this.

I hope I'm not misinterpreting commit dates and merge dates, etc.

We started doing squash merging last year, so the merge and commit date are the same (as long as the PR number is in the commit title)

[cgutman]

So you're seeing bitrate excursions even with the VAAPI strict bitrate enforcement option enabled? Please post your logs with debug log level so we can see what rate control and VBV size was chosen by Sunshine and FFmpeg.

It would also be useful to get these logs both from 9662f05 and from the latest version of Sunshine, so we can compare them.