Dear Maintainer,

I am reporting a reproducible amdgpu/VCE regression on Debian 13 (trixie)
with an AMD Radeon RX 480 / Polaris10 GPU. Active RustDesk remote-control
sessions are stable on Debian kernel 6.12.101-1 but trigger a VCE ring timeout
and GPU reset on 6.12.105-1 and 6.12.107-1.

Expected behavior
=================

Active RustDesk screen-capture and remote-control sessions should not wedge
the VCE engine, reset the GPU, or terminate the graphical desktop.

Actual behavior
===============

On 6.12.105 and 6.12.107, an active RustDesk session reproducibly reaches:

  RustDesk active session -> amdgpu ring vce0 timeout naming rustdesk
  -> GPU reset

The severe 6.12.105 runs additionally reached:

  -> VCE suspension failure -> VRAM loss -> X11/GNOME session teardown
  -> GDM

The controlled 6.12.107 run was stopped at the predetermined failure
boundary after the RustDesk connection froze, the vce0 timeout named
RustDesk, and GPU reset began. That particular run did not record VRAM loss
or full X11 teardown.

System and userspace
====================

* Debian GNU/Linux 13.6 (trixie), amd64
* AMD Radeon RX 480 / Polaris10, PCI ID 1002:67df, amdgpu
* GNOME/GDM X11
* RustDesk 1.4.9
* Official native RustDesk Debian package and package-provided system service
  are the primary reproducer
* firmware-amd-graphics 20250410-2
* Mesa 25.0.7-2+deb13u1
* No NVIDIA hardware, installed NVIDIA userspace, or loaded NVIDIA module
  participates in the final reproducer

The native RustDesk package came from the official rustdesk/rustdesk 1.4.9
release. The Debian asset was rustdesk-1.4.9-x86_64.deb, SHA-256:
7244ba47c40e804172044bfbe659467c54ce46554c98e78c8c0406f1d612fda3.

Regression matrix
=================

* linux-image-6.12.101+deb13-amd64 6.12.101-1: PASS / stable
* linux-image-6.12.105+deb13-amd64 6.12.105-1: FAIL
* linux-image-6.12.107+deb13-amd64 6.12.107-1: FAIL

6.12.105 repeatedly failed during RustDesk sessions. Flatpak RustDesk 1.4.9
independently reproduced the failure, and the official native RustDesk 1.4.9
package/system service reproduced it in a controlled test after approximately
30 seconds.

The official native RustDesk 1.4.9 service reproduced on 6.12.107 after
approximately 15 seconds of active remote use.

On the same hardware and substantially the same userspace and RustDesk
configuration, native RustDesk repeatedly passes on 6.12.101. Successful
tests include interactive use, control at GDM, GDM-to-GNOME transitions,
logout/login, sustained sessions, disconnect/reconnect cycles, and a final
7,201-second unattended monitor (1,355 samples) with zero VCE timeout, GPU
reset, VRAM-loss, or graphical-session teardown events.

This packaging-independent result shows that the observed regression is not
specific to the Flatpak sandbox or packaging. I am not claiming that RustDesk
itself is conclusively innocent or that a particular upstream commit is
responsible. The evidence establishes a kernel/AMDGPU/VCE regression, or a
kernel-specific interaction, triggered by this workload; source-level cause
has not yet been isolated.

Representative 6.12.105 native-service failure
================================================

  amdgpu: ring vce0 timeout, signaled seq=63, emitted seq=64
  amdgpu: Process information: process rustdesk pid 4790 thread rustdesk:cs0 pid 7411
  amdgpu: GPU reset begin!
  amdgpu: failed to suspend display audio
  suspend of IP block <vce_v3_0> failed -22
  amdgpu: GPU reset succeeded, trying to resume
  [drm] VRAM is lost due to GPU reset!
  amdgpu: GPU reset(1) succeeded!
  gnome-shell: X connection to :0 broken (explicit kill or server shutdown).

Representative 6.12.107 native-service failure
================================================

  amdgpu: ring vce0 timeout, signaled seq=127, emitted seq=128
  amdgpu: Process information: process rustdesk pid 4670 thread rustdesk:cs0 pid 21229
  amdgpu: GPU reset begin!
  amdgpu: failed to suspend display audio
  amdgpu: GPU reset(1) succeeded!

Reproducer
==========

1. Boot the affected Debian kernel with the RX 480 bound to amdgpu.
2. Start the normal GNOME/GDM X11 graphical session.
3. Use RustDesk 1.4.9 through its native package/system service to establish
   an active remote-control session.
4. Interact normally with the desktop while monitoring the kernel journal.
5. Within roughly 15-30 seconds in the controlled failing runs, the remote
   session freezes and the vce0 timeout/reset sequence appears.

I did not intentionally reproduce the reset again while collecting this
reportbug metadata under 6.12.107.

Workaround
==========

Persistent GRUB selection currently boots 6.12.101, which restores stable
RustDesk operation. No kernel packages are held, so future candidate kernels
can continue to be installed and explicitly tested.

SSH survives the GPU-reset event, so this should not be characterized as a
full-system crash. It is nevertheless a repeatable regression capable of
resetting the GPU and, on 6.12.105, terminating the graphical session.

The reportbug-generated kernel and package information follows.

Hi,
[...]

As you can reproduce your issue reliably, please bisect the changes
between 6.12.101 and 6.12.105 to identify the breaking commit. That
given we can have a look if it is a known regression already or needs
to be reported upstream yet.

Let me know if you need instructions for the bisecting steps!

Regards,
Salvatore

Hi Manuel, all

Manuel asked off-bug about the instructions, so here we go. To bisect
between 6.12.101 and 6.12.105 proceed as follows (once you have the
my_config, compilation could be as well on a more powerful machine).

    git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
    cd linux-stable
    git checkout v6.12.101
    cp /boot/config-$(uname -r) .config
    yes '' | make localmodconfig
    make savedefconfig
    mv defconfig arch/x86/configs/my_defconfig

    # test 6.12.101 to ensure this is "good"
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem does not exist

    # test 6.12.105 to ensure this is "bad"
    git checkout v6.12.105
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem exists

With that confirmed, the bisection can start:

    git bisect start
    git bisect good v6.12.101
    git bisect bad v6.12.105

In each bisection step git checks out a state between the oldest
known-bad and the newest known-good commit. In each step test using:

    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install, verify if problem exists

and if the problem is hit run:

    git bisect bad

and if the problem doesn't trigger run:

    git bisect good

. Please pay attention to always select the just built kernel for
booting, it won't always be the default kernel picked up by grub.

Iterate until git announces to have identified the first bad commit.

Then provide the output of

    git bisect log

In the course of the bisection you might have to uninstall previous
kernels again to not exhaust the disk space in /boot. Also in the end
uninstall all self-built kernels again.

Regards,
Salvatore

Hi Manuel, all

Manuel asked off-bug about the instructions, so here we go. To bisect
between 6.12.101 and 6.12.105 proceed as follows (once you have the
my_config, compilation could be as well on a more powerful machine).

    git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
    cd linux-stable
    git checkout v6.12.101
    cp /boot/config-$(uname -r) .config
    yes '' | make localmodconfig
    make savedefconfig
    mv defconfig arch/x86/configs/my_defconfig

    # test 6.12.101 to ensure this is "good"
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem does not exist

    # test 6.12.105 to ensure this is "bad"
    git checkout v6.12.105
    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install the resulting .deb package and confirm problem exists

With that confirmed, the bisection can start:

    git bisect start
    git bisect good v6.12.101
    git bisect bad v6.12.105

In each bisection step git checks out a state between the oldest
known-bad and the newest known-good commit. In each step test using:

    make my_defconfig
    make -j $(nproc) bindeb-pkg
    ... install, verify if problem exists

and if the problem is hit run:

    git bisect bad

and if the problem doesn't trigger run:

    git bisect good

. Please pay attention to always select the just built kernel for
booting, it won't always be the default kernel picked up by grub.

Iterate until git announces to have identified the first bad commit.

Then provide the output of

    git bisect log

In the course of the bisection you might have to uninstall previous
kernels again to not exhaust the disk space in /boot. Also in the end
uninstall all self-built kernels again.

Regards,
Salvatore

Hi Salvatore,

I completed the requested bisection between stable v6.12.101 and
v6.12.105 on the affected machine.

The kernel configuration was derived on the affected work-box while it
was running Debian 6.12.101 using the requested localmodconfig /
savedefconfig procedure. I froze that my_defconfig and used it for every
build. Compilation and packaging occurred on a separate amd64 builder;
every GOOD/BAD classification came only from the work-box with its
RX480/Polaris10 GPU under real RustDesk H.264 VAAPI use.

The endpoints reproduced as follows:

   v6.12.101  GOOD
   v6.12.105  BAD

The complete git bisect log is attached. Git mechanically identified:

   52566c150cadb1a17df840dd25862a61dcf84bed is the first bad commit
   drm/amdgpu: check ASPM on the dGPU host link

Its immediate parent, fa9624ad4d6d4b25e448dbc550ddc69e6f849640, tested
GOOD during the bisection. I subsequently qualified that exact parent in
two real RustDesk H.264 VAAPI sessions totalling about 43 minutes,
including a deliberate disconnect/reconnect. All 288 periodic samples
and the complete kernel journal were free of the characteristic
timeout/reset signatures. It is now the pinned operational kernel on
this work-box, and two consecutive normal boots plus a further 10-minute
H.264 VAAPI canary were clean.

There is an important causal qualification. I also built a clean
v6.12.105 tree with only 52566c150cad... reverted. That kernel remained
BAD: after about 79 seconds of real RustDesk H.264 VAAPI activity, the
work-box reproduced the same sequence:

   ring vce0 timeout
   GPU reset begin!
   VRAM is lost due to GPU reset!

Therefore, the mechanical bisection boundary is established, but this
single revert did not establish that 52566c... alone is sufficient for
the full v6.12.105 regression. There may be an interaction with other
changes in the later tree, or the isolated revert may be insufficient; I
do not want to overstate the mechanism.

The captured PCI topology appears to place the RX480 directly behind
Intel root port 0000:00:01.0; I did not observe an AMD internal PCIe
switch in the path. That seems relevant to reviewing the patch's effect
on this hardware. I have not run the optional one-shot amdgpu.aspm=0
mechanism probe.

For operational containment, I also directly qualified exact stable
v6.12.103 with the same frozen work-box-derived configuration. It
completed about 26 minutes of real RustDesk H.264 VAAPI activity across
three sessions with two reconnects, 174/174 clean samples, a clean
complete journal and stable visuals. Exact v6.12.103 is retained as the
emergency fallback and Debian 6.12.101 as a secondary rescue.

For the initial review I have included the complete bisect log, frozen
defconfig, the full first-bad commit metadata/diff, compact PCI
topology, a concise result summary, and short characteristic-signature
extracts from both the v6.12.105 BAD endpoint and the v6.12.105
single-revert BAD test. Full journals, package/build hashes, detailed
topology, and complete evidence manifests are available if useful.

Regards,
Manuel

Salvatore,
I am following up on the bisection results I previously submitted for
Debian bug #1146343.
After that report, the previously heavily qualified immediate-parent
custom kernel, commit
`fa9624ad4d6d4b25e448dbc550ddc69e6f849640`(`6.12.104-hnv2-b1146343-gfa9624ad4d6d`),
showed a GPU-related operational failure during normal use while a
RustDesk-related problem was observed.
The preserved journal does **not**reproduce the original bisection's
characteristic lead signature. There is no `ring vce0 timeout`and no
`amdgpu_job_timedout`. Instead, the event begins with GPU VM protection
faults attributed to `totem`, followed by a `ring gfx timeout`attributed
to Xorg, GPU reset, and VRAM loss. The VCE block appears only as a
failed suspend step during recovery.
Representative sequence:
```text
amdgpu: GPU fault detected ... for process totem
amdgpu: VM fault ...
amdgpu: ring gfx timeout ...
amdgpu: GPU reset begin!
[drm:amdgpu_device_ip_suspend_phase2 [amdgpu]] *ERROR* suspend of IP
block <vce_v3_0> failed -22
amdgpu: GPU reset succeeded, trying to resume
[drm] VRAM is lost due to GPU reset!
```
I therefore classify this later event as **DIFFERENT-SIGNATURE**, not as
a reproduction of the RustDesk H.264 VAAPI `ring vce0 timeout`used to
classify the BAD bisection kernels. It is a separate operational
observation and does not by itself invalidate the mechanical first-bad
result. I am not claiming that RustDesk, Totem, or the bisected change
caused this event.
For operational safety I have nevertheless withdrawn `fa9624...`as the
preferred default. Exact stable `v6.12.103`, commit
`25c09b42358e73e1476e517b296edb6344f2e4bd`, remains directly qualified
GOOD and is now the work-box default. After promotion it booted normally
twice and completed a further 21m06s real RustDesk H.264 VAAPI canary
with multiple connections/reconnects, 126/126 clean samples, and no VM
fault, VCE/GFX timeout, GPU reset, VRAM loss, or
`amdgpu_job_timedout`line. Manuel confirmed that the visuals were stable
throughout, with no freeze, disconnect anomaly, or visible degradation
attributable to work-box.
Given that the full v6.12.105 tree remained BAD after reverting the
mechanically first-bad commit, and that a later operational event on its
immediate GOOD parent showed a distinct AMDGPU gfx/VM-reset failure
path, it is possible that more than one nearby stable-tree change
contributes to the broader instability seen in the later kernels. I have
not isolated or proven such an interaction.
The original interpretation remains conservative:
-mechanical first bad: `52566c150cadb1a17df840dd25862a61dcf84bed`;
-immediate parent: `fa9624ad4d6d4b25e448dbc550ddc69e6f849640`, strongly 
qualified GOOD in the bounded bisection trials but now withdrawn as the
operational default because of this distinct later event;
-v6.12.105 with only `52566c...`reverted remained BAD;
-the mechanical boundary is established, while single-commit causal 
sufficiency is not established.
I can provide the complete affected-boot journal and the exact
`.103`post-promotion evidence if useful.
Regards,
Manuel

Hello Manuel,

Thanks for going through that!

:-(. So 52566c150cad wasn't the real (or single) reason for the
regression you see. If I understand correctly the symptoms on
52566c150cad were also slightly different.

Can you please test 5f46322e0b84af29e10eb951ff45bd6ea40640de,
869eb255e499112f9ffb42e7fa96ff2fca0df54b and
81b5af1fb0f14cace6c3b3130a05e5602a597820 with 52566c150cad reverted?

(That is

	git checkout 5f46322e0b84af29e10eb951ff45bd6ea40640de,
	git revert 52566c150cad
	... build and test according to Salvatore's instructions

and the same for the other two commits.)

I think with these results we can report upstream and let the amdgpu
experts work out the rest (or ask smarter questions).

Best regards
Uwe