Dear Maintainer,
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
When selecting Gnome session, there's no X11 nor Wayland option and always
starts an X11 session. I'm using an amd gpu with an nvidia discrete card.
nvidia-smi output:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02 Driver Version: 510.60.02 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 Off | N/A |
| N/A 43C P8 5W / N/A | 6MiB / 6144MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 5395 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------+
I'd like to use the amd gpu with a wayland session and use the discrete card
only with chosen apps only.
Overwritting 61-gdm.rules with ln -s /dev/null /etc/udev/rules.d/61-gdm.rules solves the issue for me. Now I have X11 and Wayland options back, tested Wayland and works great. I also can use new optimus environment variables to choose between discrete and integrated cards. For vulkan apps, the Nvidia card is selected first, but nothing I can't override by setting VK_ICD_FILENAMES environment variables for radv. Let me know if I can help testing new defaults so this can be solved. Regards Sebastián
regarding this file disables wayland if nvidia driver below 510 (which is not your case) or modesetting is disabled, could you confirm you have the nvidia_drm driver with modesetting disabled ? cat /sys/module/nvidia_drm/parameters/modeset and grep nomodeset /proc/cmdline Cheers, Alban
Hi Alban, My nvidia driver version: 510.85.02-1 cat /sys/module/nvidia_drm/parameters/modeset N grep nomodeset /proc/cmdline <Empty> Cheers, Sebastián El sáb, 27 ago 2022 a las 3:20, <alban.browaeys@gmail.com> escribió:
On Thu, 8 Sep 2022 17:22:00 -0300 =?UTF-8?Q?Sebasti=C3=A1n_Lacuesta?= <sebastianlacuesta@gmail.com> wrote:
the gdm udev rules work how they ought to, ie:
/lib/udev/rules.d/61-gdm.rules "
# disable wayland if nvidia-drm modeset is not enabled
ATTR{parameters/modeset}!="Y", GOTO="gdm_disable_wayland"
"
a way around is
https://wiki.archlinux.org/title/GDM#Wayland_and_the_proprietary_NVIDIA_driver
ie https://wiki.archlinux.org/title/NVIDIA#DRM_kernel_mode_setting,
that is, add in /etc/default/grub to "GRUB_CMDLINE_LINUX_DEFAULT":
"nvidia_drm.modeset=1"
the run update-grub as root.
8?Q?Sebasti=C3=A1n_Lacuesta?=
About drm modeset
https://ubuntuforums.org/showthread.php?t=1613132 "
The newest kernels have moved the video mode setting into the kernel.
So all the programming of the hardware specific clock rates and
registers on the video card happen in the kernel rather than in the X
driver when the X server starts.. This makes it possible to have high
resolution nice looking splash (boot) screens and flicker free
transitions from boot splash to login screen. Unfortunately, on some
cards this doesnt work properly and you end up with a black screen.
Adding the nomodeset parameter instructs the kernel to not load video
drivers and use BIOS modes instead until X is loaded.
"
So enabling Kernel Mode Setting might lead to a black screen before the
nvidia driver is loaded (gdm start) if the nvidia driver is broken (or
not loaded).
From https://wiki.archlinux.org/title/Wayland#XWayland 5.1 tells
enabling DRM KMS is required (for XWayland application only ?, ie most
games).
For early loading (console with optional "high resolution nice looking
splash (boot) screens and flicker free transitions from boot splash to
login screen") add in /etc/initramfs-tools/modules:
"nvidia nvidia_modeset nvidia_uvm nvidia_drm"
then run:
update-initramfs -u
(this only update the latest installed kernel), or for all kernels:
update-initramfs -u -k all
This if you want o try plymouth or want to set "GRUB_GFXMODE=" in
/etc/default/grub.
https://wiki.archlinux.org/title/kernel_mode_setting 2.1 tells that
NVIDIA kms has to be manually enabled (what we saw above).
All in all I do not believe this is software gdm bug that you cannot
start gdm in wayland mode with nvidia modesetting disabled (even if it
is disabled by default).
Can you confirm that with nvidia modeset enabled you do not need any
hack to start GDM in wayland mode (ie as i GDM shows gnome xorg and
wayland options)?
This is still an issue but a note on
/usr/share/doc/gdm3/README.Debian might be enough to clarify that on
nvidia setups one has to enable modeset on grub?
Kind regards,
Alban
PS: I still do not know if current NVIDIA driver default to not enable
modeset is really an issue for gnome wayland or if only helps with
screen flickers when switching from wyaland to console and helps with
advanced graphics on boot console.
Maybe this requirement could be removed, and in this case this bug
should be forwarded upstream.
About modeset, ie Kernel Mode Setting being required or not I found this about https://wiki.archlinux.org/title/Wayland#Requirements 5.1 that "Enabling DRM KMS is required.". THe next link points to https://download.nvidia.com/XFree86/Linux-x86_64/515.48.07/README/xwayland.html which tells that: " Requirements The following are necessary to enable accelerated rendering on Xwayland with the NVIDIA driver: - DRM KMS must be enabled. See Chapter 36, Direct Rendering Manager Kernel Modesetting (DRM KMS) for details. - The installed copy of Xwayland should be a build from the master branch of https://gitlab.freedesktop.org/xorg/xserver at least as recent as commit c468d34c. Note that if this requirement is not satisfied, the NVIDIA GPU can still be used for rendering, however it will fall back to a suboptimal path for presentation resulting in degraded performance. - libxcb version 1.13 or later must be present. - egl-wayland version 1.1.7 or later must be present (if installed separately from the the NVIDIA driver). - If using the GNOME desktop environment, kms-modifiers must be enabled through gsettings. This can be done with the following command gsettings set org.gnome.mutter experimental-features [\"kms- modifiers\"] " So currently, once logged in gnome wayland without modeset enabled, you should not have Xwayland application (most games) running hardware accelerated. To not end up on software acceleration you have to enable KMS (by the settings I sent you previously) and also setting an experimental feature in the gnome session: gsettings set org.gnome.mutter experimental-features [\"kms-modifiers\"] maybe you can check if you are on software acceleration on Xwayland with glxgears/glxinfo . Maybe you can send your glxinfo output without nvidia modeset set and logged in a wayland session? Kind regards, Alban
Now added nvidia_drm.modeset=1 as a kernel parameter and doing cat /sys/module/nvidia_drm/parameters/modeset gives me: Y Still need to have the `ln -s /dev/null /etc/udev/rules.d/61-gdm.rules` hack in order to have Wayland properly. I do have the kms-modifiers activated too. Versions of libxcb, egl-wayland are 1.15 and 1.1.10 respectively. Issue starts by not having wayland options in gdm and in turn, my gnome session later. El lun, 19 sept 2022 a las 14:27, Alban Browaeys (<alban.browaeys@gmail.com>) escribió:
Hi,
As stated above, the wayland is not disabled just because of modeset. It is
disabled even if /sys/module/nvidia_drm/parameters/modeset gives Y.
I did some more test and confirmed that their are 2 more reasons that wayland
is disabled on my laptop:
- missing suspend support;
- hybrid graphics laptop with vendor NVIDIA driver.
All these restriction is introduced at [1]. I do see some application
(blender) running on NVIDIA GPU broken after suspend. But overall the system is
OK.
So, to fix this, we may need to:
- fix suspend. Ubuntu provide file "/usr/bin/nvidia-sleep.sh" and friends in
package "nvidia-kernel-common-510" or similar [2]. I think we can do the
similar.
- Find out why hybrid graphics laptop is not supported. I cannot see the
reason.
[1]: https://gitlab.gnome.org/GNOME/gdm/-/merge_requests/171
[2]: https://packages.ubuntu.com/jammy/amd64/nvidia-kernel-common-510/filelist
Below is the changes I made to 61-gdm.rules in order to get wayland option
back.
--- 61-gdm.rules 2023-06-11 14:13:53.858268221 +0800
+++ /lib/udev/rules.d/61-gdm.rules 2023-06-11 15:21:58.097221687 +0800
@@ -52,6 +52,7 @@
ACTION!="add", GOTO="gdm_nvidia_end"
RUN+="/usr/bin/touch /run/udev/gdm-machine-has-vendor-nvidia-driver"
+GOTO="gdm_nvidia_end"
# Check if suspend/resume services necessary for working wayland support is
available
TEST{0711}!="/usr/bin/nvidia-sleep.sh", GOTO="gdm_disable_wayland"
TEST{0711}!="/usr/lib/systemd/system-sleep/nvidia", GOTO="gdm_disable_wayland"
@@ -89,7 +90,6 @@
TEST!="/run/udev/gdm-machine-is-laptop",
GOTO="gdm_hybrid_nvidia_laptop_check_end"
TEST!="/run/udev/gdm-machine-has-hybrid-graphics",
GOTO="gdm_hybrid_nvidia_laptop_check_end"
TEST!="/run/udev/gdm-machine-has-vendor-nvidia-driver",
GOTO="gdm_hybrid_nvidia_laptop_check_end"
-GOTO="gdm_disable_wayland"
LABEL="gdm_hybrid_nvidia_laptop_check_end"
# Disable wayland in situation where we're in a guest with a virtual gpu and
host passthrough gpu
Poking the internet around wayland and nvidia, I found that gdm3 is
set up to deactivate wayland if it finds nvidia is available but not fully
configured. This is why gdm does not show the Wayland option for the
Gnome session. It has deliberately been disabled.
The logic for the disabling lies in /lib/udev/rules.d/61-gdm.rules
as already discussed in this bug. Sending it to /dev/null is not a
great workaround, I think.
Weiwen Hu identified a small patch to disable the disabling, so
61-gdm.rules is read, but doesn't disable wayland.
But that skips some of the nvidia checks.
Inspecting the checks themselves, /lib/udev/rules.d/61-gdm.rules is
looking for capabilities set in /proc/driver/nvidia/params, as well as
suspend/hibernate support.
Reporting the checks in order (the variables checked are the entries
in /proc/driver/nvidia/params),
1) NVIDIA_ENABLE_S0IX_POWER_MANAGEMENT
This one seems not crucial. If your card has it, then the other
checks are not needed. But it's not needed itself, it just makes
power management simpler if it is available.
2) suspend/hiberate support via scripts
/usr/bin/nvidia-sleep.sh
/usr/lib/systemd/system-sleep/nvidia
These are provided by nvidia-suspend-common, so you'll already have
them if you're using the debian nvidia packages.
3) NVIDIA_PRESERVE_VIDEO_MEMORY_ALLOCATIONS
This was not activated on my system. I had to activate it in
/etc/modprobe.d/nvidia-options.conf with
options nvidia-current NVreg_PreserveVideoMemoryAllocations=1
(i.e. uncommenting the line already in the file)
4) systemctl is-enabled nvidia-hibernate
systemctl is-enabled nvidia-resume
systemctl is-enabled nvidia-suspend
These were already enabled on my system, no other action needed
(except perhaps kernel option nvidia_drm_modeset=1)
5) gdm_hybrid_graphics_check_end
gdm_virt_passthrough_check_end
gdm_virt_multi_gpu_check_end
Perhaps these tests are relevant to more complex installations
(virtual GPUs). They test /run/udev/gdm-machine-has-* scripts.
The debian package isn't providing them so these tests jump
past gdm_disable_wayland (they don't cause the disbling)
So the main action I needed to take was to set the driver option
NVreg_PreserveVideoMemoryAllocations=1
to get NVIDIA_PRESERVE_VIDEO_MEMORY_ALLOCATIONS defined.
Looks like modesetting should also be activated.
These can both be done in
/etc/modprobe.d/nvidia-options.conf
options nvidia-current NVreg_PreserveVideoMemoryAllocations=1
options nvidia_drm modeset=1
For hibernation, Archlinux also suggests setting the tmp path to a
location with enough space to hold the GPU memory,
see https://wiki.archlinux.org/title/NVIDIA/Tips_and_tricks#Preserve_video_memory_after_suspend
e.g.
options nvidia-current NVreg_TemporaryFilePath=/var/tmp
There is some chatter on the internet about activating nvidia_drm option
fbdev=1, but we don't currently need it since it's not supported in
our current 545 driver version, and might be set by default in the
future versions anyway, see
https://forums.developer.nvidia.com/t/555-release-feedback-discussion/293652/14
https://forums.developer.nvidia.com/t/drm-fbdev-wayland-presentation-support-with-linux-kernel-6-11-and-above/307920/11
After setting NVreg_PreserveVideoMemoryAllocations=1 (and nvidia_drm
modeset=1), gdm3 gives me access to a wayland Gnome session,
with the nvidia driver accessible via PRIME OFFLOADING
(i.e. __NV_PRIME_RENDER_OFFLOAD=1)
https://wiki.archlinux.org/title/PRIME#PRIME_render_offload
https://gitlab.archlinux.org/archlinux/packaging/packages/nvidia-prime/-/blob/main/prime-run?ref_type=heads
Framerate with glxinfo
(with __GL_SYNC_TO_VBLANK=0,
see https://stackoverflow.com/questions/17196117/disable-vertical-sync-for-glxgears)
seems extremely poor (100 to 10 fps instead of 7000 fps0.
Hopefully it's fixed in the coming 550/560 drivers.
That said, the real world performance (tested in wine, Ctrl-r in World of Warcraft)
is fine for me (30-50 fps).
Thank you to ArchLinux for good documentation around the issue of
using Nvidia cards with linux.