Dear Debian NVIDIA Maintainers and NVIDIA Linux Driver Team,
While testing our earlier kernel fix for `nvidia-drm.ko` on Linux 7.1
(submitted in Debian Salsa MR !17:
https://salsa.debian.org/nvidia-team/nvidia-graphics-drivers/-/merge_requests/17),
we discovered a second bug in the userspace proprietary EGL driver
(`libEGL_nvidia.so.550.163.01` vs. `libnvidia-eglcore.so.550.163.01`)
that causes all `DRM_FORMAT_NV12` (and `NV21`/`NV16`/`NV24`) DMA-BUF
`EGLImage` imports to silently drop Plane 1 (the `UV` chroma plane),
resulting in a bright green tint on all videos in Google Chrome / Chromium
under Wayland (YouTube, Google Meet, WebRTC, etc.).
========================================================================
1. Why the Kernel Fix in MR !17 Exposed This Second Bug
========================================================================
In MR !17, we completed the `NV_DRM_FBDEV_AVAILABLE` backport from
565.57.01 in `nvidia-drm.ko`, which properly initializes the GEM object
(`drm_gem_object_init()`) and VMA offset node so that
`nv_drm_gem_dma_buf_mmap()` (`drm_gem_prime_mmap()`) succeeds instead of
failing with `-EINVAL`.
* Before MR !17:
When Chromium on Wayland (`OzonePlatformWayland`) attempted to allocate
mappable `GBM_FORMAT_NV12` video buffers via `gbm_bo_create(...,
GBM_FORMAT_NV12, GBM_BO_USE_SCANOUT | GBM_BO_USE_LINEAR)` and map them
with `gbm_bo_map()`, `nv_drm_gem_dma_buf_mmap()` returned `-EINVAL`.
That failure caused `WaylandSurfaceFactory::CreateNativePixmap()` to
return `nullptr`, so Chromium silently fell back to CPU
`SharedMemoryImageBacking` (uploading `Y` and `UV` via `glTexSubImage2D`
into two `GL_TEXTURE_2D` textures), masking the broken `EGL_LINUX_DMA_BUF_EXT`
`DRM_FORMAT_NV12` import path.
* After MR !17:
`gbm_bo_create()` + `gbm_bo_map()` now succeed for `GBM_FORMAT_NV12`
buffers on Wayland (`nvidia-drm_gbm.so`). Chromium therefore uses its
zero-copy `OzoneImageBacking` path and imports the 2-plane `NV12`
DMA-BUF into EGL via `eglCreateImageKHR(dpy, EGL_NO_CONTEXT,
EGL_LINUX_DMA_BUF_EXT, NULL, attrs)` bound to `GL_TEXTURE_EXTERNAL_OES`,
hitting the bug below in `libEGL_nvidia.so.550.163.01`.
========================================================================
2. Exact Root Cause in `libEGL_nvidia.so.550.163.01` vs `libnvidia-eglcore.so.550.163.01`
========================================================================
1. Format Descriptor Table in `libEGL_nvidia.so.550.163.01` (file offset `0xd41e0`):
During `eglCreateImageKHR` (`0x674d0`), `libEGL_nvidia.so.550.163.01`
looks up `EGL_LINUX_DRM_FOURCC_EXT` in a 54-entry, 56-byte (`0x38`)
format descriptor table starting at file offset `0xd41e0`:
* Entry [44] (`DRM_FORMAT_NV12` = `0x3231564e` at file offset `0xd4b80`):
- `nplanes = 2`
- Plane 0 (`Y` at `0xd4b88`): `fmt = 0x12400408`, `type = 0x0f` (15), `sub_x = 1`, `sub_y = 1`
- Plane 1 (`UV` at `0xd4b98`): `fmt = 0x10180e10`, `type = 0x0f` (15), `sub_x = 2`, `sub_y = 2`
* Entry [45] (`DRM_FORMAT_NV21` = `0x3132564e` at file offset `0xd4bb8`):
- Plane 1 (`VU` at `0xd4bd0`): `fmt = 0x10880e10`, `type = 0x0f` (15)
* Entry [46] (`DRM_FORMAT_NV16` = `0x3631564e` at file offset `0xd4bf0`):
- Plane 1 (`UV` at `0xd4c08`): `fmt = 0x10180e10`, `type = 0x0f` (15)
* Entry [47] (`DRM_FORMAT_NV24` = `0x3432564e` at file offset `0xd4c28`):
- Plane 1 (`UV` at `0xd4c40`): `fmt = 0x10180e10`, `type = 0x0f` (15)
Depending on `EGL_YUV_COLOR_SPACE_HINT_EXT` (`REC601`/`REC709`/`REC2020`)
and `EGL_SAMPLE_RANGE_HINT_EXT` (`NARROW`/`FULL`), `libEGL_nvidia.so.550.163.01`
adjusts each plane's `type` into the range `15..20` (`0x0f..0x14`) and
passes the descriptor to `libnvidia-eglcore.so.550.163.01` (`0xa04b00`).
2. Internal Format Translator in `libnvidia-eglcore.so.550.163.01` (`0xa04b00..0xa05bc5`):
In `libnvidia-eglcore.so.550.163.01`, the per-plane loop (`0xa04bb1..0xa04d69`)
loads `uint64_t key = ((uint64_t)type << 32) | fmt` into `%rsi` and
translates it to an internal hardware surface format ID in `%edi` via
a binary search tree (`0xa04c33..0xa05bc5`):
* For Plane 0 (`fmt = 0x12400408`, `type = 0x0f..0x14`), it matches and
sets `%edi = 0x0c` (8-bit 1-channel `R8`).
* For Plane 1, `libnvidia-eglcore.so.550.163.01`'s lookup tree checks for:
- `0x0000000f10880710` .. `0x0000001410880710` -> `%edi = 0x0d` (`RG8`, `.r = byte[0] = U`, `.g = byte[1] = V`)
- `0x0000000f10180710` .. `0x0000001410180710` -> `%edi = 0xda` (`GR8`, `.r = byte[1] = V`, `.g = byte[0] = U`)
It does NOT contain `0x10180e10` or `0x10880e10` for `type = 0x0f..0x14`!
* Because `libEGL_nvidia.so.550.163.01` has `0x0e10` instead of `0x0710`
in the low 16 bits of Plane 1's `fmt` (and swaps `0x1018` vs `0x1088`
for `NV12` vs `NV21`), the final leaf comparison at `0xa059bd`:
a059b3: 48 b8 10 07 88 10 14 00 00 00 movabs $0x1410880710, %rax
a059bd: 48 39 c6 cmp %rax, %rsi
a059c0: bf 0d 00 00 00 mov $0xd, %edi
a059c5: 0f 45 f8 cmovne %eax, %edi # %eax == 0!
fails and silently sets `%edi = 0` (`NV_FORMAT_NONE`) without returning
an error!
As a result:
- `eglCreateImageKHR` returns `EGL_SUCCESS` (`0x3000`).
- `glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, img)` returns `GL_NO_ERROR`.
- Plane 0 (`Y` luma) is bound and sampled normally.
- Plane 1 (`UV` chroma) has hardware format `0` (unbound) and always
samples as `(0.0, 0.0)`, which after YUV-to-RGB conversion produces
a bright green image (`R = Y - 0.88, G = Y + 0.53, B = Y - 1.08`).
3. Secondary Missing FourCC (`DRM_FORMAT_GR88`):
In addition, `libEGL_nvidia.so.550.163.01`'s FourCC tables (`0xd4020`
and `0xd41e0`) include `DRM_FORMAT_RG88` (`0x38384752`, entry [5]) but
omit `DRM_FORMAT_GR88` (`0x38385247`). When Chromium attempts per-plane
`GL_TEXTURE_2D` DMA-BUF import of `NV12` Plane 1 using
`gfx::BufferFormat::RG_88` -> `DRM_FORMAT_GR88`, `eglCreateImageKHR`
fails with `EGL_BAD_MATCH (0x3009)`.
========================================================================
3. Binary Patch Verification & Standalone Reproducer
========================================================================
Patching Plane 1's `fmt` in `libEGL_nvidia.so.550.163.01` at:
- `0xd4b98` (`NV12` Plane 1): `0x10180e10` -> `0x10880710`
- `0xd4bd0` (`NV21` Plane 1): `0x10880e10` -> `0x10180710`
- `0xd4c08` (`NV16` Plane 1): `0x10180e10` -> `0x10880710`
- `0xd4c40` (`NV24` Plane 1): `0x10180e10` -> `0x10880710`
completely resolves the bug:
* Neutral Gray `NV12` (`Y=64, U=128, V=128`):
- Unpatched `libEGL_nvidia.so.550.163.01`: `RGBA = (0, 150, 0, 255)` (BRIGHT GREEN)
- Patched `libEGL_nvidia.so.550.163.01`: `RGBA = (56, 56, 56, 255)` (Neutral Gray)
* Pure Red `NV12` (`Y=82, U=90, V=240`):
- Unpatched `libEGL_nvidia.so.550.163.01`: `RGBA = (0, 172, 0, 255)` (BRIGHT GREEN)
- Patched `libEGL_nvidia.so.550.163.01`: `RGBA = (255, 25, 0, 255)` (Pure Red)
Below is a minimal standalone C reproducer (`gcc test_nv12.c -lgbm -lEGL -lGLESv2 -ldrm`):
```c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <gbm.h>
#include <drm/drm_fourcc.h>
#include <EGL/egl.h>
#include <EGL/eglext.h>
#include <GLES2/gl2.h>
#include <GLES2/gl2ext.h>
int main(void) {
int drm_fd = open("/dev/dri/renderD128", O_RDWR);
struct gbm_device *gbm = gbm_create_device(drm_fd);
EGLDisplay dpy = eglGetPlatformDisplay(EGL_PLATFORM_GBM_KHR, gbm, NULL);
eglInitialize(dpy, NULL, NULL);
const EGLint cfg_attrs[] = { EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT, EGL_NONE };
EGLConfig cfg; EGLint num_cfg;
eglChooseConfig(dpy, cfg_attrs, &cfg, 1, &num_cfg);
const EGLint ctx_attrs[] = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE };
EGLContext ctx = eglCreateContext(dpy, cfg, EGL_NO_CONTEXT, ctx_attrs);
eglMakeCurrent(dpy, EGL_NO_SURFACE, EGL_NO_SURFACE, ctx);
uint32_t w = 64, h = 64;
struct gbm_bo *bo = gbm_bo_create(gbm, w, h, GBM_FORMAT_NV12,
GBM_BO_USE_SCANOUT | GBM_BO_USE_LINEAR);
uint32_t stride0 = 0, stride1 = 0;
void *map0 = NULL, *map1 = NULL;
uint8_t *p0 = gbm_bo_map(bo, 0, 0, w, h, GBM_BO_TRANSFER_WRITE, &stride0, &map0);
for (uint32_t y = 0; y < h; y++) memset(p0 + y * stride0, 82, w); /* Y = 82 (Red) */
gbm_bo_unmap(bo, map0);
uint8_t *p1 = gbm_bo_map(bo, 0, 0, w / 2, h / 2, GBM_BO_TRANSFER_WRITE, &stride1, &map1);
for (uint32_t y = 0; y < h / 2; y++) {
for (uint32_t x = 0; x < w / 2; x++) {
p1[y * stride1 + x * 2 + 0] = 90; /* U (Cb) = 90 */
p1[y * stride1 + x * 2 + 1] = 240; /* V (Cr) = 240 */
}
}
gbm_bo_unmap(bo, map1);
int fd0 = gbm_bo_get_fd_for_plane(bo, 0);
int fd1 = gbm_bo_get_fd_for_plane(bo, 1);
uint64_t mod0 = gbm_bo_get_modifier(bo);
EGLint attrs[] = {
EGL_WIDTH, w, EGL_HEIGHT, h,
EGL_LINUX_DRM_FOURCC_EXT, DRM_FORMAT_NV12,
EGL_DMA_BUF_PLANE0_FD_EXT, fd0,
EGL_DMA_BUF_PLANE0_OFFSET_EXT, gbm_bo_get_offset(bo, 0),
EGL_DMA_BUF_PLANE0_PITCH_EXT, gbm_bo_get_stride_for_plane(bo, 0),
EGL_DMA_BUF_PLANE0_MODIFIER_LO_EXT, (EGLint)(mod0 & 0xffffffff),
EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT, (EGLint)(mod0 >> 32),
EGL_DMA_BUF_PLANE1_FD_EXT, fd1,
EGL_DMA_BUF_PLANE1_OFFSET_EXT, gbm_bo_get_offset(bo, 1),
EGL_DMA_BUF_PLANE1_PITCH_EXT, gbm_bo_get_stride_for_plane(bo, 1),
EGL_DMA_BUF_PLANE1_MODIFIER_LO_EXT, (EGLint)(mod0 & 0xffffffff),
EGL_DMA_BUF_PLANE1_MODIFIER_HI_EXT, (EGLint)(mod0 >> 32),
EGL_YUV_COLOR_SPACE_HINT_EXT, EGL_ITU_REC601_EXT,
EGL_SAMPLE_RANGE_HINT_EXT, EGL_YUV_NARROW_RANGE_EXT,
EGL_NONE
};
PFNEGLCREATEIMAGEKHRPROC eglCreateImageKHR =
(PFNEGLCREATEIMAGEKHRPROC)eglGetProcAddress("eglCreateImageKHR");
PFNGLEGLIMAGETARGETTEXTURE2DOESPROC glEGLImageTargetTexture2DOES =
(PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)eglGetProcAddress("glEGLImageTargetTexture2DOES");
EGLImageKHR img = eglCreateImageKHR(dpy, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, attrs);
GLuint tex; glGenTextures(1, &tex);
glBindTexture(GL_TEXTURE_EXTERNAL_OES, tex);
glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, img);
GLuint fbo, rtex;
glGenFramebuffers(1, &fbo); glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glGenTextures(1, &rtex); glBindTexture(GL_TEXTURE_2D, rtex);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, w, h, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, rtex, 0);
const char *vs_src = "attribute vec2 pos; varying vec2 uv; void main() { uv = pos * 0.5 + 0.5; gl_Position = vec4(pos, 0.0, 1.0); }";
const char *fs_src = "#extension GL_OES_EGL_image_external : require\nprecision mediump float; varying vec2 uv; uniform samplerExternalOES tex; void main() { gl_FragColor = texture2D(tex, uv); }";
GLuint vs = glCreateShader(GL_VERTEX_SHADER); glShaderSource(vs, 1, &vs_src, NULL); glCompileShader(vs);
GLuint fs = glCreateShader(GL_FRAGMENT_SHADER); glShaderSource(fs, 1, &fs_src, NULL); glCompileShader(fs);
GLuint prog = glCreateProgram(); glAttachShader(prog, vs); glAttachShader(prog, fs); glLinkProgram(prog);
glUseProgram(prog);
float verts[] = { -1, -1, 1, -1, -1, 1, 1, 1 };
GLint pos_loc = glGetAttribLocation(prog, "pos");
glEnableVertexAttribArray(pos_loc);
glVertexAttribPointer(pos_loc, 2, GL_FLOAT, GL_FALSE, 0, verts);
glViewport(0, 0, w, h);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
uint8_t px[4] = {0};
glReadPixels(w / 2, h / 2, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, px);
printf("Sampled NV12 Red (Y=82, U=90, V=240) -> RGBA = (%d, %d, %d, %d)\n",
px[0], px[1], px[2], px[3]);
return 0;
}
```
Thank you,
Raul E Rangel <rrangel@google.com>