WEBKIT_DISABLE_DMABUF_RENDERER crashes WebKitGTK on NVIDIA instead of falling back to shared memory

[kwsantiago]

webkit_rendering.rs sets WEBKIT_DISABLE_DMABUF_RENDERER=1 on NVIDIA GPUs and on AppImage packaging to avoid the #2338 startup abort, with the stated intent of falling back to the shared-memory buffer path.

On current WebKitGTK that variable no longer falls back. It returns before mode.add(RendererBufferTransportMode::SharedMemory), so the transport mode is left empty, AcceleratedBackingStore::create() returns nullptr, and webkitWebViewBaseEnterAcceleratedCompositingMode() dereferences it, guarded only by an ASSERT, which is compiled out in release builds. The app SIGSEGVs the first time content needs a compositing layer. Switching workspaces does it every time.

#0 AcceleratedBackingStore::update()   AcceleratedBackingStore.cpp:808
#1 WebPageProxy::enterAcceleratedCompositingMode()
#2 DrawingAreaProxyCoordinatedGraphics::enterAcceleratedCompositingMode()
   <- IPC Messages::DrawingAreaProxy::EnterAcceleratedCompositingMode

Measured in the failing process: transport mode 0x00000000, create() returned 0x0, and priv->acceleratedBackingStore is NULL in the core dump. The webview is never disposed (breakpoint on webkitWebViewBaseDispose never fires), so the pointer is null from creation rather than from teardown.

WEBKIT_DMABUF_RENDERER_FORCE_SHM=1 returns after SharedMemory is added. That is the documented intent, no hardware dmabuf path, and it leaves the backing store valid.

Verification

Ubuntu 24.04, WebKitGTK 2.52.3, GeForce RTX 3050 Ti (driver 580.173.02), X11.

build	module log line	workspace switch
0.4.24 AppImage (ships DISABLE_DMABUF)	WEBKIT_DISABLE_DMABUF_RENDERER=1 — NVIDIA GPU, AppImage	SIGSEGV
main with HEURISTIC = [FORCE_SHM]	WEBKIT_DMABUF_RENDERER_FORCE_SHM=1 — NVIDIA GPU	works

Same result running the AppImage with the two variables set by hand, so the variable is the only difference and not the build.

Both runs pin EGL to Mesa (__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/50_mesa.json). Debian and Ubuntu ship disable-nvidia-dmabuf.patch in webkit2gtk, which empties the same transport mode on NVIDIA regardless of this variable; without the pin that patch masks the difference. It also means this change alone will not fix Debian/Ubuntu users. I am reporting that to Debian separately.

Not verified

Whether FORCE_SHM still avoids the #2338 abort. I do not have hardware that reproduces #2338, and that is the reason the setting exists, so it is worth confirming before changing it.

If the swap looks right I am happy to send the PR. The module's unit tests pin the current variable, so they would need updating too, and OWNED would need a decision: adding FORCE_SHM keeps the "everything this module may set" invariant, but changes which user assignments make it stand down.

[orestes74]

Confirming this on DGX Spark (Ubuntu 24.04 ARM64, NVIDIA GPU 10de:2e12, WebKitGTK 2.52.3, X11), Buzz v0.5.2→v0.5.3 built from source.

Symptoms matched exactly: just production completes successfully, Vite/buzz-desktop/WebKitWebProcess all running, but the window renders transparent/blank, or only the first frame of the splash screen shows before rendering freezes.

FORCE_SHM alone wasn't enough for me — I had to explicitly override the NVIDIA heuristic:

bash
WEBKIT_DISABLE_DMABUF_RENDERER=0
WEBKIT_DMABUF_RENDERER_FORCE_SHM=1
just production

Without WEBKIT_DISABLE_DMABUF_RENDERER=0, the app's own NVIDIA detection in webkit_rendering.rs re-sets DISABLE_DMABUF_RENDERER=1 and FORCE_SHM alone doesn't override it.

Also noticed: adding a temporary webview.open_devtools() call in .on_webview_ready() (guarded by #[cfg(debug_assertions)]) forced a repaint that made the content render correctly — and it stayed rendered correctly even after closing devtools. Didn't fully confirm the mechanism, but it looks like devtools forces a resize/expose event that gets WebKit's compositor to retry creating the backing store, which succeeds once FORCE_SHM is already in effect.

Things that did not help in my case: WEBKIT_DISABLE_COMPOSITING_MODE=1 alone, WEBKIT_SKIA_ENABLE_CPU_RENDERING=1 alone, GDK_BACKEND=x11, forcing Mesa EGL (got failed to create dri2 screen / driver (null)), full reinstall, clearing WebKit/Tauri caches.

Happy to test a PR if one gets opened for this.

[Chessing234]

opened #4505 — switches the heuristic to FORCE_SHM and updates the linux troubleshooting docs.