fix(vulkan): the 27B load held the model TWICE on a unified box (#203)

[localai-bot]

Fixes #203.

On a unified-memory box the Vulkan backend held two full copies of the
weights. For Qwen3.6-27B that is an extra 50 GiB, and it hard-rebooted this
GB10 twice in one session with NVRM ... NV_ERR_NO_MEMORY @ mem_desc.c:1359.

Measured

Qwen3.6-27B bf16 (50.89 GiB), GB10, one binary, VT_ADOPT_DEVICE_BYTES A/B:

arm	Vulkan live	buffers	VmRSS	MemAvailable floor	outcome
OFF (old behaviour)	50.755 GiB	863 of 894	100.759 GiB	13.85	watchdog killed it, still allocating
ON (now default)	50.756 GiB	894	53.413 GiB	47.33	completed

The gap is a flat 50.003 GiB across the last four high-water lines — one
whole extra copy of the model. Qwen3-4B: VmHWM 16.392 → 9.607 GiB. The unfixed
27B never completed, so 100.759 GiB is a lower bound.

Cause

ResidentWeight uploaded each weight and kept OwnedTensor.bytes as well.
Vulkan allocates every buffer HOST_VISIBLE | HOST_COHERENT | DEVICE_LOCAL and
persistently mapped, so on unified memory both copies are the same RAM.
src/vllm/platforms/vulkan.cpp had reasoned "there is exactly one copy of the
bytes"; dense_attn_block.h:190 made a second one.

The investigation refuted my own first diagnosis

I briefed this row with "the Vulkan loader lacks the CUDA path's shard release".
That was wrong, and the row was told not to build on it. Windowed release is
not the defect and it does fire: during load the RSS holds one copy, not the mmap
as well. The other candidates were measured and cleared too:

• No allocator waste. requested == committed exactly, every run — 50.756
  GiB over 894 buffers. Nothing to win from suballocation, and
  maxMemoryAllocationCount is nowhere near 894.
• No transient held too long. A staging or dequant scratch would show as a
  bump; the excess is a constant equal to the model.
• Page cache is not the trigger. It tracks copied bytes 1:1 and is
  reclaimable. (Worth recording: MADV_DONTNEED on a private file mapping does
  not evict page cache — that needs POSIX_FADV_DONTNEED.)
• The 89.72 GiB Vulkan heap was never the binding constraint. The machine was.

A first watchdog keyed on MemFree killed a healthy load at 11.46 GiB free while
MemAvailable was 60.65 — recorded as a negative result.

Fix

AdoptDeviceBytesAsHost re-points bytes at the device allocation via the
existing OwnedBytes borrow, keyed alive by d_dev's control block. An adoption,
not a release: every .bytes reader sees the same bytes, so unlike ReleaseHost
it needs no "device path committed" proof. Gated on a new
Backend::DeviceMemoryIsHostAddressable(), default false, deliberately
narrower than UnifiedMemory() — CUDA on GB10 is unified yet cudaMalloc is not
host-dereferenceable. Discrete and non-Vulkan paths are byte-identical.

Gates, operator-verified on GB10 with the real driver

All four arms (VT_ADOPT_DEVICE_BYTES x VT_VULKAN_INFLIGHT), re-run by me
rather than taken on report:

• test_vulkan_backend 2650/2650 assertions in all four
• test_opt_paged_engine 6/6 prompts token-exact (96/96), 0 declines, in all
  four, on device type 3
• test_backend_cross_device 11/11 (132)

On llvmpipe from a clean build: test_vulkan_backend 35/35 (2107),
test_backend_cross_device 11/11, test_opt_paged_engine 6/6 token-exact on
device type 3, and the three new mechanism tests 7/7 (45 assertions).
gen-vulkan-spirv.py --check clean.

The row's three mechanism tests each go red under mutation — no-op body (3
assertions), dropped guard (3), dropped keep-alive (2, a real use-after-free
reading 128 where 167 was written).

A trap this uncovered, worth knowing repo-wide

VLLM_CPP_DEVICE is read nowhere in the tree. The engine selects via
CurrentPlatform(). The campaign's recorded gate command
VLLM_CPP_DEVICE=vulkan test_opt_paged_engine has been selecting Vulkan only
because those builds had no CUDA compiler; with CUDA on PATH the identical
command reports device type 1 and passes 6/6 on CUDA. Every Vulkan gate in
this PR was therefore checked for device type 3 explicitly, and so were the
barrier gates in #198 (confirmed type 3 in all four arms). The env var gives
false confidence and the docs should stop implying it selects anything.

Left open, named

Peak is now the load phase: the host build reaches ~51 GiB before the first
upload. Copying from the mmap straight into the device buffer would cut that too.

Not tested

No CUDA or Metal run — argued byte-identical from the default-false gate, not
measured. Only Qwen3-4B and Qwen3.6-27B. test_qwen36_weights has one
pre-existing failure on a CUDA-less build (in_proj_qkv_fp8 behind
#ifdef VT_CUTLASS_FP8) that is unrelated and touches no Backend.

FOLLOWING_AGENTS_PROTOCOL