Skip to content

fix(#664): the video registry's existence probes stop reaching Windows with POSIX stat - #677

Closed
localai-bot wants to merge 3 commits into
mainfrom
row/FIX-WINDOWS-POSIX-VIDEO-ENGINE
Closed

fix(#664): the video registry's existence probes stop reaching Windows with POSIX stat#677
localai-bot wants to merge 3 commits into
mainfrom
row/FIX-WINDOWS-POSIX-VIDEO-ENGINE

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Closes #664.

windows-msvc-cpu and windows-msvc-vulkan are failing on every open pull request in this repository — nine sampled across five unrelated lanes (#661, #662, #663, #601, #596, #592, #578, #638) — and one file causes all of it. src/vllm/multimodal/video_engine.cpp landed in cefacd2d0 carrying <sys/stat.h>, ::stat and S_ISDIR; scripts/check-windows-portability.py:1675-1688 flags all three under full_source_posix, which applies to every scanned source and not only the REQUIRED_CPP platform-boundary set. main was never a denominator, because the Windows jobs are PR-only and are skipped on push runs (#584).

Repaired at the source. The checker's semantics, allowlists and scope are untouched — the checker is right and the source was wrong.

The change

IsDir / Exists now take the std::error_code overloads of <filesystem> through a file-local NativePath, and <sys/stat.h> is gone.

Two things preserved deliberately

1. The non-throwing contract. ::stat reported an uninspectable path — ENAMETOOLONG, ELOOP, EACCES on a parent — by returning -1, which arrived here as a plain false and became the registry's ordinary "no such file or directory" refusal. The throwing std::filesystem::exists(p) / is_directory(p) overloads raise filesystem_error for exactly those cases. That exception escapes ReadVideoCheckpointTensorNames, whose header contract (include/vllm/multimodal/video_engine.h:125-126) is to return false with *why set, and through DescribeCheckpoint it escapes LoadVideoEngine in place of the refusal that names the registered families.

2. NativePath(). Nothing in the tree exports one to call — fs_io.cpp:31 and minimax_h3_sharded.cpp:55 both define NativePath in their own anonymous namespace, and gguf_reader.cpp:22 / safetensors_reader.cpp:29 define Utf8Path the same way. The spelling adopted here is fs_io.cpp:31's, byte-for-byte, and deliberately not minimax_h3_sharded.cpp:55's MultiByteToWideChar variant: that one throws std::invalid_argument on a malformed-UTF-8 path under _WIN32, which would reintroduce on Windows precisely the escaping exception this change removes on POSIX.

RED first, twice

The portability red is the checker itself, on the unmodified tree at the base SHA:

$ python3 scripts/check-windows-portability.py          # exit 1
ERROR: src/vllm/multimodal/video_engine.cpp:21: unguarded POSIX include/call reaches Windows
ERROR: src/vllm/multimodal/video_engine.cpp:59: unguarded POSIX include/call reaches Windows
ERROR: src/vllm/multimodal/video_engine.cpp:64: unguarded POSIX include/call reaches Windows

After: exit 0, Windows portability contract OK.

The behavioural red is what a naive rewrite silently breaks. The throwing rewrite was written first and the new test run against it:

test_video_engine.cpp:281: FATAL ERROR: REQUIRE_NOTHROW( ... ) THREW exception:
"filesystem error: status: File name too long [...]"
[doctest] test cases: 1 | 0 passed | 1 failed | assertions: 1 | 0 passed | 1 failed

No token gate, golden or e2e render could have caught this — every checkpoint they hand over is perfectly stattable. That is also why the probe is a 300-character path component and not a missing file: a missing file is ENOENT, and the throwing overloads do not throw for ENOENT, so a missing-path test passes with and without the guarantee.

Mutations

Both anchors asserted count == 1, tree restored and verified by sha256sum -c.

Mutation Result
Exists → throwing overload RED — 1 case failed, exit 1
IsDir → throwing overload SURVIVES — 12/12, 260 assertions, exit 0

The survivor is reachability, and it is measured rather than asserted. IsDir has exactly one call site (video_engine.cpp:180), gated by Exists at :176. A probe over nine pathological path shapes — ENAMETOOLONG (36), ELOOP (40), EACCES (13), ENOTDIR (20), ENOENT (2), dangling symlink, empty path, real file, real directory — produced no row in which exists() succeeded while is_directory() set an error code, because on this libstdc++ both resolve through the same stat(). Short of a TOCTOU race, no input distinguishes the two IsDir spellings through the shipped surface. The error_code overload is kept there anyway: same contract, and a later caller probing a directory without a preceding Exists would otherwise inherit the throw.

Gate

  • scripts/check-windows-portability.py: exit 0.
  • ctest -N: 423, unchanged from main, so the denominator did not drift.
  • Focused, the one suite touched — test_video_engine: 11 cases / 254 assertions → 12 cases / 260 assertions, exit 0 both times.
  • BUILD_EXIT=0, zero No space left / BFD assertion lines in the build log.

Two pre-existing reds, both verified against the base SHA and neither caused here:

  • test_cpu_x86_llamacpp_floorNO_QUIET_WINDOW (exit 4, load=118) while this branch's own -j4 build was running. Documented load artifact.
  • tests/scripts/test_check_windows_portability.py — 2 of 71 fail on this tree and identically at the base SHA (verified by stash): the ltx2.cpp / ltx2_video_vae.cpp / ltx2_audio_vae.cpp allocation-and-math case, plus one fake-runner-scoped case. Not invoked by any workflow.

Overlap with #524 — stated rather than discovered later

Open PR #524 (row/ENG-RELEASE-WINDOWS-512, "fix(release): run no-argv tests on Windows") already carries a version of this same repair. It is weaker in two exact ways:

  1. it passes the narrow std::string straight to std::filesystem, with no NativePath — so on Windows the path is interpreted in the active code page and a non-ASCII checkpoint path silently resolves to the wrong file;
  2. its added test covers directory / regular file / missing path — all ENOENT, which passes with the throwing overloads too, so it does not hold the contract it looks like it holds.

The two hunks will conflict. Whichever lands second should take this file's version wholesale. Separately, #524 still carries the tests/vt/test_backend_cross_device.cpp setenv repair that 11cc1d5896 already landed via tests/support/test_env.h.

Scope

src/vllm/multimodal/video_engine.cpp, its test, and the .agents/roadmap_v1.md issue-table row for #664. No other src/ file, and not scripts/check-windows-portability.py.

mudler added 2 commits August 13, 2026 21:14
…s with POSIX stat

windows-msvc-cpu and windows-msvc-vulkan were RED on EVERY open pull request in
this repository - nine sampled across five unrelated lanes (#661, #662, #663,
#601, #596, #592, #578, #638) - and one file caused all of it.
src/vllm/multimodal/video_engine.cpp landed in cefacd2 carrying <sys/stat.h>
at :21, ::stat + S_ISDIR at :57-60, and ::stat at :62-65.
check-windows-portability.py:1675-1688 flags all three under full_source_posix,
which applies to EVERY scanned source and not only to the REQUIRED_CPP
platform-boundary set, so no exemption applied and none is added here. main was
never a denominator: the Windows jobs are PR-only and are `skipped` on push runs
(#584), so the breakage was invisible on the branch it landed on and visible on
every branch cut from it afterwards.

Repaired at the SOURCE. The checker's semantics, its allowlists and its scope
are untouched - the checker is right and the source was wrong.

RED FIRST, twice, because there are two separate claims here.

  1. THE PORTABILITY RED is the checker itself. On the unmodified tree at
     11cc1d5:

       $ python3 scripts/check-windows-portability.py    # exit 1
       ERROR: src/vllm/multimodal/video_engine.cpp:21: unguarded POSIX include/call reaches Windows
       ERROR: src/vllm/multimodal/video_engine.cpp:59: unguarded POSIX include/call reaches Windows
       ERROR: src/vllm/multimodal/video_engine.cpp:64: unguarded POSIX include/call reaches Windows

     After: exit 0, "Windows portability contract OK".

  2. THE BEHAVIOURAL RED is what a naive rewrite silently breaks, and it is the
     reason this is 42 lines and not 6. ::stat reported an UNINSPECTABLE path -
     ENAMETOOLONG, ELOOP, EACCES on a parent - by returning -1, which arrived
     here as a plain false and became the registry's ordinary "no such file or
     directory" refusal. The THROWING std::filesystem::exists(p) /
     is_directory(p) overloads raise filesystem_error for exactly those cases
     instead. That exception escapes ReadVideoCheckpointTensorNames, whose
     header contract (include/vllm/multimodal/video_engine.h:125-126) is to
     return false with *why set, and through DescribeCheckpoint it escapes
     LoadVideoEngine in place of the refusal that names the registered families.

     So the throwing rewrite was written FIRST and the new test run against it:

       test_video_engine.cpp:281: FATAL ERROR: REQUIRE_NOTHROW( ... ) THREW
       exception: "filesystem error: status: File name too long [...]"
       [doctest] test cases: 1 | 0 passed | 1 failed | assertions: 1 | 0 passed | 1 failed

     Then the std::error_code overloads: 1 passed, 6 assertions, exit 0.

     No token gate, golden or e2e render could ever have caught this - every
     checkpoint they hand over is perfectly stattable, which is why the probe is
     a 300-character path component rather than a missing file. A MISSING file
     is ENOENT, and the throwing overloads do not throw for ENOENT, so a
     missing-path test passes with and without the guarantee.

NativePath() is file-local, and deliberately so. Nothing in the tree exports one
to call: fs_io.cpp:31 and minimax_h3_sharded.cpp:55 both define NativePath in
their own anonymous namespace, and gguf_reader.cpp:22 / safetensors_reader.cpp:29
define Utf8Path the same way. The spelling adopted here is fs_io.cpp:31's,
byte-for-byte, and NOT minimax_h3_sharded.cpp:55's MultiByteToWideChar variant -
that one THROWS std::invalid_argument on a malformed-UTF-8 path under _WIN32,
which would reintroduce on Windows precisely the escaping exception this change
removes on POSIX. The u8string step is not decoration either: on Windows a
narrow std::string handed to std::filesystem::path is interpreted in the ACTIVE
CODE PAGE, so a non-ASCII checkpoint path silently resolves to the wrong file.

MUTATIONS, both anchors asserted count == 1 and the tree restored and verified
by sha256:

  * Exists -> throwing overload   -> RED, 1 case failed, exit 1. The guard bites.
  * IsDir  -> throwing overload   -> SURVIVES, 12/12, 260 assertions, exit 0.

    The survivor is REACHABILITY, and it is measured rather than asserted. IsDir
    has exactly one call site (video_engine.cpp:180) and it is gated by
    Exists at :176. A probe over nine pathological path shapes - ENAMETOOLONG
    (36), ELOOP (40), EACCES (13), ENOTDIR (20), ENOENT (2), dangling symlink,
    empty path, real file, real directory - produced NO row in which exists()
    succeeded while is_directory() set an error code, because on this libstdc++
    both resolve through the same stat() call. Short of a TOCTOU race, no input
    can distinguish the two IsDir spellings through the shipped surface. The
    error_code overload is kept there anyway: it is the same contract, and a
    later caller that probes a directory WITHOUT a preceding Exists would
    otherwise inherit the throw.

Gate: ctest -N registers 423, unchanged from main, so the denominator did not
drift. Focused before/after on the one suite touched, test_video_engine:
11 cases / 254 assertions -> 12 cases / 260 assertions, exit 0 both times.
BUILD_EXIT=0 with zero ENOSPC or BFD-assertion lines in the build log.

agent-preflight.sh: one gate failed, test_cpu_x86_llamacpp_floor, with
NO_QUIET_WINDOW (exit 4, load=118) while this branch's own -j4 build was
running; it is the documented load artifact and it fails the same way on
unmodified main. tests/scripts/test_check_windows_portability.py fails 2 of 71
on this tree AND, verified by stash, identically at the base SHA 11cc1d5 -
pre-existing, unrelated (ltx2.cpp / ltx2_video_vae.cpp / ltx2_audio_vae.cpp
allocation-and-math, plus one fake-runner-scoped case), and not run by any
workflow.

OVERLAP, stated rather than discovered later: open PR #524
(row/ENG-RELEASE-WINDOWS-512, "fix(release): run no-argv tests on Windows")
already carries a version of this same repair. It is weaker in two exact ways -
it passes the narrow std::string straight to std::filesystem, with no
NativePath and therefore the active-code-page defect above, and its added test
covers directory / regular file / MISSING path, which is ENOENT and so passes
with the throwing overloads too. Its hunk and this one will conflict; whichever
lands second should take this file's version wholesale. #524 also still carries
the tests/vt/test_backend_cross_device.cpp setenv repair that 11cc1d5 already
landed via tests/support/test_env.h.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]
@localai-bot

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #736, which carries byte-identical content with clean commit trailers.

Why

This PR failed agent-record, and the failure was mine, not the implementer's:

d4e8dd41e504: [attribution] AI-Assisted must appear exactly once
72fb570dd191: [trailers] FOLLOWING_AGENTS_PROTOCOL must appear exactly once as a separate paragraph
72fb570dd191: [trailers] Following-Agents-Protocol must appear exactly once
72fb570dd191: [attribution] AI-Assisted must appear exactly once

As operator I merged origin/main into the branch twice using git merge --no-edit. That writes the default merge message, which carries no trailers at all, and the protocol requires them on every commit — merge commits included. Two other branches of mine hit the same thing, so it is a habit rather than a slip: a merge is a commit.

Why a new PR rather than a fixed history here

Repairing it in place means rewriting commits that are already pushed, and I am not going to force-push. So the corrected history went to a new branch instead, and this one closes with the reason attached to the diff it excuses — which is what this project uses instead of a waiver registry.

Nothing substantive was lost

The two sanitize-cpu and windows-msvc-* reds seen here are unrelated to this change and are tracked separately: the UBSan misaligned load at ltx2_loader.cpp:1325 is #627 (PR #688 open against it), and the Windows lane's remaining error C set is M_PI#720, fixed by #732.

Continue at #736.

localai-bot pushed a commit that referenced this pull request Aug 14, 2026
FOLLOWING_AGENTS_PROTOCOL

Main moved again while this waited on CI. Merged so the operator gate runs on
the tree that will actually land, and so the keyed records are reconciled by key
rather than by an accepted auto-merge.

This merge message carries the trailers deliberately: the predecessor PR (#677)
was closed because two `git merge --no-edit` commits of mine wrote the DEFAULT
message and therefore no trailers at all, reddening `agent-record`. A merge is a
commit and owes the same attribution as any other.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5[1m] [claude-code]

# Conflicts:
#	.agents/roadmap_v1.md
@localai-bot

Copy link
Copy Markdown
Collaborator Author

Closing as already landed — the change here is on main by another route.

Verified rather than assumed:

  • git diff origin/main...HEAD on this branch, after merging current main, is empty. There is no residual change to apply.
  • src/vllm/multimodal/video_engine.cpp on main has no unguarded POSIX include or call left.
  • python3 scripts/check-windows-portability.py on plain main reports Windows portability contract OK — and that is now the stricter checker from fix(GATE-WINDOWS-WARNING-POLICY): /WX- is not /WX, and OBJCXX is not the C++ compile (#774) #795, which closed the substring blindness where /WX- satisfied /WX and a # comment satisfied the whole policy.

The three video_engine.cpp errors this PR existed to fix were the entire non-baseline portion of the Windows portability gate's red. They are gone.

For anyone tracing this later: the Windows arm's remaining red is windows-msvc-cpu / windows-msvc-vulkan, which are PR-only jobs with no main baseline (#584) and fail on every PR in the repository for reasons unrelated to portability classification.

No work is lost by closing this; the fix is in the tree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

video_engine.cpp reaches Windows with unguarded POSIX stat — reddens windows-msvc-* on EVERY open PR

2 participants