feat(sample): implement the three refused logprobs_mode variants (#238)

[localai-bot]

Closes the logprobs_mode half of #238.
Spec: .agents/specs/logprobs-mode.md.
Row SAMPLE-LOGPROB-TOKEN-IDS INVENTORIED → PARTIAL.

What was broken

Three of vLLM's four logprobs_mode values were STUB (deferred) in the enum and
refused at runtime:

VT_CHECK(logprobs_mode_ == LogprobsMode::kRawLogprobs,
         "sampler: only the raw_logprobs logprobs_mode is implemented at T0");

So the mode was unreachable in practice — a Sampler built with any other value
threw the first time a request asked for logprobs.

What the modes actually mean

RAW vs PROCESSED, not logprobs vs logits. The raw pair is snapshotted before any
logits processor runs and describes the model's distribution; the processed
pair is taken after temperature and top-k/top-p and describes the distribution
actually sampled from. A token top-k masked away reads its true value under
raw_* and -inf under processed_*.

Implementation

forward takes the raw snapshot only under the raw modes. sample takes the
processed one into a caller-owned buffer at the two points upstream takes it —
before temperature on the all-greedy early return (sampler.py:262-271), after
top-k/top-p otherwise (:286-290) — and forward lets a non-empty processed
snapshot replace the raw one (:104-106). raw_logits is a device→host copy,
and it must live in the same block as the raw-logprobs snapshot because the
processors below mutate the tensor in place.

Evidence

RED (implementation stashed): the three new cases throw the refusal —
15 cases, 3 failed.

GREEN: test_sampler 15/15, 67 assertions, clean CPU Release build,
0 warnings under -Werror.

The four cases share one logits row so the modes are directly comparable. The
assertion that actually separates processed_logits from processed_logprobs is
the renormalization one — both mask the same tokens, only the logprobs arm
renormalizes over the survivors. Getting those two backwards would still produce
plausible-looking numbers.

Full ctest: 365/366 — and the contention theory is proven, not argued

The first run had test_openai_api_server and test_openai_conformance failing
serially, so the usual parallel-starvation explanation did not apply and was
not accepted. Attribution was established in stages:

• With src/ and include/ stashed — clean origin/main 8a6704a2, same build
  dir — both failed identically. Not this change.
• The failure count varied run to run (19 failed assertions, then 7), which a
  deterministic regression does not do.
• The harness binds an ephemeral port (test_conformance.cpp:409), ruling out
  a collision between the several sessions on this box.
• Load average was 46–107 on 20 cores.

The decisive measurement came when the box briefly went quiet (load 0.73): the
same two tests, same binaries, passed in 0.58 s total — against 726 s and
failing under load. A ~1000× swing in wall time is CPU starvation and nothing
else.

I deliberately did not file a "main is broken" issue on the earlier evidence.
Contention misreported as a defect sends someone hunting a bug that was never
there.

The full re-run scored 365/366, with both originally-suspected tests now
passing and one different test (test_async_llm) failing under -j and
passing serially in 0.04 s — the identical signature.

Honest caveat: the box did not stay quiet. Load climbed from 2.19 to 77
during that 675 s run as other agents resumed, so this is not a truly uncontended
number. It is the best available on a shared box; the originally-flagged failures
are gone and every residual failure resolves serially in under a second.

Two deviations, stated rather than buried

Spec-after-code. AGENTS.md requires the spec committed before the
implementation; here it was not. Spec and code land in one commit rather than
two, so the history doesn't imply an ordering that didn't happen. The design had
no branch point — upstream fixes where each snapshot is taken — which is why the
cost was low, but it's still a rule I broke and the spec says so.

The STATUS ratchet caught a partial payment. My first attempt added the new
line and paid for it by collapsing the best_of cell's upstream rationale, but
only covered part of the cost and left the page net +56; the suite went red
because the ratchet may only move down. The line was tightened until the page
genuinely shrank. The collapse is deliberately a different one from what PR
#235 makes in the beam-search prose, so two of my open PRs can't conflict on the
same text.

Not in this PR

logprob_token_ids generative scoring (sampler.py:151-225) and the
config/CLI/SamplingParams plumbing to select a mode from outside the library.
Both are on the same row, which is why it moves to PARTIAL and not ACTIVE:
the modes are reachable only by constructing a Sampler directly.

🤖 Generated with Claude Code