fix(record): the logprobs=-1 SIGSEGV was in slice_request, not the processor (#231)

[localai-bot]

Record repair. The logprobs=-1 fix landed correctly as
fd9af7d9 (merged
723d96a8, PR #236) — the
admission-time widening is right and is not touched here. What shipped wrong
was the record of where the process died, plus one false sentence about the HTTP
surface. Both are now the project's published account, which is why this is its
own PR rather than a footnote.

Follow-up to #231. Spec:
.agents/specs/logprobs-all-sentinel.md.
Row SAMPLE-LOGPROBS stays DONE.

1. The crash site was misattributed, in five shipped places

The record says the SIGSEGV was inside LogprobsProcessor::UpdateSampleLogprobs.
It is inside LogprobsTensors::slice_request (src/vllm/v1/outputs.cpp:31-37),
called from src/vllm/v1/core/sched/scheduler.cpp:920-924.

The sampler's raw-vocab shape sets num_tokens_per_position == vocab while
logprob_token_ids and selected_token_ranks are empty, so the per-request
slice builds a num_positions * vocab range out of a null begin() and memcpys
from address 0.

UpdateSampleLogprobs is a real second consumer with the identical defect — it
indexes the same two empty arrays, and its width <= 0 guard never screened that
shape, whose width is the vocab. It is simply never reached: the scheduler slices
before the output processor runs. The reviewer's instrumented proof, reproduced
here before editing anything (both consumers instrumented, widening reverted):

[INSTR] ENTER slice_request req_idx=0 num_positions=1 ntpp=24 ids=0 lps=24 ranks=0
[doctest] TEST CASE:  llm_engine: logprobs=-1 returns a full-vocab logprobs dict
FATAL ERROR: test case CRASHED: SIGSEGV - Segmentation violation signal

UpdateSampleLogprobs never prints.

Upstream's own slice_request (outputs.py:41-50) is numpy row-slicing, which
silently yields empty arrays for that shape; our flat-vector port cannot. It is a
faithful port of a call upstream never makes on this shape.

Why it matters beyond accuracy: it changes what the fix is for. The widening
fixes two consumers, not one — which is precisely the argument for fixing it at
admission instead of inside a consumer. The shipped write-up made that argument
from a single-consumer premise.

Corrected in: the spec's Our baseline, Tests to port and Outcome; the
SAMPLE-LOGPROBS cell in .agents/engine-matrix.md; and the section-8 comment in
tests/vllm/v1/test_llm_engine.cpp.

2. docs/USAGE.md shipped a false sentence — and hid a real capability

Shipped: "(Over HTTP the OpenAI logprobs field keeps its own 0..5 range.)", and
the spec's Scope excluded the HTTP surface on that premise.

Nothing in our tree enforces any range. grep max_logprobs src include returns
nothing; protocol.cpp:520 is a bare assignment; sampling_params.cpp:110 rejects
only < 0 && != -1; upstream's check_logprobs is not ported.

Worse, upstream deliberately admits -1 on the chat surface
(chat_completion/protocol.py:785-796: "top_logprobs must be a positive value or
-1"), and both ends of that path already existed here:

• ChatCompletionRequest::to_sampling_params (protocol.cpp:562) maps
  top_logprobs straight into sp.logprobs;
• ChatTopLogprobs (serving_utils.cpp:165) already reads -1 as "keep every
  entry".

Only the middle was broken. So {"logprobs": true, "top_logprobs": -1} is a real
capability the widening unblocked, and it was untested. It now has an end-to-end
case in tests/vllm/entrypoints/openai/test_serving.cpp — kVocab distinct
entries per generated token, summing to 1.0, serialized into the JSON body — which
is RED (SIGSEGV in slice_request) with the widening reverted.

USAGE now says what is true, including that {"logprobs": -1} on the completion
surface returns empty top_logprobs maps where upstream answers 400
(completion/protocol.py:496-500). Porting that validation is
#249, filed separately and
deliberately not done here.

3. Two smaller repairs

Stale comment (src/vllm/v1/engine/logprobs.cpp:55) claimed the
num_logprobs==-1 raw-vocab shape "is not wired". It never screened that shape
(width == vocab passes it), and since the widening the shape is unreachable from
a live request. It is a plain zero-width guard and now says so.

A test that could not fail. llm_engine: a finite logprobs count is unaffected by the -1 widening asserted >= 1 && <= 3, which no mutation of the widening can
break. It now asserts the row is exactly 2 entries at ranks 1 and 2
([sampled | top-2] is k+1 == 3 wide; greedy dedups the sampled token against
top-1).

The comment also records what that case honestly cannot cover: the sampler-side
gather width. AppendLogprobsForNextPosition truncates to the request's own count,
so widening every request to vocab_size leaves the client payload
byte-identical and no engine-level assertion can see it. That width is pinned where
it is observable — mutating add_request to widen unconditionally turns two
existing input-batch assertions RED, verified:

test_input_batch.cpp:641: ERROR: CHECK( *md.max_num_logprobs == 4 ) is NOT correct!
test_input_batch.cpp:687: ERROR: CHECK( batch.num_logprobs.at("a") == 3 ) is NOT correct!

(The reviewer suggested asserting exactly 3 here. That value is wrong — greedy
dedup makes it 2, verified by running it — and neither 2 nor 3 can see the widening
mutation, so the fix is the exact width plus an honest note about where the
sampler-side width actually is pinned.)

No behaviour change

add_request, max_num_logprobs() and the sampler are untouched. The only
executable additions are one test case and one strengthened assertion; everything
else is comments and records.

Evidence

RED — new/repaired cases with the widening reverted:

TEST CASE:  llm_engine: logprobs=-1 returns a full-vocab logprobs dict
FATAL ERROR: test case CRASHED: SIGSEGV - Segmentation violation signal

TEST CASE:  serving_chat: top_logprobs=-1 returns every vocab entry per token
FATAL ERROR: test case CRASHED: SIGSEGV - Segmentation violation signal

GREEN — focused:

test_llm_engine      13/13 test cases, 231 assertions
test_input_batch     26/26 test cases, 190 assertions
test_openai_serving  42/42 test cases, 556 assertions

Full gate — clean CPU Release build, zero warnings under -Werror:

Full ctest -j 6: 363/365 green in one pass (cut at 364/365 by the harness, not by
a failure). The two socket-based server suites starved under FOREIGN load -- two
other agent sessions were running their own full ctests on this box concurrently,
load average peaked at 260 on 20 cores -- and both PASS on serial re-run as it
quietened:

test_openai_api_server PASSED on serial re-run (load ~94)
test_openai_conformance PASSED on serial re-run (load ~53)

Every one of the 365 tests has passed.

Not attributable to this branch, and provably so: the only src/ edit is five
comment lines, and the object file is BYTE-IDENTICAL to main's --
logprobs.cpp.o md5 5e63b2d498593e6d42d84e7bf983c45f with main's version and
with mine. libvllm.a is therefore bit-identical, and those two suites compile no
file this branch touches: they run exactly the binary main produces.

Branch note

These repairs were first prepared on row/SAMPLE-LOGPROBS-ALL (PR #236). While
that was in flight the same fix landed on main as fd9af7d9 (merge 723d96a8,
from row/SAMPLE-LOGPROBS-ALL-SENTINEL), so #236's content is now on main and
force-pushing the repairs there would have repaired a branch nobody merges. That
branch is left untouched at d75de383 and the repairs are re-based onto
current main (1c1749cb) here instead. PR #236 is obsolete and wants closing
with that reason. .agents/coordination.md carries an entry for this branch
pointing back at the original one, so the history stays followable.

The sixth place the wrong attribution shipped is fd9af7d9's own commit message,
which is immutable now; this PR's message and the records supersede it.

🤖 Generated with Claude Code