logprobs=-1 crashes the engine: the sampler's raw-vocab shape is read as the gathered one

[localai-bot]

logprobs=-1 ("give me every vocab entry") is accepted by SamplingParams and
reaches the sampler, which takes a different tensor shape for it than for any
logprobs=k — and the engine-side consumer reads that shape out of bounds.

Where it breaks

src/vllm/v1/sample/sampler.cpp:344-352 — for num_logprobs == -1 the sampler
builds a LogprobsTensors that carries ONLY logprobs (the raw [n, vocab]
block), leaving logprob_token_ids and selected_token_ranks empty, with
num_tokens_per_position = vocab:

LogprobsTensors lt;
lt.num_positions = static_cast<int>(n);
lt.num_tokens_per_position = static_cast<int>(vocab);
lt.logprobs = raw_logprobs;          // ids + ranks deliberately left empty
logprobs_tensors = std::move(lt);

src/vllm/v1/engine/logprobs.cpp:51-73 (UpdateSampleLogprobs) then indexes all
three arrays for every position:

const int width = lists.num_tokens_per_position;
if (width <= 0) return;              // the only guard -- vocab is > 0, so it passes
const int32_t* id_row = &lists.logprob_token_ids[base];      // EMPTY -> OOB
const int rank = lists.selected_token_ranks[pos];            // EMPTY -> OOB

The guard at :55 says "the num_logprobs==-1 raw-vocab shape is not wired", but
it only catches width <= 0, which this shape never is.

Repro

Any request through the engine with SamplingParams::logprobs = -1:

SamplingParams sp;  sp.temperature = 0.0;  sp.max_tokens = 1;
sp.logprobs = -1;
engine.generate(prompt_token_ids, sp, "req");   // SIGSEGV

Observed as SIGSEGV in test_llm_engine while writing the gate for
#223; the -1 prompt-logprobs
path is unaffected (the runner widens -1 to vocab_size before gathering, so it
produces the ordinary k+1 shape).

Expected

vLLM's sampling_params.py treats logprobs=-1 as "all logprobs" and
gpu_input_batch.py:435-440 stores vocab_size, so the sampler produces the
same [n, vocab+1] gathered shape as any other k and nothing downstream needs a
special case. Either mirror that (widen at admission, drop the special shape) or
teach UpdateSampleLogprobs the raw shape. The first is what upstream does.

Row: SAMPLE-LOGPROBS / SAMPLE-LOGPROB-TOKEN-IDS (.agents/engine-matrix.md:131,133).

[localai-bot]

Correction to the "Expected" section above. I wrote that upstream has no
special case for logprobs=-1. That is wrong, and worth stating plainly before
anyone fixes this on my word: vllm/v1/sample/sampler.py:120-131 at the pin
carries exactly the branch we ported —

if num_logprobs is None:
    logprobs_tensors = logprob_token_ids_tensors
elif num_logprobs == -1:
    # Return the full unsorted and unranked logprobs.
    logprobs_tensors = LogprobsTensors(torch.empty(0), raw_logprobs, torch.empty(0))
else:
    logprobs_tensors = self.gather_logprobs(raw_logprobs, num_logprobs, token_ids=sampled)

So src/vllm/v1/sample/sampler.cpp:344-352 is a faithful port, not the defect.

The actual divergence is one layer up. Upstream's max_num_logprobs is fed from
input_batch.num_logprobs, and gpu_input_batch.py:434-440 widens the sentinel
at admission:

if sampling_params.logprobs is not None:
    self.num_logprobs[req_id] = (
        self.vocab_size if sampling_params.logprobs == -1 else sampling_params.logprobs
    )

max_num_logprobs can therefore never be -1 on the V1 path, which makes
upstream's elif num_logprobs == -1 branch unreachable — defensive code for
a value the input batch cannot produce. A user request for logprobs=-1 gets
the ordinary gathered [n, vocab+1] shape, and every consumer downstream reads
one shape.

We preserve the -1 sentinel instead (input_batch.cpp:292-297, and
max_num_logprobs() at :481-497 propagates it deliberately — it is written
down as a recorded deviation in input_batch.h). That routes live requests into
the branch upstream never reaches, and UpdateSampleLogprobs reads it as the
gathered shape. The segfault is the deviation, not the port.

Fix

Mirror gpu_input_batch.py:434-440: widen -1 to vocab_size at admission and
drop the sentinel from max_num_logprobs(). The sampler's -1 branch then
becomes exactly as dead as upstream's, one shape reaches every consumer, and the
recorded deviation goes away rather than growing a second reader.