kv-cache: add LLAMA_STATE_SEQ_FLAGS_APPEND for incremental state restore

[sammshen]

Overview

state_read_meta unconditionally calls seq_rm(dest_seq_id, -1, -1) before restoring so every state_seq_set_data_ext wipes the sequence first, making chunk by chunk restore impossible (e.g. restoring chunk 1 destroys chunk 0)

This PR adds a new flag LLAMA_STATE_SEQ_FLAGS_APPEND (value 2) that skips the seq_rm call, allowing repeated state_seq_set_data_ext calls to append to the same sequence, using the existing llama_state_seq_flags mechanism (precedentLLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY is from #16382).

Exact changes:

• include/llama.h: add #define LLAMA_STATE_SEQ_FLAGS_APPEND 2
• src/llama-kv-cache.h: add flags param to state_read_meta
• src/llama-kv-cache.cpp: thread flags through state_read → state_read_meta, gate seq_rm on !(flags & APPEND)

default behavior (flags=0) is unchanged.

motivation: this enables external KV cache systems (e.g. LMCache) to store and restore opaque state blobs one chunk at a time, without reassembling into a single monolithic blob or parsing the internal serialization format.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure:

[ggerganov]

But how do you store the chunks in the first place? We currently only store full sequences.

[ggml-gh-bot]

Hi @sammshen, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[sammshen]

default no change

[sammshen]

@ggerganov thanks for the quick response!

we can use the existing API via seq_cp.

for instance:

to serialize only positions [256, 512) from seq 0, we can do:

1. seq_cp(0, TEMP_SEQ, 256, 512) // tag cells [256, 512) with metadata w/o GPU xfer
2. state_seq_get_data(ctx, buf, size, TEMP_SEQ) // serialize only cells tagged TEMP_SEQ (i.e. read only chunk's rows)
3. seq_rm(TEMP_SEQ, -1, -1) // remove temp tag (w/o GPU xfer)

so store is seq_cp + state_seq_get_data + seq_rm and restore is state_seq_set_data_ext w/ APPEND flag (from this PR

sorry I chose 256 here because I am planning on using this to integrate with LMCache's "chunk" size

[ggerganov]

Ah, yes. Interesting.

[sammshen]

@ggerganov if the change doesn't make sense, I would happy to make any needed changes (or please lmk if you think it doesn't make sense in general, I can go back to the drawing board). I can link the LMCache integration once it's ready if that helps? (even though this PR has nothing to do with LMCache)