server-context: fall back to full seq clear when partial KV eviction is refused

[ServeurpersoCom]

Overview

To reproduce on master, run llama-server with a recent hybrid attention model such as Qwen3.6-MoE, fill the KV cache with a few conversation turns, then click "Continue" at the end of an assistant reply and watch the server abort on a partial seq_rm refusal.

Additional information

Fix this

[57067] 0.52.065.769 W slot update_slots: id  0 | task 1900 | n_past was set to 1739
[57067] /root/llama.cpp.pascal/common/common.cpp:1489: failed to remove sequence 0 with p0=1739, p1=-1
[57067]
[57067] [New LWP 304479]
...
[57067] [New LWP 304336]
[57067] [Thread debugging using libthread_db enabled]
[57067] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[57067] __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56
[57067] warning: 56     ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S: Aucun fichier ou dossier de ce nom
[57067] #0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56
[57067] 56      in ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S
[57067] #1  0x00007f9d601ab668 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=0, nr=61) at ./nptl/cancellation.c:49
[57067] warning: 49     ./nptl/cancellation.c: Aucun fichier ou dossier de ce nom
[57067] #2  0x00007f9d601ab6ad in __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=0, nr=61) at ./nptl/cancellation.c:75
[57067] 75      in ./nptl/cancellation.c
[57067] #3  0x00007f9d602167c7 in __GI___wait4 (pid=<optimized out>, stat_loc=<optimized out>, options=<optimized out>, usage=<optimized out>) at ../sysdeps/unix/sysv/linux/wait4.c:30
[57067] warning: 30     ../sysdeps/unix/sysv/linux/wait4.c: Aucun fichier ou dossier de ce nom
[57067] #4  0x00007f9d6075f6bb in ggml_print_backtrace () from /root/llama.cpp.pascal/build/bin/libggml-base.so.0
[57067] #5  0x00007f9d6075f80e in ggml_abort () from /root/llama.cpp.pascal/build/bin/libggml-base.so.0
[57067] #6  0x00007f9d60dd3c1b in common_context_seq_rm(llama_context*, int, int, int) () from /root/llama.cpp.pascal/build/bin/libllama-common.so.0
[57067] #7  0x000055e24f870716 in server_context_impl::update_slots() ()
[57067] #8  0x000055e24f904e11 in server_queue::start_loop(long) ()
[57067] #9  0x000055e24f7ce42b in main ()
[57067] [Inferior 1 (process 304330) detached]
1.07.545.637 E srv    operator(): http client error: Failed to read connection

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. Opus 4.7 + MCP rootless containers with GPU access

[ggerganov]

Could you try the following patch applied to master instead:

diff --git a/tools/server/server-context.cpp b/tools/server/server-context.cpp
index 0f3fb9efa..7b801eac0 100644
--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
@@ -2583,9 +2583,9 @@ private:
                             llama_pos pos_next = slot.prompt.tokens.pos_next(n_past);
 
                             // the largest pos_min required for a checkpoint to be useful
-                            const auto pos_min_thold = std::max(0, pos_next - n_swa);
+                            const auto pos_min_thold = std::max(0, pos_next - n_swa - 1);
 
-                            if (n_past > 0 && n_past < slot.prompt.n_tokens()) {
+                            if (n_past > 0 && n_past <= slot.prompt.n_tokens()) {
                                 const auto pos_min = llama_memory_seq_pos_min(llama_get_memory(ctx_tgt), slot.id);
                                 if (pos_min == -1) {
                                     SLT_ERR(slot, "n_past = %d, slot.prompt.tokens.size() = %d, seq_id = %d, pos_min = %d\n", n_past, (int) slot.prompt.tokens.size(), slot.id, pos_min);

I think this will fix both this problem and also #23223.

[ServeurpersoCom]

Could you try the following patch applied to master instead:

diff --git a/tools/server/server-context.cpp b/tools/server/server-context.cpp
index 0f3fb9efa..7b801eac0 100644
--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
@@ -2583,9 +2583,9 @@ private:
                             llama_pos pos_next = slot.prompt.tokens.pos_next(n_past);
 
                             // the largest pos_min required for a checkpoint to be useful
-                            const auto pos_min_thold = std::max(0, pos_next - n_swa);
+                            const auto pos_min_thold = std::max(0, pos_next - n_swa - 1);
 
-                            if (n_past > 0 && n_past < slot.prompt.n_tokens()) {
+                            if (n_past > 0 && n_past <= slot.prompt.n_tokens()) {
                                 const auto pos_min = llama_memory_seq_pos_min(llama_get_memory(ctx_tgt), slot.id);
                                 if (pos_min == -1) {
                                     SLT_ERR(slot, "n_past = %d, slot.prompt.tokens.size() = %d, seq_id = %d, pos_min = %d\n", n_past, (int) slot.prompt.tokens.size(), slot.id, pos_min);

I think this will fix both this problem and also #23223.

Look better, I try now !

[ServeurpersoCom]

Successfully tested in Master+ on my PR sse-replay-buffer, It fixes the problem at its source, much better than my fallback. thanks !

[ggerganov]

Ok, let's do some testing also with non-recurrent models to make sure I am not overlooking something and we can merge.

[ServeurpersoCom]

Tested with Qwen3 30B A3B, GPT-OSS, and Llama 3.3 (all pure transformers), multi-turn continuation works as expected, no regression.