Skip to content

<system-reminder> keeps moving, causing unnecessary prompt processing in llama.cpp #23595

Description

@jacekpoplawski

Description

For some reason, OpenCode keeps moving <system-reminder> around. As a result, the prompt history changes, so the cache cannot work correctly. In llama.cpp, this leads to a lot of unnecessary prompt processing time being wasted.

Would it be possible to keep <system-reminder> in the same place once it has been inserted for the first time? If something changes, you could add a new <system-reminder> instead.

details explained in the comment: ggml-org/llama.cpp#22031 (comment)

Plugins

No response

OpenCode version

1.14.19

Steps to reproduce

  1. Start in "plan" mode
  2. Switch to "build" mode to trigger <system-reminder>
  3. Tell opencode to do something complex, like use lots of tools and write lots of files
  4. Use any next prompt (like "thank you") and observe

<system-reminder> moves from the previous prompt to the next prompt and all the tokens (thousands!) between prompts must be processed again because cache is broken

Screenshot and/or share link

example:

 1184:  <|im_start|>user
 1185:  add one comment to it
 1186:  <system-reminder>
 1187:  Your operational mode has changed from plan to build.
 1188:  You are no longer in read-only mode.
 1189:  You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed.
 1190:  </system-reminder><|im_end|>
 1191:  <|im_start|>assistant
 1192:  <think>
 1193:  The user wants me to add a comment to the `build_norm_gated` function. I should add a descriptive comment before the function.
 1194:  </think>
 1195:
 1196:  <tool_call>

then it moves after another prompt:

 1213:  <|im_start|>assistant
 1214:  Added.<|im_end|>
 1215:  <|im_start|>user
 1216:  thank you
 1217:  <system-reminder>
 1218:  Your operational mode has changed from plan to build.
 1219:  You are no longer in read-only mode.
 1220:  You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed.
 1221:  </system-reminder><|im_end|>
 1222:  <|im_start|>assistant
 1223:  <think>

Operating System

Linux and Windows

Terminal

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions