feat(core): make system prompt immutable after session creation#33241
feat(core): make system prompt immutable after session creation#332410byte-coding wants to merge 2 commits into
Conversation
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, I found several PRs related to system prompt handling and caching: Potentially Related PRs:
These PRs address related concerns about system prompt caching and session-level prompt stability, though none appear to be direct duplicates. PR #33241 (the current PR) takes a more comprehensive approach with in-memory HashMap-based caching per session, while the others focus on specific provider optimizations or structural changes for cache stability. |
Issue for this PR
Closes #29672
Type of change
What does this PR do?
Caches the system prompt in-memory per session ID so it becomes immutable after the session is created. This prevents unnecessary cache misses when the same session makes multiple requests.
The implementation stores system prompts in a HashMap keyed by session ID. This is very minimal and focused - just enough to lock the prompt after initialization.
How did you verify your code works?
I tested locally that:
What it does NOT do
<system-reminder>for things like date changesChecklist