[None][fix] Remove context KV cache allocation rollback#14849
Conversation
|
/bot run |
|
PR_Github #51489 [ run ] triggered by Bot. Commit: |
62658c4 to
a70edf3
Compare
|
/bot run |
|
PR_Github #51538 [ run ] triggered by Bot. Commit: |
|
PR_Github #51489 [ run ] completed with state |
|
PR_Github #51538 [ run ] completed with state
|
|
/bot run |
|
PR_Github #51563 [ run ] triggered by Bot. Commit: |
|
PR_Github #51563 [ run ] completed with state
|
|
/bot run |
|
PR_Github #51621 [ run ] triggered by Bot. Commit: |
|
PR_Github #51621 [ run ] completed with state
|
|
/bot run |
|
PR_Github #51740 [ run ] triggered by Bot. Commit: |
|
PR_Github #51740 [ run ] completed with state
|
a70edf3 to
d574913
Compare
|
/bot run |
|
PR_Github #51757 [ run ] triggered by Bot. Commit: |
|
PR_Github #51757 [ run ] completed with state
|
d574913 to
50839a5
Compare
|
PR_Github #51818 [ run ] triggered by Bot. Commit: |
|
PR_Github #51818 [ run ] completed with state
|
|
/bot run |
|
PR_Github #51865 [ run ] triggered by Bot. Commit: |
|
PR_Github #51865 [ run ] completed with state
|
|
/bot run |
|
PR_Github #51983 [ run ] triggered by Bot. Commit: |
|
/bot run |
ad0d588 to
50839a5
Compare
|
/bot run |
|
PR_Github #52007 [ run ] triggered by Bot. Commit: |
|
PR_Github #51983 [ run ] completed with state
|
|
PR_Github #52009 [ run ] triggered by Bot. Commit: |
|
PR_Github #52009 [ run ] completed with state
|
|
/bot run |
1 similar comment
|
/bot run |
|
PR_Github #52041 [ run ] triggered by Bot. Commit: |
|
PR_Github #52041 [ run ] completed with state
|
Signed-off-by: Jiagan Cheng <jiaganc@nvidia.com>
50839a5 to
f874c5d
Compare
|
/bot run |
|
PR_Github #52222 [ run ] triggered by Bot. Commit: |
|
PR_Github #52222 [ run ] completed with state |
Source-Commit: 95cd755 Signed-off-by: Jiagan Cheng <jiaganc@nvidia.com>
Source-Commit: 95cd755 Signed-off-by: Jiagan Cheng <jiaganc@nvidia.com>
4 participants
Description
This PR reverts #13805 for context allocation rollback. The rollback path can resize scratch-reuse-backed KV caches in a way that violates the history/capacity invariant.
Remove the V2 context KV cache allocation rollback path. Keep generation allocation rollback unchanged, since scratch reuse is disabled for generation requests.
Update the KV cache stats coverage so waited context allocations are reported when they are actually scheduled, and add the missing disaggregated-generation flag to the local request test double.
Test Coverage
CUDA_VISIBLE_DEVICES=0 python3 -m pytest -q tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.pyon GB200gb-nvl-082-compute09: 11 passed, 2 warningsgit diff --checkpython3 -m py_compile tensorrt_llm/_torch/pyexecutor/resource_manager.py tensorrt_llm/_torch/pyexecutor/py_executor.py tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.pyPR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why.
PR Follows TRT-LLM CODING GUIDELINES to the best of my knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.