Skip to content

support cache_max_entry_count >= 1 for Turbomind backend#3913

Merged
lvhan028 merged 1 commit intoInternLM:mainfrom
lh9171338:lh9171338-patch-1
Aug 29, 2025
Merged

support cache_max_entry_count >= 1 for Turbomind backend#3913
lvhan028 merged 1 commit intoInternLM:mainfrom
lh9171338:lh9171338-patch-1

Conversation

@lh9171338
Copy link
Copy Markdown
Contributor

Motivation

The Turbomind backend already supports cache_max_entry_count >= 1. However, the assertion check was not updated accordingly and still enforces 0 < cache_max_entry_count < 1. This inconsistency prevents valid configurations (e.g., cache_max_entry_count = 50) from being used, even though the backend supports them.

Modification

Relaxed the assertion condition for cache_max_entry_count:

- assert 0 < self.cache_max_entry_count < 1, "invalid cache_max_entry_count"
+ assert self.cache_max_entry_count > 0, "invalid cache_max_entry_count"

@lvhan028 lvhan028 merged commit 63553a8 into InternLM:main Aug 29, 2025
4 of 5 checks passed
littlegy pushed a commit to littlegy/lmdeploy that referenced this pull request Sep 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants