Skip to content

Commit b298334

Browse files
committed
[NVIDIA#12560][fix] Fix disagg serving hang on block reuse after eviction
Claim block from eviction policy free queue before pinning in storeBlocks so that the later unpinBlocksById / releaseBlock cycle does not create a duplicate queue entry. Signed-off-by: Iman Tabrizian <itabrizian@nvidia.com>
1 parent 50379d0 commit b298334

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1618,6 +1618,14 @@ std::pair<SizeType32, std::optional<KVCacheBlock::IdType>> WindowBlockManager::s
16181618
}
16191619
if (pinBlocks)
16201620
{
1621+
// Claim block from free queue before pinning so that
1622+
// unpinBlocksById can safely releaseBlock it back later.
1623+
// Without this, the block stays in the free queue while pinned,
1624+
// and the subsequent releaseBlock creates a duplicate entry.
1625+
if (!searchRoot->hasRefs())
1626+
{
1627+
mEvictionPolicy->claimBlock(searchRoot, searchRoot->getPriority(), searchRoot->getDurationMs());
1628+
}
16211629
searchRoot->incRefCount();
16221630
}
16231631
lastStoredId = searchRoot->getBlockId();

0 commit comments

Comments
 (0)