Skip to content

feat: NIXL interface integration - #3934

Merged
Shixiaowei02 merged 1 commit into
NVIDIA:mainfrom
Shixiaowei02:user/xiaoweis/nixl-interface
May 19, 2025
Merged

feat: NIXL interface integration#3934
Shixiaowei02 merged 1 commit into
NVIDIA:mainfrom
Shixiaowei02:user/xiaoweis/nixl-interface

Conversation

@Shixiaowei02

@Shixiaowei02 Shixiaowei02 commented Apr 29, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

This PR, in conjunction with PR 3769 , provides an interface solution for dynamically linking NIXL.

image

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from bde538b to b8ce590 Compare April 29, 2025 06:25
@Shixiaowei02
Shixiaowei02 requested a review from a team as a code owner April 29, 2025 15:18
@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch 2 times, most recently from 0671f8f to 1eeff6f Compare April 29, 2025 15:20
@Shixiaowei02

Shixiaowei02 commented Apr 29, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Apr 29, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #3740 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Apr 30, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #3740 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #2648 completed with status: 'FAILURE'

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from 1eeff6f to 9e8e220 Compare April 30, 2025 01:38
@Shixiaowei02

Shixiaowei02 commented Apr 30, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented Apr 30, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #3785 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented Apr 30, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #3785 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #2678 completed with status: 'SUCCESS'

@kaiyux
kaiyux removed the request for review from a team May 1, 2025 15:25
zackyoray
zackyoray reviewed May 4, 2025
View reviewed changes
Comment thread cpp/tensorrt_llm/executor/cache_transmission/nixl_utils/CMakeLists.txt Outdated
@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch 3 times, most recently from 48a7640 to 732cf72 Compare May 12, 2025 13:01
@Shixiaowei02

Shixiaowei02 commented May 12, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

tensorrt-cicd commented May 12, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #4881 [ run ] triggered by Bot

MartinMarciniszyn
MartinMarciniszyn requested changes May 12, 2025
View reviewed changes

@MartinMarciniszyn MartinMarciniszyn left a comment •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Shixiaowei02 , you will need to move the installation of NIXL back into the devel stage in docker. Otherwise, this MR will break the build of the TRT-LLM release image since you introduce a compile time dependency if I understand this correctly.

@tensorrt-cicd

tensorrt-cicd commented May 12, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #4881 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #3535 completed with status: 'FAILURE'

zackyoray
zackyoray reviewed May 13, 2025
View reviewed changes
Comment thread cpp/tensorrt_llm/executor/cache_transmission/nixl_utils/transferAgent.cpp Outdated
zackyoray
zackyoray reviewed May 13, 2025
View reviewed changes
Comment thread cpp/tensorrt_llm/executor/cache_transmission/ucx_utils/connection.h Outdated
@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch 2 times, most recently from ea0692a to b43aa2a Compare May 13, 2025 08:10
@Shixiaowei02

Shixiaowei02 commented May 13, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

@Shixiaowei02 , you will need to move the installation of NIXL back into the devel stage in docker. Otherwise, this MR will break the build of the TRT-LLM release image since you introduce a compile time dependency if I understand this correctly.

@MartinMarciniszyn Okay, I have modified the installation in the image, which will increase the overall image size by approximately 150 MB. I will make further modifications to install precompiled libraries directly instead of building from source to shrink the image size. Currently, NIXL can be disabled during the build, but it will be required for testing in the pre-merge/post-merge pipeline.

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from b43aa2a to f82d6ad Compare May 13, 2025 08:16
@tensorrt-cicd

tensorrt-cicd commented May 16, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5502 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 16, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5495 [ run ] completed with state ABORTED

@tensorrt-cicd

tensorrt-cicd commented May 16, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5502 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4009 completed with status: 'FAILURE'

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch 2 times, most recently from 4126433 to 4e51d56 Compare May 18, 2025 02:45
@Shixiaowei02

Shixiaowei02 commented May 18, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

tensorrt-cicd commented May 18, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5584 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 18, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5584 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4072 completed with status: 'FAILURE'

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from 4e51d56 to c62472f Compare May 19, 2025 00:33
@Shixiaowei02

Shixiaowei02 commented May 19, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5639 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5639 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4121 completed with status: 'FAILURE'

@Shixiaowei02

Shixiaowei02 commented May 19, 2025

Copy link
Copy Markdown
Collaborator Author

/bot run --stage-list "A100X-TensorRT-2"

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5686 [ run ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5686 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4154 (Partly Tested) completed with status: 'SUCCESS'

@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from c62472f to 819375e Compare May 19, 2025 09:42
@Shixiaowei02

Shixiaowei02 commented May 19, 2025

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "PreMerge pipelines 4121 and 4154 have passed"

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5709 [ skip ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5710 [ skip ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5709 [ skip ] completed with state ABORTED

@Shixiaowei02
NIXL interfaces
4491b4a 
Signed-off-by: ShiXiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
@Shixiaowei02
Shixiaowei02 force-pushed the user/xiaoweis/nixl-interface branch from 819375e to 4491b4a Compare May 19, 2025 10:02
@Shixiaowei02

Shixiaowei02 commented May 19, 2025

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "PreMerge pipelines 4121 and 4154 have passed"

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5710 [ skip ] completed with state SUCCESS
Skipping testing for commit 819375e

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5715 [ skip ] triggered by Bot

@tensorrt-cicd

tensorrt-cicd commented May 19, 2025

Copy link
Copy Markdown
Collaborator

PR_Github #5715 [ skip ] completed with state SUCCESS
Skipping testing for commit 4491b4a

@Shixiaowei02
Shixiaowei02 merged commit df2798e into NVIDIA:main May 19, 2025
@Shixiaowei02
Shixiaowei02 deleted the user/xiaoweis/nixl-interface branch May 19, 2025 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chuangz0 chuangz0 chuangz0 left review comments
@MartinMarciniszyn MartinMarciniszyn MartinMarciniszyn approved these changes
@pcastonguay pcastonguay Awaiting requested review from pcastonguay
+1 more reviewer
@zackyoray zackyoray zackyoray left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@Shixiaowei02 Shixiaowei02

@zackyoray zackyoray

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Shixiaowei02 @tensorrt-cicd @MartinMarciniszyn @zackyoray @chuangz0