Skip to content

[https://nvbugs/6211193][fix] etcd listen all interfaces - #14863

Merged
reasonsolo merged 1 commit into
NVIDIA:mainfrom
reasonsolo:fix/etcd-listen-all-interfaces
Jun 3, 2026
Merged

[https://nvbugs/6211193][fix] etcd listen all interfaces#14863
reasonsolo merged 1 commit into
NVIDIA:mainfrom
reasonsolo:fix/etcd-listen-all-interfaces

Conversation

@reasonsolo

@reasonsolo reasonsolo commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

The etcd server in the multi-node service discovery test was only listening on 127.0.0.1, causing clients on other nodes to fail to connect. Fix by passing --listen-client-urls http://0.0.0.0:2379.

Also unwaive the test case.

Summary by CodeRabbit

  • Tests
    • Improved multi-node service discovery test configuration with explicit network binding settings for the service discovery backend.
    • Removed skip waiver for etcd-based service discovery tests, enabling expanded test coverage.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@reasonsolo

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c6527dbf-c9d9-43ef-a01e-23837c9fa821

📥 Commits

Reviewing files that changed from the base of the PR and between 33e0ee3 and 4a3d600.

📒 Files selected for processing (2)
  • tests/integration/test_lists/waives.txt
  • tests/unittest/llmapi/apps/_test_disagg_serving_multi_nodes_service_discovery.py
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

This PR fixes multi-node etcd service discovery by updating the test fixture to advertise client URLs for cross-node connectivity, and removes the corresponding test waiver so the test runs in CI with the corrected infrastructure.

Changes

Multi-node etcd service discovery

Layer / File(s) Summary
etcd fixture client URL advertisement
tests/unittest/llmapi/apps/_test_disagg_serving_multi_nodes_service_discovery.py
The service_discovery fixture's etcd startup now includes explicit --listen-client-urls and --advertise-client-urls parameters to bind to http://0.0.0.0:2379 and advertise http://{disagg_host}:2379 for cross-node reachability.
Remove test waiver for etcd service discovery
tests/integration/test_lists/waives.txt
The SKIP waiver for test_openai_disagg_multi_nodes_completion_service_discovery[etcd] is removed, allowing the test to run in CI with the fixed etcd configuration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#14783: Both modify tests/integration/test_lists/waives.txt to adjust test-case waiver entries.
  • NVIDIA/TensorRT-LLM#14787: Directly conflicts with this PR; it adds the exact SKIP waiver entry that this PR removes for test_openai_disagg_multi_nodes_completion_service_discovery[etcd].
  • NVIDIA/TensorRT-LLM#14854: Both modify tests/integration/test_lists/waives.txt to adjust SKIP/waiver entries for integration tests related to service discovery.

Suggested reviewers

  • jieli-matrix
  • StanleySun639
  • LarryXFly
  • xinhe-nv
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the issue and solution in the body text, but the Description and Test Coverage sections are missing and not filled in per the template requirements. Fill in the Description and Test Coverage sections in the PR description to explain the changes and relevant test coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: configuring etcd to listen on all interfaces. It follows the required format with NVBugs ID and [fix] type.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51593 [ run ] triggered by Bot. Commit: 4a3d600 Link to invocation

The etcd server in the multi-node service discovery test was only
listening on 127.0.0.1, causing clients on other nodes to fail to
connect. Fix by passing --listen-client-urls http://0.0.0.0:2379.

Also unwaive the test case.

Signed-off-by: Lizhi Zhou <lizhiz@nvidia.com>
Signed-off-by: Lizhi Zhou <1432185+reasonsolo@users.noreply.github.com>
@reasonsolo
reasonsolo force-pushed the fix/etcd-listen-all-interfaces branch from 4a3d600 to 3e1a3d1 Compare June 2, 2026 10:58
@reasonsolo

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51593 [ run ] completed with state SUCCESS. Commit: 4a3d600
/LLM/main/L0_MergeRequest_PR pipeline #40979 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51601 [ run ] triggered by Bot. Commit: 3e1a3d1 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51601 [ run ] completed with state SUCCESS. Commit: 3e1a3d1
/LLM/main/L0_MergeRequest_PR pipeline #40987 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@reasonsolo

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51741 [ run ] triggered by Bot. Commit: 3e1a3d1 Link to invocation

@reasonsolo

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51790 [ run ] triggered by Bot. Commit: 3e1a3d1 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51741 [ run ] completed with state ABORTED. Commit: 3e1a3d1

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #51790 [ run ] completed with state SUCCESS. Commit: 3e1a3d1
/LLM/main/L0_MergeRequest_PR pipeline #41154 completed with status: 'SUCCESS'

CI Report

Link to invocation

@reasonsolo
reasonsolo enabled auto-merge (squash) June 3, 2026 10:36
@reasonsolo
reasonsolo merged commit 3630e16 into NVIDIA:main Jun 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants