Skip to content

[None] [feat] Python SLURM launcher #7255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kaiyux wants to merge 9 commits into from
Closed

[None] [feat] Python SLURM launcher #7255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8d08df4
SLURM Python launcher
kaiyux Aug 26, 2025
6029a48
Update
kaiyux Aug 26, 2025
ae86a52
Fix
kaiyux Aug 26, 2025
519d4bd
Update
kaiyux Aug 26, 2025
37e5a0b
Update
kaiyux Aug 26, 2025
9ca31bc
Update
kaiyux Aug 26, 2025
a9bbb5b
Update
kaiyux Aug 26, 2025
e2887f2
Update
kaiyux Aug 26, 2025
35031e4
Fix
kaiyux Aug 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions examples/disaggregated/slurm/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
exec:
config:
context:
tp: 4
ep: 4
pp: 1
max_batch_size: 4
max_num_tokens: 1024
max_seq_len: 1024
config:
kv_cache_config:
free_gpu_memory_fraction: 0.75
enable_block_reuse: false
print_iter_log: true
dp: 1
generation:
tp: 4
ep: 4
pp: 1
max_batch_size: 1
max_num_tokens: 4096
max_seq_len: 2048
config:
print_iter_log: true
kv_cache_config:
free_gpu_memory_fraction: 0.75
enable_block_reuse: false
dp: 1
model_path: TinyLlama/TinyLlama-1.1B-Chat-v1.0
profile:
dataset_path: /tmp/ShareGPT_V3_unfiltered_cleaned_split.json
isl: 1024
osl: 1024
num_prompts: 8192
use_benchmark_serving: true
concurrency:
- 128
- 256
ignore_eos: true
11 changes: 11 additions & 0 deletions examples/disaggregated/slurm/disagg_profiler/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
"""

@coderabbitai coderabbitai Bot Aug 26, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Missing NVIDIA copyright header.

According to the coding guidelines, all source files including Python files should include the NVIDIA copyright header with the current year (2025).

Add the NVIDIA copyright header at the top of the file:

+# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 """
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"""
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
🧰 Tools
🪛 Ruff (0.12.2)

1-4: 1 blank line required between summary line and description

(D205)

🤖 Prompt for AI Agents 
In examples/disaggregated/slurm/disagg_profiler/__init__.py around line 1, the
file is missing the required NVIDIA copyright header for 2025; add the standard
NVIDIA copyright header block at the very top of the file (before any code or
docstrings), using the current year 2025 and matching the project's canonical
header format so the file complies with coding guidelines.

Disaggregated serving profiler package.
This package contains the job management and parameter sweeping functionality
for the TRT-LLM disaggregated serving launcher.
"""
Comment on lines +1 to +5

@coderabbitai coderabbitai Bot Aug 26, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix docstring formatting according to PEP 257.

The docstring needs a blank line between the summary and description.

 """
 Disaggregated serving profiler package.
+
 This package contains the job management and parameter sweeping functionality
 for the TRT-LLM disaggregated serving launcher.
 """
🧰 Tools
🪛 Ruff (0.12.2)

1-4: 1 blank line required between summary line and description

(D205)

🤖 Prompt for AI Agents 
examples/disaggregated/slurm/disagg_profiler/__init__.py lines 1-5: The module
docstring does not follow PEP 257 because it lacks a blank line between the
one-line summary and the subsequent description; update the docstring to insert
a single blank line after the first summary line so the summary is separated
from the detailed description (keep the existing text otherwise).


from .job_manager import JobManager, wait_for_server

__all__ = [
'JobManager', 'wait_for_server',
]
Loading
Loading