Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,13 +133,9 @@ jobs:
- name: Acceptance harness self-tests
if: needs.changes.outputs.heavy == 'true'
run: |
python scripts/pytorch_therock_gpu_test.py --self-test
python scripts/llama_cpp_therock_gpu_test.py --self-test
python scripts/comfyui_therock_gpu_test.py --self-test
python scripts/local_assistant_therock_gpu_test.py --self-test
python scripts/vllm_therock_gpu_test.py --self-test
python scripts/sglang_therock_gpu_test.py --self-test
python scripts/atom_therock_gpu_test.py --self-test
python scripts/wsl_preflight.py --self-test

- name: Portable WSL build deps self-test
Expand Down Expand Up @@ -173,7 +169,7 @@ jobs:
shell: pwsh
run: |
cargo build --workspace --all-targets
cargo build --release -p rocm -p rocmd -p rocm-engine-pytorch -p rocm-engine-llama-cpp -p rocm-engine-lemonade -p rocm-engine-atom -p rocm-engine-vllm -p rocm-engine-sglang -p xtask
cargo build --release -p rocm -p rocmd -p rocm-engine-lemonade -p rocm-engine-vllm -p xtask

- name: Test
if: needs.changes.outputs.heavy == 'true'
Expand All @@ -189,13 +185,9 @@ jobs:
if: needs.changes.outputs.heavy == 'true'
shell: pwsh
run: |
python .\scripts\pytorch_therock_gpu_test.py --self-test
python .\scripts\llama_cpp_therock_gpu_test.py --self-test
python .\scripts\comfyui_therock_gpu_test.py --self-test
python .\scripts\local_assistant_therock_gpu_test.py --self-test
python .\scripts\vllm_therock_gpu_test.py --self-test
python .\scripts\sglang_therock_gpu_test.py --self-test
python .\scripts\atom_therock_gpu_test.py --self-test
python .\scripts\wsl_preflight.py --self-test

- name: Release readiness self-test
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,13 +128,13 @@ Current workspace members:

- apps: `apps/rocm`, `apps/rocmd`
- shared crates: `crates/rocm-core`, `crates/rocm-engine-protocol`
- engine crates: `engines/atom`, `engines/lemonade`, `engines/llama-cpp`, `engines/pytorch`, `engines/sglang`, `engines/vllm`
- engine crates: `engines/lemonade`, `engines/vllm`

Guardrails:

- `crates/rocm-engine-protocol` is a contract surface; verify all impacted engines after protocol changes
- preserve strict GPU-required behavior; do not introduce silent CPU fallback
- respect platform gates (for example, native Windows handling for vLLM/SGLang)
- respect platform gates (for example, native Windows handling for vLLM)
- pin third-party GitHub Actions to a full commit SHA with a trailing `# vX.Y.Z` comment, never a moving tag (`@v2`, `@main`); a retagged or compromised action otherwise enters CI silently. Bump the SHA and comment together when upgrading
- supported host platforms are Windows and Linux only (including WSL where documented)
- platforms outside Windows/Linux are unsupported; do not implement, debug, or "fix" unsupported-platform behavior
Expand Down
55 changes: 0 additions & 55 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,7 @@ members = [
"crates/rocm-dash-collectors",
"crates/rocm-dash-daemon",
"crates/rocm-dash-tui",
"engines/llama-cpp",
"engines/atom",
"engines/pytorch",
"engines/lemonade",
"engines/sglang",
"engines/vllm",
"xtask",
]
Expand Down
15 changes: 7 additions & 8 deletions MANIFEST.md
Original file line number Diff line number Diff line change
Expand Up @@ -577,7 +577,7 @@ from the workspace source. The user-facing binaries are:
processes and expose a local HTTP API

In addition, each engine crate under `engines/` builds its own
`rocm-engine-<name>` host binary (for example `rocm-engine-pytorch`,
`rocm-engine-<name>` host binary (for example `rocm-engine-lemonade`,
`rocm-engine-vllm`), spawned by `rocmd` to run a specific inference engine.

All binaries are compiled from the workspace source using the standard Cargo
Expand Down Expand Up @@ -629,13 +629,12 @@ distribution; no Python packages are installed for Lemonade by rocm-cli.

### Engine-Specific Python Dependencies

The PyTorch engine manages its own Python virtual environment using the `uv`
binary described above. Python packages are installed from the TheRock PyPI
index and, where applicable, from public PyPI (`https://pypi.org`). No Python
packages are bundled in the repository.

The vLLM, SGLang, ATOM, and llama.cpp engines do not install Python packages
automatically; they record externally-provided runtimes supplied by the user.
The vLLM engine manages Python packages through the `uv` binary described
above, installed from the TheRock PyPI index and, where applicable, from public
PyPI (`https://pypi.org`). No Python packages are bundled in the repository. The
vLLM engine can also record an externally-provided runtime supplied by the user
instead of installing one. Lemonade ships as a self-contained embeddable
distribution and installs no Python packages through rocm-cli.

## Repository Structure Notes

Expand Down
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ ROCm CLI is a command-line tool for setting up and running local AI on AMD GPUs,
full-screen TUI dashboard for GPU telemetry, model serving, and chat.

A single prebuilt binary for Linux and Windows. No Python, Rust, or existing
ROCm install required. Ships with inference engine adapters for PyTorch,
llama.cpp, Lemonade, ATOM, vLLM, and SGLang.
ROCm install required. Ships with inference engine adapters for Lemonade and
vLLM.

> [!IMPORTANT]
> **Tech Preview** -- This software is provided as-is, without warranty or
Expand Down Expand Up @@ -166,14 +166,14 @@ rocm engines install <engine> [--runtime-id KEY] [--python-version X.Y] [--reins
rocm engines shell <engine> [--runtime-id KEY | --env-id ID] [--shell PATH]
```

Supported engines: `lemonade`, `pytorch`, `llama.cpp`, `atom`, `vllm`, `sglang`.
Supported engines: `lemonade`, `vllm`.

### Model serving

Start a local OpenAI-compatible model server:

```
rocm serve <model> [--engine lemonade|pytorch|llama.cpp|atom|vllm|sglang]
rocm serve <model> [--engine lemonade|vllm]
[--device gpu_required|gpu_preferred|cpu_only]
[--gpu auto|<index>]
[--runtime-id KEY | --env-id ID]
Expand Down Expand Up @@ -369,6 +369,4 @@ See `docs/commit-signatures.md` for details (GPG signing, GitHub "Verified", and
- Testing and verification: `docs/testing.md`
- Developer manual QA: `docs/manual-testing.md`
- Engine plugin policy: `docs/engine-plugins.md`
- ATOM adapter: `docs/atom.md`
- vLLM adapter: `docs/vllm.md`
- SGLang adapter: `docs/sglang.md`
4 changes: 0 additions & 4 deletions apps/rocm/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,8 @@ rocm-core = { path = "../../crates/rocm-core" }
rocm-dash-daemon = { path = "../../crates/rocm-dash-daemon" }
rocm-dash-tui = { path = "../../crates/rocm-dash-tui" }
tokio = { workspace = true }
rocm-engine-atom = { path = "../../engines/atom" }
rocm-engine-lemonade = { path = "../../engines/lemonade" }
rocm-engine-llama-cpp = { path = "../../engines/llama-cpp" }
rocm-engine-pytorch = { path = "../../engines/pytorch" }
rocm-engine-protocol = { path = "../../crates/rocm-engine-protocol" }
rocm-engine-sglang = { path = "../../engines/sglang" }
rocmd = { path = "../rocmd" }
rocm-engine-vllm = { path = "../../engines/vllm" }
rpassword.workspace = true
Expand Down
Loading
Loading