fix(dash): give instances a real Ready status instead of a Running catch-all

[michaelroy-amd]

Summary

InstanceStatus had no Ready state, so merge_instance hardcoded every discovered instance to Running regardless of whether it had actually passed a readiness probe. Managed services with a "ready" record status, Lemonade endpoints (only ever built after a health probe), and Docker-discovered containers all collapsed into the same Running label, hiding real lifecycle differences from the TUI and any future readiness-aware consumer (e.g. chat endpoint selection).

Root Cause

rocm_dash_core::traits::merge_instance unconditionally set status: InstanceStatus::Running on every Instance it built, and InstanceStatus itself had no variant to represent "passed its readiness probe" as distinct from "process is up." Every discovery source (managed-service registry, Lemonade collector, Docker collector) therefore reported the same status label no matter what it actually knew about the endpoint's readiness.

Changes

• Add InstanceStatus::Ready and an is_serving() helper (Ready | Running) for call sites that only care about "is this actively serving."
• Add a status field to DiscoveredService so each discovery source reports what it actually knows; merge_instance now uses svc.status instead of a hardcoded Running.
• rocm-dash-daemon::registry::discovered_from_record maps a managed-service record's status string (ready/running/starting/recovering) to the matching InstanceStatus. "recovering" was previously dropped from the scrapeable set entirely (a bug relative to apps/rocm/src/main.rs's own "is this service live" checks, which treat recovering as live) — it's now scraped and mapped to Starting.
• rocm-dash-collectors::lemonade::lemonade_service builds as Ready since every call site only constructs it after a successful /api/v1/health probe.
• rocm-dash-collectors::docker has no readiness signal of its own, so Docker-discovered instances start Starting; rocm-dash-daemon::runner::run_loop promotes them to Ready on their first successful vLLM Prometheus scrape or Lemonade stats fetch (documented scope: this is the only readiness signal available for Docker-discovered instances).
• Add Ready arms to the exhaustive status_meta/status_role matches in crates/rocm-dash-tui/src/ui/tabs/instances.rs; switch the == InstanceStatus::Running equality checks in home.rs/dock.rs/launcher.rs to .is_serving().

Test Plan

• cargo build -p rocm-dash-core -p rocm-dash-collectors -p rocm-dash-daemon -p rocm-dash-tui
• cargo clippy -p rocm-dash-core -p rocm-dash-collectors -p rocm-dash-daemon -p rocm-dash-tui --all-targets --all-features -- -D warnings
• cargo test -p rocm-dash-core -p rocm-dash-collectors -p rocm-dash-daemon (all green, including 2 new tests: instance_status_ready_round_trips_as_lowercase_json, instance_status_is_serving_covers_ready_and_running_only, and discovered_from_record_maps_status_string_to_instance_status)
• cargo test -p rocm-dash-tui -- --test-threads=1 (543 + 16 + 5 tests green)

Relates to EAI-7350

[rominf]

LGTM. The new Ready variant is propagated exhaustively: both match status sites in instances.rs handle it (non-wildcard), and every == InstanceStatus::Running filter across dock/launcher/home was converted to is_serving(), so Ready is never misclassified in the running-count, activity feed, or launcher check. Serde is order-independent (rename_all = "lowercase", round-trip tested), #[default] stays on Unknown, and adding recovering to the scrapeable set matches the supervisor's canonical starting/recovering → Starting semantics (effectively a latent fix — recovering services were previously skipped). Unused-import hygiene handled.

Behavior note (intended): a Docker container whose vLLM never becomes scrapeable (crash loop / wrong port / non-vLLM image) now sits as Starting and is excluded from is_serving(), instead of showing Running on discovery. More correct, but a user-visible change to what counts as "running" for the Docker path — worth a line in the description.

Approving.

[volen-silo]

🔴 Automated review · pr-review-watcher · 83ac66e

Summary

• Change: Adds InstanceStatus::Ready + an is_serving() helper, threads a real status through DiscoveredService/merge_instance, and maps each discovery source (managed-service registry, Lemonade, Docker) to a status it actually knows instead of a hardcoded Running. 1 commit, 10 files (+130/-31).
• Overall assessment: Approve with one design question. No blocking correctness bugs found. Verified locally: cargo clippy --all-targets --all-features -D warnings clean; cargo test for core/collectors/daemon all green; cargo test -p rocm-dash-tui -- --test-threads=1 → 543 + 16 + 5 pass (matches the PR's stated test plan exactly).
• Key point for the author: the daemon's optimistic Starting → Ready promotion is looser than the CLI's own readiness bar — see the one design finding below.

Design review finding (worth an explicit author decision)

runner.rs:433-437 — promotion overrides recovering semantics (non-blocking, but call it out).
run_loop promotes any instance with if inst.status == Starting { inst.status = Ready } on its first successful vLLM Prometheus scrape (and, dead-code aside, on a Lemonade stats fetch). Because a managed-service record with status recovering (or starting) is mapped to InstanceStatus::Starting (registry.rs:87), a crash-recovering service that answers a single scrape gets flipped to Ready on the dashboard.

The CLI itself uses a stricter bar for the same concept: apps/rocm/src/main.rs::refresh_managed_service_runtime_liveness (L13076-83) only treats an endpoint as ready when status ∈ {ready, running} and an explicit managed_service_endpoint_model_ready HTTP probe succeeds — it deliberately does not promote starting/recovering to ready. So the dashboard can now show Ready for a service the CLI's own logic would still classify as starting.

Neither behavior is obviously wrong (a served Prometheus scrape is real evidence of serving), but it's a divergence from the app's own liveness model. Options: (a) accept the optimistic promotion and note it as intended; or (b) only promote instances that were Starting from a source with no authoritative status (i.e. Docker), leaving registry-derived recovering/starting states owned by the registry. A one-line guard or a distinct "Docker-sourced, unknown-readiness" marker would express intent.

Non-blocking suggestions

1. Drift risk between the two status lists (registry.rs). is_scrapeable_status (L84-88: ready|running|starting|recovering) and instance_status_for_record_status (L85-93) independently enumerate the same live set. They agree today, but nothing forces them to stay in sync — a future added status could pass is_scrapeable_status yet silently fall into the _ => Unknown arm. Consider deriving one from the other, or a shared const slice, so the "the _ arm is unreachable" invariant is structural rather than a comment.

2. Dead promotion branch for Lemonade (runner.rs:483-489). The Starting → Ready promotion in the Lemonade stats-fetch branch can never fire: lemonade_service builds instances as Ready from the start (lemonade.rs:136), and managed Lemonade records take a separate path excluded from this scrape branch. Not harmful, but the branch is currently unreachable — either drop it or add a comment that it's defensive for a future Docker-discovered Lemonade path.

3. Stale doc comment (traits.rs:72-76). The DiscoveredService.status doc says sources with no authoritative status "(Docker containers) leave this at the default" — but docker.rs:313 explicitly sets Starting, not the Unknown default. The comment describes a fallback Docker no longer uses; tighten it to match.

4. Test gaps (cheap, high-value).

   • No test asserts InstanceStatus::default() == InstanceStatus::Unknown. Given the PR inserts Ready as the first variant, a one-line assertion is the most direct guard against an accidental default change. (Verified safe manually — #[default] is pinned to Unknown at metrics.rs:75 and Rust's enum derive(Default) requires the explicit attribute, so declaration order doesn't matter — but the guard is worth having.)
   • instances.rs status_meta_maps_each_variant enumerates every variant except the new Ready; add status_meta(InstanceStatus::Ready, …) to match the test's own pattern. The READY label is currently only exercised transitively.

5. AGENTS.md: internal ticket id in the PR body. The PR description ends with "Relates to EAI-7350". This repo's AGENTS.md (§2) forbids internal ticket identifiers on upstream surfaces, explicitly including PR bodies. It is not in the code, commits, or diff (leak scan clean), so this is a description-only cleanup — remove or neutralize the reference.

Tradeoffs (deliberate choices, surfaced for confirmation)

• Docker instances now start Starting (warn/yellow) instead of Running. A Docker container that never produces a successful scrape (wrong port, slow/no vLLM) will now render STARTING indefinitely rather than the old misleading RUNNING. This is a net honesty improvement, but it is a visible behavior change — dashboards that previously showed green for never-actually-ready containers will now show a persistent warning state. Worth a line in the PR description so it isn't read as a regression.
• status_role folds Ready into Success (same green border as Running) while status_meta keeps a distinct READY/RUNNING label. Reasonable: same at-a-glance "safe to use" color, distinct text so the source lifecycle is still visible.
• is_serving() treats Running and Ready identically. Intentional transitional shim — the precise distinction lives at the display layer, not in the boolean gate. Consistent with all current call sites (TUI boolean gates only).

Positive signals

• The default-variant risk (inserting Ready first) was handled correctly: #[default] stays on Unknown, so no silent default change — the most plausible way this PR could have introduced a subtle bug, avoided.
• The recovering fix is a genuine bug fix: pre-PR is_scrapeable_status omitted recovering, so a supervisor-recovering service silently vanished from the dashboard; mapping recovering → Starting matches how apps/rocm/src/main.rs buckets it everywhere else (L11915, L11938, L13025, L13035).
• New tests are meaningful and well-scoped: the is_serving test enumerates all six variants explicitly (no matches!-mirroring blind spot), and the serde round-trip test correctly guards the lowercase rename for the new variant.
• Removing the post-merge inst.status = Running overrides in lemonade_instance/instance_from_discovered in favor of merge_instance propagating svc.status is a clean simplification — verified equivalent (no other field was touched by the removed lines).

---

Review-only. No approval/request-changes/merge; no changes pushed. Fan-out: 3 parallel file-scoped reviewers + orchestrator synthesis/verification; all build/test/clippy claims re-run locally at 83ac66e.