Skip to content

chore(mesh): move to MeshLLM v0.75.0 and stop pre-seeding the native runtime - #5289

Draft
micspiral wants to merge 3 commits into
mainfrom
micspiral/mesh-llm-0.75
Draft

chore(mesh): move to MeshLLM v0.75.0 and stop pre-seeding the native runtime#5289
micspiral wants to merge 3 commits into
mainfrom
micspiral/mesh-llm-0.75

Conversation

@micspiral

@micspiral micspiral commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

What this does

  • Bumps the eight mesh-llm pins (six in desktop/src-tauri, two in buzz-relay) from v0.74.0 to v0.75.0, refreshing both lockfiles.
  • Deletes scripts/ensure-mesh-native-runtime.sh and all six of its call sites, plus a runbook reference to it.

Why the script is unnecessary

It built llama.cpp from source locally to pre-seed a native-runtime cache. Nothing on the app or CI path needs that:

  • The app installs the signed runtime itself. initialize_host_runtime()install_native_runtime(), whose allow_download defaults to true (mesh-llm-runtime-install/src/lib.rs:108). v0.75.0 publishes native-runtimes.json (13 artifacts, incl. darwin-aarch64-metal); manifest and artifact sha256 sidecars both verified by hand.
  • CI never called it. mesh-lifecycle.ymlscripts/ci-mesh-lifecycle-smoke.shmesh_relay_lifecycle_smoke.rs:156, which initialises the runtime itself. The workflow caches the app's own ~/.cache/mesh-llm/native-runtimes.
  • The three mesh-e2e-* recipes that also used it are hand-run only.

Blocker — do not merge

On a machine with a pre-0.75 runtime cache, startup fails:

native runtime artifact meshllm-native-runtime-darwin-aarch64-metal does not declare file checksums

Mechanism. v0.75.0 added NativeRuntimeManifest::verify_contents, which bail!s on empty runtime.files (mesh-llm-native-runtime/src/manifest.rs:104). v0.74.0 had no such field or check (e60b2fe/.../manifest.rs:16,58), so pre-checksum caches were tolerated. Startup calls cache.installed()? (host-runtime/src/system/native_runtime.rs:273), which enumerates every version under the cache root and propagates the first error via ? (cache.rs:99,249) — so one stale entry aborts the scan even when a valid 0.75.0 entry sits beside it.

This is an ordering bug, not just a compat break. Had that scan returned None instead of erroring, startup would have reached install_native_runtime (native_runtime.rs:221), which resolves only the requested version via NativeRuntimeResolverinstalled_for_version — whose own doc comment names this exact case (cache.rs:122, "TODO(issue #1162) ... pre-checksum runtime caches"). The version-scoped boundary exists; startup just doesn't use it.

Bisected on a real machine:

cache state bundle dir result
stale 0.73.1 + 0.74.0 + rc8, plus valid 0.75.0 none fails
only 0.75.0 none passes
stale entries present supplied passes

Why the installed CLI is unaffected: its installer copies a composed bundle to ~/.local/bin/native-runtimes (install.sh:621,676), found by executable_candidates (discovery.rs:180). Non-empty bundle discovery skips the cache enumeration entirely (native_runtime.rs:192). That bundle discovery is itself strict (InvalidManifestPolicy::Reject, discovery.rs:121) — it works because the bundle is valid, not because it is lenient.

Candidate fixes, least invasive last:

  1. Upstream 0.75.x routes startup cache resolution through the version-scoped resolver; pin that. Preserves offline start, mutates nothing.
  2. Use the documented embedder flow: mesh_llm_sdk::native_runtime::install_native_runtime(...) then start the node, skipping initialize_host_runtime() (docs/sdk/rust.md:123). Client-only mode needs no native runtime at all.
  3. Ship an adjacent bundle like the CLI. Deterministic and offline-capable; costs app size and per-platform packaging.
  4. Prune/quarantine stale cache entries — mutates a cache shared with other mesh-llm consumers, and is schema-dependent.

Explicit native_runtime config does not help: it changes the target version only and still hits the same full-cache scan (lib.rs:93).

Also worth knowing

  • mesh-lifecycle.yml's broad restore-keys (:66) can restore a v0.74 cache into a v0.75 run, so this PR's own lifecycle job may hit the blocker.
  • mesh_serve_client_smoke.rs:44 and mesh_admission_smoke.rs:57 refuse to install a missing runtime and use cache.installed(), so those hand-run recipes need a prepared runtime and are exposed to the same stale-cache failure.
  • mesh=1 is not strictly unchanged: runtime provisioning moves from pre-seeding to startup download.

Validation

  • cargo check -p buzz-desktop --features mesh-llm — clean
  • cargo check -p buzz-relay — clean
  • just --summary — parses
  • Empty-cache run downloaded the v0.75.0 runtime and reached model load

Bumps the eight mesh-llm pins (six in the desktop Tauri crate, two in
buzz-relay) from v0.74.0 to v0.75.0 and refreshes both lockfiles.

Also deletes scripts/ensure-mesh-native-runtime.sh and its six call sites.
The script built llama.cpp from source locally to pre-seed a native-runtime
cache, but nothing needs that:

- The app installs the signed runtime itself. `initialize_host_runtime()`
  reaches `install_native_runtime()`, whose `allow_download` defaults to
  true, and v0.75.0 publishes `native-runtimes.json` (13 artifacts,
  including darwin-aarch64-metal) alongside the release. Verified by
  fetching the manifest and artifact and checking both sha256 sidecars.
- CI never called it. The mesh lifecycle workflow caches the app's own
  `~/.cache/mesh-llm/native-runtimes` and says so in a comment: "the
  mesh-llm SDK downloads a signed native runtime on first init". The
  desktop build job compiles llama.cpp through mesh-llm's own
  prepare-llama.sh / build-llama.sh.
- The three `mesh-e2e-*` recipes that also used it are hand-run only,
  referenced nowhere else in the repo.

`mesh=1` behaviour is unchanged; only the pre-seed line is gone from
dev / staging / production.

KNOWN ISSUE (do not merge before resolving): on a machine that already has
a pre-0.75 runtime cache, startup fails with "native runtime artifact ...
does not declare file checksums". v0.75.0 added
`NativeRuntimeManifest::verify_contents`, which requires per-file
checksums; caches written by older loaders have none, and the startup path
enumerates the whole cache strictly, so one stale entry aborts the scan
even when a valid 0.75.0 entry is present. Reproduced and bisected here:
failing with the stale entries present, passing with only 0.75.0 cached,
and passing with the stale entries present once a bundle dir is supplied.
That is why the installed CLI is unaffected -- it ships a runtime directory
beside its binary, which is discovered leniently.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
The runbook told readers to run scripts/ensure-mesh-native-runtime.sh,
which this branch removes. The desktop installs the signed runtime itself
on first init, so the only manual step left is clearing stale pre-0.75
cache entries.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
@michaelneale

Copy link
Copy Markdown
Contributor

🤖 (from micn's agent) Confirmed your "known issue" independently, and there's now an upstream fix open: Mesh-LLM/mesh-llm#1196.

Findings that back up your bisect table:

  • Reproduced in isolation on macOS arm64: a mesh binary with no adjacent native-runtimes/ bundle dir plus a cache containing only a stale pre-checksum 0.74.0 entry → v0.75.0 startup fails hard with does not declare file checksums. Same box, same cache, installed CLI works.
  • Root cause is a second, unfixed path beyond upstream chore(release): release version 0.3.28 #1170: resolve_installed_native_runtime_plan in mesh-llm-host-runtime/src/system/native_runtime.rs calls the strict full-cache NativeRuntimeCache::installed(), where one stale manifest ?-aborts the whole scan — before the allow_download fallback can run. chore(release): release version 0.3.28 #1170 only fixed the resolver's enumeration (installed_for_version), not this startup fast-path.
  • Your "why the CLI is unaffected" theory is exactly right: install.sh ships native-runtimes/ beside the binary, so startup takes the bundle-discovery branch (lenient read_installed_runtime_lenient, warn + skip) and never reaches the strict cache scan. SDK embedders have no bundle dir → strict scan → abort. Same shared code, different packaging → different behavior. This was upstream issue chore(deps): pin dependencies #1162's CLI shape (runtime install --bundle-dir on Windows); the SDK init path is the same bug with no workaround.

The upstream fix (mesh-llm#1196) adds NativeRuntimeCache::installed_lenient() and switches startup resolution to it — stale entries warn + skip, strict checksum verification stays for install/download/load of the selected runtime. Verified against this exact repro: polluted cache, no bundle dir → warns and proceeds.

Suggestion for this PR: keep it draft until the mesh fix ships, then bump the pin to the tag containing it (v0.75.1 or the fix rev) rather than adding a buzz-side cache-clearing workaround. The rest of this PR (dropping ensure-mesh-native-runtime.sh, relying on the signed runtime download) checks out — CI never called the script, and the SDK-owned install path handles clean machines.

@micspiral micspiral self-assigned this Aug 8, 2026
…ttee

Shared-compute `auto` now maps straight onto MeshLLM's virtual `mesh` model
and sends it unconditionally. MeshLLM >= 0.75.0 resolves that per request:
a Mixture-of-Agents committee when two or more workers are reachable, and a
single served model when they are not
(`moa_gateway::degrade_to_single_model`). Before 0.75.0 a `model=mesh`
request could 503 on a lone node, so the agent kept its own hysteretic view
of whether a committee was currently possible and only then dared send
`mesh`.

That client-side machinery is now redundant, and removing it deletes the
polling and the debounce it needed to avoid flapping:

- `resolve_openai_model`, `observe_mesh_virtual_model`,
  `cool_down_collective`, `mesh_catalog_supports_collective`
- `MeshAutoState`, `MeshCatalogObservation`, and the `Llm` field holding it
- the `/models` probe: 5s catalog TTL, 2s probe timeout, 30s cooldown, and
  the two-observation confirmation count
- `PostError::MeshFallback` with `is_mesh_moa_unavailable_body` /
  `is_mesh_moa_failure_body` and the `detect_mesh_fallback` parameter
- `looks_like_unstructured_tool_call`, whose only call site was gated on the
  same adaptive-mesh flag

`looks_like_unstructured_tool_call` arrived with this feature (#2825) rather
than as a general guard, so it leaves with it. It retried once through `auto`
when a committee answered tool-call markup as prose; if that turns out to
matter in its own right it should come back provider-agnostic, not gated on
shared compute.

Behaviour that is deliberately unchanged: an explicit model is still sent
verbatim and never rewritten, and plain OpenAI `auto` (a real provider model
name) is still left alone. What does change is that a 503 from an explicit
`mesh` request now takes the ordinary transport retry under the same model
instead of failing fast to a second model -- there is no second model to
fall back to once MeshLLM resolves `mesh` itself.

Three test helpers (`moa_failure`, `model_catalog`,
`complete_model_with_tool`) went unused once the fourteen catalog/debounce
tests were removed. New tests pin the surviving contract: shared-compute
`auto` sends `mesh` and never requests `/v1/models`, plain `auto` is
untouched, and an explicit model keeps its name across retries.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants