chore(mesh): move to MeshLLM v0.75.0 and stop pre-seeding the native runtime - #5289
chore(mesh): move to MeshLLM v0.75.0 and stop pre-seeding the native runtime#5289micspiral wants to merge 3 commits into
Conversation
Bumps the eight mesh-llm pins (six in the desktop Tauri crate, two in buzz-relay) from v0.74.0 to v0.75.0 and refreshes both lockfiles. Also deletes scripts/ensure-mesh-native-runtime.sh and its six call sites. The script built llama.cpp from source locally to pre-seed a native-runtime cache, but nothing needs that: - The app installs the signed runtime itself. `initialize_host_runtime()` reaches `install_native_runtime()`, whose `allow_download` defaults to true, and v0.75.0 publishes `native-runtimes.json` (13 artifacts, including darwin-aarch64-metal) alongside the release. Verified by fetching the manifest and artifact and checking both sha256 sidecars. - CI never called it. The mesh lifecycle workflow caches the app's own `~/.cache/mesh-llm/native-runtimes` and says so in a comment: "the mesh-llm SDK downloads a signed native runtime on first init". The desktop build job compiles llama.cpp through mesh-llm's own prepare-llama.sh / build-llama.sh. - The three `mesh-e2e-*` recipes that also used it are hand-run only, referenced nowhere else in the repo. `mesh=1` behaviour is unchanged; only the pre-seed line is gone from dev / staging / production. KNOWN ISSUE (do not merge before resolving): on a machine that already has a pre-0.75 runtime cache, startup fails with "native runtime artifact ... does not declare file checksums". v0.75.0 added `NativeRuntimeManifest::verify_contents`, which requires per-file checksums; caches written by older loaders have none, and the startup path enumerates the whole cache strictly, so one stale entry aborts the scan even when a valid 0.75.0 entry is present. Reproduced and bisected here: failing with the stale entries present, passing with only 0.75.0 cached, and passing with the stale entries present once a bundle dir is supplied. That is why the installed CLI is unaffected -- it ships a runtime directory beside its binary, which is discovered leniently. Signed-off-by: Michael Neale <michael.neale@gmail.com>
The runbook told readers to run scripts/ensure-mesh-native-runtime.sh, which this branch removes. The desktop installs the signed runtime itself on first init, so the only manual step left is clearing stale pre-0.75 cache entries. Signed-off-by: Michael Neale <michael.neale@gmail.com>
|
🤖 (from micn's agent) Confirmed your "known issue" independently, and there's now an upstream fix open: Mesh-LLM/mesh-llm#1196. Findings that back up your bisect table:
The upstream fix (mesh-llm#1196) adds Suggestion for this PR: keep it draft until the mesh fix ships, then bump the pin to the tag containing it (v0.75.1 or the fix rev) rather than adding a buzz-side cache-clearing workaround. The rest of this PR (dropping |
…ttee Shared-compute `auto` now maps straight onto MeshLLM's virtual `mesh` model and sends it unconditionally. MeshLLM >= 0.75.0 resolves that per request: a Mixture-of-Agents committee when two or more workers are reachable, and a single served model when they are not (`moa_gateway::degrade_to_single_model`). Before 0.75.0 a `model=mesh` request could 503 on a lone node, so the agent kept its own hysteretic view of whether a committee was currently possible and only then dared send `mesh`. That client-side machinery is now redundant, and removing it deletes the polling and the debounce it needed to avoid flapping: - `resolve_openai_model`, `observe_mesh_virtual_model`, `cool_down_collective`, `mesh_catalog_supports_collective` - `MeshAutoState`, `MeshCatalogObservation`, and the `Llm` field holding it - the `/models` probe: 5s catalog TTL, 2s probe timeout, 30s cooldown, and the two-observation confirmation count - `PostError::MeshFallback` with `is_mesh_moa_unavailable_body` / `is_mesh_moa_failure_body` and the `detect_mesh_fallback` parameter - `looks_like_unstructured_tool_call`, whose only call site was gated on the same adaptive-mesh flag `looks_like_unstructured_tool_call` arrived with this feature (#2825) rather than as a general guard, so it leaves with it. It retried once through `auto` when a committee answered tool-call markup as prose; if that turns out to matter in its own right it should come back provider-agnostic, not gated on shared compute. Behaviour that is deliberately unchanged: an explicit model is still sent verbatim and never rewritten, and plain OpenAI `auto` (a real provider model name) is still left alone. What does change is that a 503 from an explicit `mesh` request now takes the ordinary transport retry under the same model instead of failing fast to a second model -- there is no second model to fall back to once MeshLLM resolves `mesh` itself. Three test helpers (`moa_failure`, `model_catalog`, `complete_model_with_tool`) went unused once the fourteen catalog/debounce tests were removed. New tests pin the surviving contract: shared-compute `auto` sends `mesh` and never requests `/v1/models`, plain `auto` is untouched, and an explicit model keeps its name across retries. Signed-off-by: Michael Neale <michael.neale@gmail.com>
What this does
mesh-llmpins (six indesktop/src-tauri, two inbuzz-relay) fromv0.74.0tov0.75.0, refreshing both lockfiles.scripts/ensure-mesh-native-runtime.shand all six of its call sites, plus a runbook reference to it.Why the script is unnecessary
It built llama.cpp from source locally to pre-seed a native-runtime cache. Nothing on the app or CI path needs that:
initialize_host_runtime()→install_native_runtime(), whoseallow_downloaddefaults totrue(mesh-llm-runtime-install/src/lib.rs:108). v0.75.0 publishesnative-runtimes.json(13 artifacts, incl.darwin-aarch64-metal); manifest and artifactsha256sidecars both verified by hand.mesh-lifecycle.yml→scripts/ci-mesh-lifecycle-smoke.sh→mesh_relay_lifecycle_smoke.rs:156, which initialises the runtime itself. The workflow caches the app's own~/.cache/mesh-llm/native-runtimes.mesh-e2e-*recipes that also used it are hand-run only.Blocker — do not merge
On a machine with a pre-0.75 runtime cache, startup fails:
Mechanism. v0.75.0 added
NativeRuntimeManifest::verify_contents, whichbail!s on emptyruntime.files(mesh-llm-native-runtime/src/manifest.rs:104). v0.74.0 had no such field or check (e60b2fe/.../manifest.rs:16,58), so pre-checksum caches were tolerated. Startup callscache.installed()?(host-runtime/src/system/native_runtime.rs:273), which enumerates every version under the cache root and propagates the first error via?(cache.rs:99,249) — so one stale entry aborts the scan even when a valid0.75.0entry sits beside it.This is an ordering bug, not just a compat break. Had that scan returned
Noneinstead of erroring, startup would have reachedinstall_native_runtime(native_runtime.rs:221), which resolves only the requested version viaNativeRuntimeResolver→installed_for_version— whose own doc comment names this exact case (cache.rs:122, "TODO(issue #1162) ... pre-checksum runtime caches"). The version-scoped boundary exists; startup just doesn't use it.Bisected on a real machine:
0.73.1+0.74.0+rc8, plus valid0.75.00.75.0Why the installed CLI is unaffected: its installer copies a composed bundle to
~/.local/bin/native-runtimes(install.sh:621,676), found byexecutable_candidates(discovery.rs:180). Non-empty bundle discovery skips the cache enumeration entirely (native_runtime.rs:192). That bundle discovery is itself strict (InvalidManifestPolicy::Reject,discovery.rs:121) — it works because the bundle is valid, not because it is lenient.Candidate fixes, least invasive last:
mesh_llm_sdk::native_runtime::install_native_runtime(...)then start the node, skippinginitialize_host_runtime()(docs/sdk/rust.md:123). Client-only mode needs no native runtime at all.Explicit
native_runtimeconfig does not help: it changes the target version only and still hits the same full-cache scan (lib.rs:93).Also worth knowing
mesh-lifecycle.yml's broadrestore-keys(:66) can restore a v0.74 cache into a v0.75 run, so this PR's own lifecycle job may hit the blocker.mesh_serve_client_smoke.rs:44andmesh_admission_smoke.rs:57refuse to install a missing runtime and usecache.installed(), so those hand-run recipes need a prepared runtime and are exposed to the same stale-cache failure.mesh=1is not strictly unchanged: runtime provisioning moves from pre-seeding to startup download.Validation
cargo check -p buzz-desktop --features mesh-llm— cleancargo check -p buzz-relay— cleanjust --summary— parses