fix(distributed): stage backend companion assets to remote nodes - #10330
Merged
Conversation
A model whose ModelFile is a single file (e.g. sherpa-onnx VITS/piper: the .onnx) failed to load on remote worker nodes because the sibling assets the backend resolves from the model dir — tokens.txt, lexicon.txt, the espeak-ng-data / dict directories, Kokoro's voices.bin — were never staged. Only the declared ModelFile was shipped, so the worker hit "failed to create sherpa-onnx TTS engine" and TTS produced no audio. Lean on the existing option-path staging instead of hardcoding filenames: - stageGenericOptions now also resolves an option value relative to the model's own directory (not just the frontend models dir), so a shared config can declare companions with bare names regardless of whether Model includes a subdirectory; and it expands directory-valued options (e.g. espeak-ng-data) file-by-file rather than handing a directory fd to the stager. - gallery/sherpa-onnx-tts.yaml declares the companion assets as option paths (tokens, lexicon, espeak-ng-data, voices.bin, dict, per-lang lexicons). The backend ignores these keys and keeps resolving siblings from the model dir; they exist only so distributed staging ships them. Absent files are skipped. Adds router_optionstage_test.go covering file + directory companion staging via the model-dir fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In distributed inference, a model whose
ModelFileis a single file (e.g. sherpa-onnx VITS/piper, whereparameters.modelis the.onnx) fails to load on a remote worker. The backend resolves sibling assets from the model directory —tokens.txt,lexicon.txt, theespeak-ng-data/dictdirectories, Kokoro'svoices.bin— but staging only ships the declaredModelFile. The worker then hits:and TTS produces no audio. (
countStageableFilesreturns 1 for a regular file; whole-directory expansion only happens whenModelFileitself is a directory.)Fix
Lean on the existing option-path staging (
vae_path:-style) rather than hardcoding filenames in the router:stageGenericOptionsresolves an option value relative to the model's own directory as well as the frontend models dir, so a shared config can declare companions with bare names regardless of whetherModelincludes a subdirectory (newresolveOptionPathhelper).espeak-ng-data) file-by-file viastageOptionDir, instead of handing a directory fd to the stager.gallery/sherpa-onnx-tts.yamldeclares the companion assets as option paths once for all sherpa TTS voices (tokens,lexicon,espeak-ng-data,voices.bin,dict, per-language lexicons). The backend ignores these keys and keeps resolving siblings from the model dir; they exist only so distributed staging ships them. Files a given voice doesn't ship are skipped.Tests
core/services/nodes/router_optionstage_test.gocovers staging an option-declared sibling file and expanding a directory option via the model-dir fallback.core/services/nodessuite passes;go vetclean.🤖 Generated with Claude Code