Skip to content

feat(telegram): voice-out via ElevenLabs TTS + shared TTS foundation (#25)#1397

Merged
vybe merged 3 commits into
devfrom
feature/25-telegram-voice-out
Jul 2, 2026
Merged

feat(telegram): voice-out via ElevenLabs TTS + shared TTS foundation (#25)#1397
vybe merged 3 commits into
devfrom
feature/25-telegram-voice-out

Conversation

@dolho

@dolho dolho commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

First slice of epic #24 (outbound voice across channels): Telegram voice-out. Builds the shared TTS foundation that Slack (#26) and WhatsApp (trinity-enterprise#56) reuse unchanged, then wires Telegram as the first consumer.

Shared foundation (epic #24)

  • services/tts_service.py — one provider-agnostic TTS layer. ElevenLabs synth → ffmpeg MP3→OGG/Opus transcode (the chat voice-note codec) → OGG bytes. Shared char cost-cap (TTS_MAX_CHARS, default 1500). Fail-soft everywhere — every path returns None on any problem (no key, over cap, provider/transcode error, ffmpeg missing) so the caller delivers text. Voice is strictly additive.
  • Shared agent-level config (no per-channel column sprawl, per Consistent context usage tracking for stateless executions #56): agent_ownership.tts_voice_replies_enabled + tts_voice_id via a new TtsMixin. Dual-track migration (SQLite db/migrations.py + Alembic 0011), schema.py + tables.py.
  • GET/PUT /api/agents/{name}/voice-replies (owner-only; enabling requires a voice_id; GET exposes available = platform key present).
  • ffmpeg added to the backend image; ELEVENLABS_API_KEY / ELEVENLABS_MODEL_ID / TTS_MAX_CHARS in config + .env.example.

Telegram wiring (#25)

  • telegram_adapter._send_voice (Telegram sendVoice, multipart OGG) + a voice branch in send_response: when enabled with a voice id, speak the reply; fall back to text on any miss.
  • UI: voice-replies toggle + ElevenLabs voice-id field in the Sharing → Telegram panel.

Design decisions (confirmed)

  • Config home: shared agent-level primitive (not per-binding) — one config all channels reuse.
  • Trigger policy: Always-when-enabled. Mirror is infeasible — Telegram voice-in isn't implemented, so there's no inbound-voice signal to mirror.
  • Format: ffmpeg → OGG/Opus (native voice-note bubble; also what Consistent context usage tracking for stateless executions #56 needs).
  • Voice selection: paste ElevenLabs voice id (validated on save).
  • OSS core, ungated — channels live in OSS; the issue carries no entitlement.

Test plan

  • tts_service: cost-cap / no-key / no-voice / empty / provider-error / success; transcode success + ffmpeg-missing + nonzero-rc; end-to-end + short-circuit — 16 tests pass.
  • Adapter voice branch: disabled → no send; success → sendVoice; TTS None → text fallback.
  • Schema/migration parity green (test_schema_parity, test_migrations, alembic guard); 48 telegram tests pass; app wiring imports clean under conftest.
  • Live end-to-end against a real ElevenLabs key + Telegram bot (needs a key; not exercised locally).

Notes

  • #56 (WhatsApp voice-out) was un-claimed back to status-ready — it's blocked on this shared layer and will ride it once merged.
  • #25 was status-incubating with open design questions; promoted to status-in-progress after the decisions above.

Related to abilityai/trinity-enterprise#25

Establishes epic #24's shared outbound-voice foundation and wires Telegram as
the first consumer:

- services/tts_service.py — shared ElevenLabs client + ffmpeg MP3→OGG/Opus
  transcode + shared char cost-cap; fail-soft everywhere (any error → text).
- Shared agent-level config (no per-channel sprawl): agent_ownership
  .tts_voice_replies_enabled + .tts_voice_id, via TtsMixin. Dual-track migration
  (SQLite + Alembic 0011). GET/PUT /api/agents/{name}/voice-replies (owner-only).
- telegram_adapter: _send_voice (sendVoice) + a voice branch in send_response
  that speaks the reply when enabled, falling back to text on any miss.
- ffmpeg added to the backend image; ELEVENLABS_API_KEY/ELEVENLABS_MODEL_ID/
  TTS_MAX_CHARS in config + .env.example.
- UI: voice-replies toggle + ElevenLabs voice-id field in the Telegram panel.
- Tests: tts_service (cost-cap/no-key/provider-error/success, transcode +
  ffmpeg-missing, end-to-end) and the adapter voice branch (16 tests).

Trigger policy is Always-when-enabled (Mirror is infeasible — Telegram voice-in
isn't implemented). Slack (#26) and WhatsApp (trinity-enterprise#56) reuse this
shared layer unchanged.

Related to Abilityai/trinity-enterprise#25

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

⚠️ Nightly unit-suite check skipped — merge conflict against dev.

Resolve by running git merge dev locally and pushing the result. The next nightly run will re-test once the conflict is gone.

…ests (#762)

The `lint (sys.modules pollution check)` CI job failed on
tests/unit/test_25_telegram_voice_out.py: two bare `sys.modules[...]`
assignments (the import-time `utils.helpers` preload and the `database` stub in
the `adapter` fixture) are new violations vs the #762 baseline.

Adopt the sanctioned snapshot/restore pattern (precedent:
test_telegram_webhook_backfill.py) — a top-level `_STUBBED_MODULE_NAMES` list +
an autouse `_restore_sys_modules` fixture — which both satisfies the lint and
adds the real per-test restore hygiene #762 is about, so the stubs can't leak
across files. No test behavior change (16 tests still pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dolho
dolho requested a review from vybe July 2, 2026 14:34
The three new config vars were read by src/backend/config.py but wired into
neither docker-compose.yml nor docker-compose.prod.yml backend.environment:,
so the .env lever was inert on deploy — voice replies could never be enabled
(the #1056 VOIP_* / trinity-enterprise#31 LOG_* packaging class, caught by
/validate-pr step 4.9). Non-empty defaults for MODEL_ID per the #1076 pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vybe

vybe commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

PR Validation Report (/validate-pr)

PR: #1397 — feat(telegram): voice-out via ElevenLabs TTS + shared TTS foundation
Author: dolho · Branch: feature/25-telegram-voice-outdev · Files: 18 → 20 (+793/−2)

Category Status Notes
Commit Messages conventional feat/test prefixes, descriptive
Base Branch targets dev
PR Size 20 files
Status Auto-Promotion ⚠️ body says "Related to trinity-enterprise#25" — no closing keyword; cross-tracker anyway, so manual status-in-dev bump on ent#25 after merge
Requirements ⚠️ new capability, requirements.md not updated (epic #24 entry) — non-blocking per merge directive; fold into a follow-up slice
Architecture architecture.md updated (endpoint + service + schema)
Feature Flows ⚠️ no flow doc; acceptable for first slice, architecture carries the design
Security Check no keys/emails/IPs/secrets/cred files in diff; key never logged
Infrastructure ffmpeg added to backend image — justified in body
Build Packaging no new top-level backend modules; services/+db/ copied wholesale
Config Packaging ❌→✅ ELEVENLABS_API_KEY/ELEVENLABS_MODEL_ID/TTS_MAX_CHARS were wired into neither compose file (#1056/#1039 class — feature unreachable on deploy). Fixed on-branch in 8753600; .env.example was already documented
Schema/Migrations dual-track: SQLite agent_ownership_tts_voice + Alembic 0011 (parented on 0010), schema.py/tables.py updated — Alembic revision present (the schema-parity blind spot checked manually)
Test Adequacy 19 tests incl. non-happy paths: no-key, over-cap, provider error, ffmpeg missing/nonzero-rc, TTS-None→text fallback; pytest green on 6 seeds
Code Quality Invariants #1/#2/#8/#14 followed (router→service→db, TtsMixin, OwnedAgentByName on PUT, model in models.py); fail-soft everywhere; audit-logged config change

Recommendation: APPROVE (after the on-branch packaging fix)

Suggestions (non-blocking):

  • to_ogg_opus has no timeout on proc.communicate() — a wedged ffmpeg would hang the adapter send; consider asyncio.wait_for(..., 30) in a follow-up.
  • PR body says voice id is "validated on save" but the PUT only checks non-empty — either validate against ElevenLabs or drop the claim.
  • After merge: manually set status-in-dev on trinity-enterprise#25 (cross-tracker refs never auto-promote).

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated via /validate-pr — see report comment. Compose packaging gap fixed on-branch (8753600); everything else clean. Approving; merging once CI re-runs green.

@vybe
vybe merged commit 7897267 into dev Jul 2, 2026
21 checks passed
vybe pushed a commit that referenced this pull request Jul 2, 2026
/#1335/#1406 conflicts

Resolutions:
- requirements.md: take dev's index (#1406 split); relocate the #1205 section
  into requirements/public-access.md as Section 47 (46 was taken by #1335's
  roster requirement)
- Alembic: renumber 0011_agent_ownership_public_channel_prompt -> 0012,
  re-parent on 0011_agent_ownership_tts_voice (#1397 landed first; whichever
  lands second renumbers)
- db/migrations.py: keep both MIGRATIONS entries (tts_voice + public_channel_prompt)
- routers/sharing.py + SharingPanel.vue + architecture.md: keep both #1335
  roster and #1205 public-prompt surfaces (independent features)
- .claude + src/backend/enterprise submodules: reset to dev's pointers (the
  branch pinned unrelated/stale commits)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vybe pushed a commit that referenced this pull request Jul 3, 2026
- migrations.py, test_25_telegram_voice_out.py: parent-squash echoes; PR does
  not touch them -> take dev's versions (keeps #1337 public_channel_prompt
  migration and #1397's final test set)
- TelegramChannelPanel.vue: dev == feature/25 tip here; keep this branch's
  intended replacement of the inline voice block with shared VoiceRepliesControl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants