diff --git a/.claude/agents/test-runner.md b/.claude/agents/test-runner.md
index 2b3677083..6900fb39b 100644
--- a/.claude/agents/test-runner.md
+++ b/.claude/agents/test-runner.md
@@ -189,6 +189,8 @@ The test suite covers:
- **File Upload** (unit/test_file_upload.py) - Telegram file extraction, download, message router validation, parse_message with files (#354 Phase 1); workspace delivery — filename sanitization (NFKC, traversal, length, collision dedup), chat injection format `[File uploaded by {uploader}]`, all-writes-failed signaling, partial failure handling (#487 Phase 2) [UNIT]
- **Telegram Voice** (unit/test_telegram_voice.py) - Voice transcription validation (duration/size limits), formatting, placeholder constants (#318) [UNIT]
- **Inter-Agent Timeout** (test_inter_agent_timeout_unit.py) - FanOutRequest Optional timeout, per-subtask None dispatch, conditional asyncio.timeout wrap (#418) [UNIT]
+- **Subprocess PGroup** (unit/test_subprocess_pgroup.py) - Process-group lifecycle: terminate, drain_reader_threads (natural-drain ordering #531, buffered-data preservation), safe_close_pipes, signal_process_tree (#407, #531) [UNIT]
+- **Empty Result Classification** (unit/test_empty_result_classification.py) - Clean exit with lost result line → 502, two-field null check, raw_messages num_turns fallback, populated-metadata pass-through (#520, #521, #531) [UNIT]
### Avatars & Image Generation
- **Avatars** (test_avatars.py) - Avatar serving, generation, regeneration, deletion, emotions, identity prompts, default generation (AVATAR-001/002/003) [SMOKE + Agent]
@@ -231,9 +233,9 @@ The test suite covers:
## Test Suite Statistics
-**Total Tests**: ~2,257 tests across 124 test files
+**Total Tests**: ~2,262 tests across 124 test files
**Smoke Tests**: ~578 tests (fast, no agent creation)
-**Unit Tests**: ~165 tests (no backend needed, rate limit detection, watchdog logic, context formula, OTel trace logging, file upload, voice transcription, inter-agent timeout, scheduler sync loop, team-share gate, WhatsApp adapter (67))
+**Unit Tests**: ~170 tests (no backend needed, rate limit detection, watchdog logic, context formula, OTel trace logging, file upload, voice transcription, inter-agent timeout, scheduler sync loop, team-share gate, WhatsApp adapter (67), subprocess pgroup (11), empty result classification (13))
**Core Tests (not slow)**: ~2,112 tests
**Slow Tests**: ~99 tests (chat execution, fleet ops, system agent ops, execution termination, WhatsApp live-backend integration)
**WebSocket Tests**: ~10 tests (web terminal, execution streaming)
@@ -263,6 +265,40 @@ Use these thresholds to assess test health (based on **executed** tests, not inc
- **Warning**: 75-90% pass rate, <5 failures
- **Critical**: <75% pass rate or >5 failures
+## Recent Test Additions (2026-04-27)
+
+| Test File | Description | Tests Added |
+|-----------|-------------|-------------|
+| `unit/test_subprocess_pgroup.py` | Regression test for #531 — `test_buffered_data_preserved_after_grandchild_kill` verifies that data written to stdout before a grandchild is killed (including the final result JSON line) is not dropped when `drain_reader_threads` is called | +1 test (file total 11) |
+| `unit/test_empty_result_classification.py` | Four new tests for `_classify_empty_result` raw_messages fallback (#531): `test_raw_messages_fallback_derives_num_turns`, `test_raw_messages_fallback_not_used_when_num_turns_present`, `test_raw_messages_empty_falls_back_to_zero`, `test_raw_messages_none_falls_back_to_zero` | +4 tests (file total 13) |
+
+**Pipe drain ordering fix (#531)** — `drain_reader_threads` in `subprocess_pgroup.py`:
+
+Root cause of the "Execution completed without a result message" HTTP 502 on long agentic tasks. The old code called `safe_close_pipes()` immediately after `terminate_process_group()`, discarding the kernel pipe buffer (including the `{"type":"result"}` final JSON line) before the reader thread could drain it. Fixed: kill grandchildren first, then wait up to `post_kill_grace=30s` for natural drain; force-close only as a last resort for genuine wedges.
+
+The new `test_buffered_data_preserved_after_grandchild_kill` test spawns a subprocess that writes a sentinel `RESULT_LINE` then forks a grandchild that holds stdout open; verifies the sentinel survives the grandchild kill. The `_classify_empty_result` tests verify that `num_turns` is derived from `raw_messages` when the result line was lost (since `metadata.num_turns` is only populated by that line), while `tool_count` continues to come from metadata (accumulated per-message during parsing, so accurate even without the result line).
+
+---
+
+## Recent Test Additions (2026-04-25)
+
+| Test File | Description | Tests Added |
+|-----------|-------------|-------------|
+| `unit/test_github_init_gitignore.py` | Gitignore widen + migrate-on-push (#462) — full exclusion list coverage, idempotency, doc/constant sync, `git rm --cached` for newly-ignored files | 4 new tests (file total 6) |
+
+**Gitignore Widen + Migrate (#462)** (`unit/test_github_init_gitignore.py`):
+
+- `test_preserves_preexisting_gitignore` — existing `.gitignore` content is retained after init (pre-existing test)
+- `test_fresh_agent_ignores_env_and_mcp_json` — fresh agent gets `.env` and `.mcp.json` exclusions (pre-existing test)
+- `test_full_documented_exclusion_list_present` — all patterns in `_GITIGNORE_PATTERNS` constant are written to `.gitignore` (new)
+- `test_idempotent_double_run` — running `initialize_git_in_container` twice does not duplicate patterns (new)
+- `test_doc_and_constant_in_sync` — `_GITIGNORE_PATTERNS` constant matches the exclusion block in `TRINITY_COMPATIBLE_AGENT_GUIDE.md` (new)
+- `test_rm_cached_for_newly_ignored_files` — `sync_to_github` calls `git rm --cached` for files that are tracked but now match a `.gitignore` pattern (new)
+
+Pure unit tests over `git_service.py` stubs; no backend container required. Run via `pytest tests/unit/test_github_init_gitignore.py`.
+
+---
+
## Recent Test Additions (2026-04-23)
| Test File | Description | Tests Added |
diff --git a/.claude/skills/cso/SKILL.md b/.claude/skills/cso/SKILL.md
index f55c958a3..7e0281bbf 100644
--- a/.claude/skills/cso/SKILL.md
+++ b/.claude/skills/cso/SKILL.md
@@ -132,7 +132,20 @@ For each workflow file in `.github/workflows/`:
**Docker socket access**: Check if backend has Docker socket mounted and what permissions it has.
-**Severity**: CRITICAL for prod DB URLs with credentials in committed config. HIGH for root containers in prod / Docker socket access without read-only. MEDIUM for missing USER directive / exposed ports.
+**Redis authentication and network isolation**:
+- Check `redis.conf` or Docker env for `requirepass` — unauthenticated Redis reachable from the agent container network is CRITICAL (agents get full read/write to all task queues and secrets including other tenants')
+- Check if agent containers have direct TCP access to Redis (e.g., `redis:6379`) — they should reach the platform only via the backend HTTP API, not the Redis port directly
+- Check Redis keyspace for per-user isolation — if all users share a Redis namespace without user-scoped key prefixes, cross-tenant enumeration is possible even with auth
+
+**Docker `base_image` allowlist**:
+- Search agent creation code for the `base_image` field — verify it's validated against an allowlist (regex or explicit list), not accepted as free text from the API request body
+- An unvalidated `base_image` lets any authenticated user pull and execute arbitrary Docker images inside the agent network, giving access to Redis, the MCP server, and credential env vars
+
+**Setup/onboarding endpoint exposure**:
+- Check if the first-run setup endpoint (`/api/setup`, `/setup`, etc.) is network-accessible or bound to 127.0.0.1 only
+- If accessible from the public internet, check whether it uses a single-use bootstrapping token to prevent a race-condition hijack granting full admin control to an external caller
+
+**Severity**: CRITICAL for Redis without auth accessible from agent network. CRITICAL for unvalidated `base_image`. HIGH for setup endpoint reachable from public internet without a single-use token. HIGH for root containers in prod / Docker socket access without read-only. MEDIUM for missing USER directive / exposed ports.
**FP rules**: `docker-compose.yml` for local dev with localhost = not a finding.
@@ -140,6 +153,14 @@ For each workflow file in `.github/workflows/`:
**Webhook routes**: Find files containing webhook/hook/callback route patterns. Check whether they also contain signature verification.
+**Internal endpoint auth completeness**:
+- For every route in the internal router (routes under `/api/internal/` or any "internal" prefix), verify each endpoint enforces the shared-secret header (`X-Internal-Secret`) — not just that the pattern exists in the file, but that every individual route uses it
+- Probe: send request to each internal endpoint without the header and confirm 401/403, not 200
+
+**Webhook trigger auth (not just signature)**:
+- For webhook trigger endpoints, check for the presence of `Depends(get_current_user)` or an equivalent auth gate — not just HMAC signature verification
+- Trigger routes have been found missing auth entirely while signature verification exists elsewhere in the same file; the two are independent checks
+
**TLS verification disabled**: Search for `verify=False`, `VERIFY_NONE`, etc.
**MCP server security**: Check `src/mcp-server/` for authentication, input validation, and tool permission boundaries.
@@ -156,8 +177,9 @@ Trinity-specific AI security concerns:
- **AI API keys in code**: Hardcoded API key assignments (not env vars)
- **Credential injection safety**: Are injected credentials exposed to agent code in unexpected ways?
- **Cost/resource attacks**: Can a user trigger unbounded LLM calls via agent chat?
+- **Agent container network reach**: From inside an agent container, check what internal services are directly reachable (Redis, MCP server backend port, etc.). Agent containers should reach the platform only through the backend HTTP API — direct TCP access to Redis, internal service ports, or other agents' containers outside the API layer is a CRITICAL isolation failure. Check `docker-compose.yml` network definitions and confirm agent containers cannot reach `redis:6379` or `mcp-server:8080` directly.
-**Severity**: CRITICAL for user input in system prompts / unsanitized LLM output rendered as HTML. HIGH for missing tool call validation / exposed AI API keys. MEDIUM for unbounded LLM calls.
+**Severity**: CRITICAL for user input in system prompts / unsanitized LLM output rendered as HTML. CRITICAL for agent containers with direct Redis or internal service access bypassing the API. HIGH for missing tool call validation / exposed AI API keys. MEDIUM for unbounded LLM calls.
**FP rules**: User content in the user-message position of an AI conversation is NOT prompt injection.
@@ -178,10 +200,14 @@ For each OWASP category, perform targeted analysis. Scope file extensions to Pyt
- Missing auth on routes (check FastAPI dependency injection for auth)
- Direct object reference (can user A access user B's agents by changing IDs?)
- Horizontal/vertical privilege escalation
+- **Internal router completeness**: Every endpoint in the internal router must use the shared-secret dependency — audit the full router, not a sample. No route should return 200 without the secret header.
+- **Role-permission consistency across ALL creation routes**: For each resource-creation endpoint (`POST /api/agents`, `/api/processes`, `/deploy-local`, webhook-triggered deployments, etc.), verify the same role-check dependency is used. The lowest-privilege role must not bypass the creation gate via any alternate route.
+- **File-write path policy completeness**: For any file-write API, verify the path-deny/allowlist is evaluated in full — not just basename matching. Specifically check: can `.mcp.json`, `.env`, `.credentials.enc`, or `.ssh/*` be reached via path traversal or a sibling endpoint (e.g., `/credentials/inject`) that enforces a shorter deny list?
#### A02: Cryptographic Failures
- Weak crypto (MD5, SHA1, DES, ECB) or hardcoded secrets
- JWT implementation (algorithm, expiration, secret management)
+- **SSH key server-side generation**: Check if SSH private keys are generated server-side and returned in API responses. Flag as HIGH — clients should generate keypairs locally and submit only the public key. Server-generated private keys are exposed in transit and in API/access logs.
#### A03: Injection
- SQL injection: raw queries, string interpolation in SQL (check `database.py`)
@@ -192,6 +218,8 @@ For each OWASP category, perform targeted analysis. Scope file extensions to Pyt
- Rate limits on authentication endpoints?
- Account lockout after failed attempts?
- Business logic validated server-side?
+- **Rate limiting quality**: Per-IP-only rate limits are bypassable via rotating proxies/Tor/IPv6 subnets. Check that rate limiting on auth endpoints (login, OTP, password reset) uses a per-account/per-email counter as the primary bucket. Also check that hard lockout on IP isn't itself a DoS primitive — progressive delay or CAPTCHA is safer than a hard block.
+- **OTP brute-force window**: For email OTP flows, verify the code space (6 digits = 1M possibilities) is protected by an attempt counter that survives process restarts (stored in Redis/DB, not in-memory), and that codes expire within 5–10 minutes.
#### A05: Security Misconfiguration
- CORS configuration (wildcard origins?)
diff --git a/.claude/skills/generate-user-docs/SKILL.md b/.claude/skills/generate-user-docs/SKILL.md
index e27bbe66f..95bbdfd30 100644
--- a/.claude/skills/generate-user-docs/SKILL.md
+++ b/.claude/skills/generate-user-docs/SKILL.md
@@ -30,9 +30,10 @@ Read backend routers, frontend views, feature flows, and recent changes to produ
| Feature flow index | `docs/memory/feature-flows.md` | Yes | No |
| Requirements | `docs/memory/requirements.md` | Yes | No |
| Architecture | `docs/memory/architecture.md` | Yes | No |
-| Deployment config | `.env.example`, `scripts/deploy/*.sh`, `docker-compose.yml` | Yes | No |
+| Deployment config | `.env.example`, `scripts/deploy/*.sh`, `docker-compose.yml`, `docker-compose.prod.yml`, `deploy.config.example` | Yes | No |
| Trinity Docs site | `../trinity-docs/app/getting-started/*.tsx` | Yes | No |
| Abilities repo | `github.com/abilityai/abilities` (README) | Yes | No |
+| Ops runbook (private, pattern source for ops content) | `../ops-runbook/playbooks/*.md`, `../ops-runbook/instances/_template/scripts/*.sh`, `../ops-runbook/instances/_template/CLAUDE.md` | Yes | No |
| Git history | `git log --since` | Yes | No |
| Existing user docs | `docs/user-docs/**/*.md` | Yes | Yes |
@@ -47,19 +48,70 @@ These tutorial-style guides walk users through end-to-end tasks. Keep them in sy
| Guide | Source | Purpose |
|-------|--------|---------|
-| `guides/deploying-trinity.md` | `trinity-docs/app/getting-started/deploying-trinity/page.tsx` | Cloud vs self-hosted setup |
+| `guides/deploying-trinity.md` | `trinity-docs/app/getting-started/deploying-trinity/page.tsx` + this skill's deploy hub rules | Hub: cloud vs self-hosted decision, prerequisites, links into spokes |
+| `guides/deploying/local-development.md` | `docker-compose.yml`, `scripts/deploy/start.sh`, `.env.example` | Docker Desktop, dev compose, hot reload, base image build |
+| `guides/deploying/single-server.md` | `docker-compose.prod.yml`, `deploy.config.example`, `.env.example` | VPS/bare-metal: prod compose, env, base image, email, Redis password, host paths |
+| `guides/deploying/public-access.md` | `docker-compose.prod.yml` (cloudflared profile), `.env.example` (TUNNEL_TOKEN, PUBLIC_CHAT_URL) | Cloudflare Tunnel, public webhook surface, DNS |
+| `guides/deploying/upgrading.md` | This skill's operational template + ops runbook patterns | Pre-flight → backup → pull → rebuild platform services → restart → verify → rollback |
+| `guides/deploying/backup-and-restore.md` | This skill's operational template + ops runbook patterns | Volume-mounted alpine `cp` pattern, retention, daily cron template |
+| `guides/deploying/monitoring.md` | This skill's operational template + ops runbook patterns + `/api/ops/fleet/health` router | Six health probes, fleet-health API, resource thresholds table, common-recovery patterns |
| `guides/using-trinity.md` | `trinity-docs/app/getting-started/using-trinity/page.tsx` | UI tour: dashboard, agents, monitoring |
| `guides/building-agents.md` | `trinity-docs/app/getting-started/building-agents/page.tsx` | Create, develop, deploy with abilities |
**Sync rule**: When the trinity-docs source changes, update the corresponding guide to match. Convert TSX to markdown, preserving structure and content. **Code wins on conflict** — if trinity-docs disagrees with `.env.example`, `scripts/deploy/*.sh`, or `docker-compose.yml`, fix the local guide to match observed repo behavior and note the divergence for upstream.
+**Ops-runbook rule**: Pages under `guides/deploying/` (especially `upgrading.md`, `backup-and-restore.md`, `monitoring.md`) draw their *patterns* from the local private ops runbook. Treat that as a pattern library, not a paste source. See Step 2h for what's safe to import vs. what must stay private.
+
+### Deployment config reading rules
+
+When reading scripts and compose files to produce deployment docs:
+
+1. Read `scripts/deploy/start.sh` literally — document what it **actually does**, not what comments say it does
+2. Check `docker-compose.yml` environment blocks for which vars are **actually forwarded** to each service
+3. For vars in `.env.example` marked `[PROD]` or `[OVERLAY]`, note their scope clearly — do not present them as universally available in dev compose
+4. Never describe a feature as "auto" unless the script code proves it is
+5. `verify-platform.sh` is the canonical six-probe checklist — reference it by name, don't duplicate it inline
+
+### Operational guide template
+
+Pages under `guides/deploying/` that cover operations (upgrade, backup, monitoring) follow this structure:
+
+```markdown
+## [Operation Name]
+
+[One sentence: what this operation does and when to run it]
+
+### Pre-flight
+
+[What to check before starting. Backup steps. Smoke tests.]
+
+### Steps
+
+[Numbered, concrete commands. No vague instructions.]
+
+### Verify
+
+[How to confirm success. Commands to run. What to look for.]
+
+### Rollback
+
+[How to undo if something went wrong. Commands to restore prior state.]
+```
+
## Target Structure
```
docs/user-docs/
├── README.md # Index + navigation
├── guides/ # Tutorial-style walkthroughs
-│ ├── deploying-trinity.md # Cloud vs self-hosted setup
+│ ├── deploying-trinity.md # Hub: cloud vs self-hosted decision, links into spokes
+│ ├── deploying/ # Self-hosted deployment spokes (per-scenario + ops)
+│ │ ├── local-development.md # Docker Desktop, dev compose, hot reload, base image
+│ │ ├── single-server.md # VPS/bare-metal: prod compose, env, email, backups
+│ │ ├── public-access.md # Cloudflare Tunnel, PUBLIC_CHAT_URL, webhook surface
+│ │ ├── upgrading.md # Pre-flight → backup → rebuild → restart → verify → rollback
+│ │ ├── backup-and-restore.md # Volume-mounted alpine cp, retention, daily cron
+│ │ └── monitoring.md # Six health probes, /api/ops/fleet/health, thresholds
│ ├── using-trinity.md # UI tour: dashboard, agents, monitoring
│ └── building-agents.md # Create, develop, deploy with abilities
├── getting-started/
@@ -140,7 +192,7 @@ git log --oneline --since="2 weeks ago" | head -30
```
This identifies what has changed recently and which docs may need updating.
-**2e. Deployment config (for deploying/setup guides)** — Read `.env.example`, `scripts/deploy/start.sh`, `scripts/deploy/stop.sh`, and `docker-compose.yml`. Extract: required env vars, auto-generated vs user-set secrets, default ports and override vars (e.g., `FRONTEND_PORT`), first-boot behavior, and what the start/stop scripts actually print and do. This is the authoritative source for deploy/setup docs.
+**2e. Deployment config (for deploying/setup guides)** — Read `.env.example`, `scripts/deploy/start.sh`, `scripts/deploy/stop.sh`, `scripts/deploy/build-base-image.sh`, `docker-compose.yml`, `docker-compose.prod.yml`, `docker-compose.gitea.yml`, and `deploy.config.example`. Extract: required env vars, auto-generated vs user-set secrets, default ports and override vars (e.g., `FRONTEND_PORT`), first-boot behavior, and what the start/stop scripts actually print and do. **Cross-check `.env.example` keys against the `environment:` blocks in each compose file** — flag any key present in `.env.example` but not forwarded by a given compose as "prod-only / overlay-only / requires compose edit," because docs should not promise behavior the chosen compose can't deliver. This is the authoritative source for deploy/setup docs.
**2f. Backend routers** — Glob `src/backend/routers/*.py` and read router files relevant to the section being written. Extract:
- Endpoint paths and HTTP methods
@@ -152,6 +204,29 @@ This identifies what has changed recently and which docs may need updating.
- User-facing labels and actions
- State management patterns
+**2h. Ops-runbook patterns (for `guides/deploying/upgrading.md`, `backup-and-restore.md`, `monitoring.md`)** — Read the local private ops runbook: `playbooks/upgrade-instance.md`, `playbooks/monitoring-instance.md`, `instances/_template/CLAUDE.md`, and the helper scripts in `instances/_template/scripts/` (`update.sh`, `health-check.sh`, `restart.sh`, `status.sh`). Treat as a **pattern library**, not a paste source.
+
+**Safe to import (the rules and shapes that make production work):**
+- The "use `docker compose restart`, never `down/up`" rule and *why* (preserves agent containers and `trinity-agent-network`).
+- The "rebuild only platform services, not the base image" rule (`docker compose build --no-cache backend frontend mcp-server scheduler`).
+- The pre-flight → backup → pull → rebuild → restart → verify → rollback sequence shape.
+- The six-probe verification list (backend `/health`, scheduler `:8001/health`, frontend HTTP 200, redis PONG, MCP `/health`, Vector `:8686/health`).
+- The fleet-status API surface (`/api/ops/fleet/health`, `/api/ops/fleet/status`).
+- The resource-thresholds table (warning/critical bands for context %, CPU, memory, disk, error rate, container restarts, DB size, log size).
+- The volume-mounted alpine `cp` pattern for SQLite backup/restore.
+- Common-recovery patterns (agent network not found, agent context >90%, database locked).
+- The daily-DB-backup + 14-day-retention cron template.
+
+**Forbidden to import (private detail; would leak in a public repo):**
+- Any host name, IP address, Tailscale identity, or tailnet name.
+- Any `sshpass`, `ssh -i`, or `ssh user@host`-prefixed command — local-host examples only.
+- Any reference to specific instance directories (`instances/dgx/` etc.) or instance-specific scripts.
+- Any password, token, or API-key value (even masked) from ops `.env` files.
+- Any private repo name or internal path that reveals the ops repo's structure.
+- Multi-instance management workflows (`source .env && ./scripts/run.sh ...`) — that's an operator-fleet pattern, not a single-instance user pattern.
+
+When in doubt, rewrite the *idea* in localhost form. A production-ops `sshpass -p $PW ssh user@host "sudo docker logs trinity-backend"` becomes the public `docker logs trinity-backend`. The rule survives; the access pattern stays private.
+
### Step 3: Generate/Update Documentation
For each section in the target structure, produce or update the markdown file following these rules:
@@ -208,6 +283,137 @@ Only include if meaningful.]
Use judgment: if the same content keeps needing to sit awkwardly inside "How It Works" or "Limitations," promote it to its own `##` section.
+#### Operational guide template (use for `guides/deploying/upgrading.md`, `backup-and-restore.md`, `monitoring.md`)
+
+The dual-audience template above is for *features*. Operational guides are *procedures with checkpoints* — different shape. Use this instead:
+
+```markdown
+# [Procedure Name]
+
+[1-2 sentence summary: what this procedure achieves and when an operator runs it.]
+
+## When to Run This
+
+[Trigger conditions. E.g. "Before every Trinity update," "Daily as a cron job,"
+"When the dashboard sync-health dot goes red." Concrete, not aspirational.]
+
+## Pre-flight
+
+- [ ] [Check 1 — e.g., backup target has space]
+- [ ] [Check 2 — e.g., no scheduled tasks running in the next N minutes]
+- [ ] [Check 3 — e.g., on the right branch / version]
+
+## Procedure
+
+### Step 1: [name]
+
+[What and why. Then the command in a code block. Then "expected output: ..."]
+
+### Step 2: [name]
+...
+
+## Verify
+
+After the procedure, confirm each of these returns the expected result. If any
+fails, do NOT proceed — go to **Rollback** or **Recovery** below.
+
+| Check | Command | Expected |
+|-------|---------|----------|
+| Backend | `curl -s http://localhost:8000/health` | `{"status":"healthy",...}` |
+| ... | ... | ... |
+
+## Rollback / Recovery
+
+[For procedures with destructive or risky steps, include the inverse procedure.
+For monitoring guides, include common-issue resolution table instead.]
+
+## Automation
+
+[Optional: cron template, CI hook, or scheduling guidance if the procedure is
+intended to run regularly.]
+
+## See Also
+```
+
+**Why this shape:** operators care about (1) when to run, (2) safety pre-checks, (3) numbered steps, (4) explicit verification, (5) rollback. The feature template's "How It Works / For Agents / Limitations" doesn't map to any of those.
+
+#### Reusable snippets for operational guides
+
+The following are canonical snippets. Use verbatim across the deploy spokes so the same rule reads identically in every place. Do **not** paraphrase; consistency is the point.
+
+**The "use restart, never down/up" rule:**
+
+> **Use `docker compose restart`, not `down/up`.** `docker compose down` removes the `trinity-agent-network`, which orphans every running agent container — they keep running but lose their network and have to be removed and recreated. `restart` preserves both the agents and the network. The only times to use `down` are: (1) intentional full teardown, (2) recovering from a corrupted compose state.
+
+**The "rebuild platform services only" rule:**
+
+> When updating Trinity code, rebuild the platform images only:
+> ```bash
+> docker compose build --no-cache backend frontend mcp-server scheduler
+> ```
+> The `trinity-agent-base` image is **not** rebuilt by this command. It changes much less often, and rebuilding it forces every agent to be re-deployed. Rebuild it only when `docker/base-image/Dockerfile` itself changes, via `./scripts/deploy/build-base-image.sh`.
+
+**The six-probe verification list:**
+
+```bash
+# 1. Backend
+curl -s http://localhost:8000/health
+# Expected: {"status":"healthy",...}
+
+# 2. Scheduler
+curl -s http://localhost:8001/health
+# Expected: {"status":"healthy","active_schedules":N}
+
+# 3. Frontend (HTTP 200)
+curl -s -o /dev/null -w '%{http_code}' http://localhost
+# Expected: 200
+
+# 4. Redis
+docker exec trinity-redis redis-cli ping
+# Expected: PONG
+
+# 5. MCP Server
+curl -s http://localhost:8080/health
+# Expected: 200 OK
+
+# 6. Vector (log aggregation)
+docker exec trinity-vector wget -q -O - http://localhost:8686/health
+# Expected: non-empty response
+```
+
+**Resource thresholds table:**
+
+| Metric | Warning | Critical | Action |
+|---|---|---|---|
+| Backend `/health` | — | not 200 | Restart `trinity-backend` |
+| Scheduler `/health` | — | not 200 | Restart `trinity-scheduler` |
+| Agent context usage | >75% | >90% | Reset agent context or restart agent container |
+| Host CPU | >80% | >95% | Investigate runaway processes |
+| Host memory | >85% | >95% | Check container memory limits |
+| Disk free | <20% | <5% | Prune Docker, archive logs |
+| Error rate (per hour) | >10 | >50 | Inspect platform.json log |
+| Container restarts | any | repeated | `docker logs ` |
+| `trinity.db` size | >1 GB | >5 GB | Archive old data |
+| Vector log size | >5 GB | >10 GB | Trigger archival rotation |
+
+**SQLite backup pattern (volume-mounted alpine):**
+
+```bash
+# Backup (run on the host, with services running)
+docker run --rm \
+ -v trinity_trinity-data:/data \
+ -v ~/backups:/backup \
+ alpine cp /data/trinity.db /backup/trinity-$(date +%Y%m%d-%H%M%S).db
+
+# Restore (services stopped first)
+docker compose stop backend scheduler
+docker run --rm \
+ -v trinity_trinity-data:/data \
+ -v ~/backups:/backup \
+ alpine cp /backup/trinity-YYYYMMDD-HHMMSS.db /data/trinity.db
+docker compose start backend scheduler
+```
+
3. **No redundancy** — Do not repeat information from other docs. Cross-reference instead. The `Concepts` section in `getting-started/overview.md` is the canonical glossary; other docs reference it rather than re-defining terms.
4. **Code-derived accuracy** — Every claim must trace to code or a feature flow. Do not invent features. If a feature flow says "planned" or a router has TODO comments, note it as upcoming rather than documenting it as available.
@@ -263,25 +469,46 @@ Create directories and write all approved files:
```bash
mkdir -p docs/user-docs/{getting-started,agents,credentials,collaboration,automation,operations,sharing-and-access,integrations,advanced,api-reference}
+mkdir -p docs/user-docs/guides/deploying
```
Write each markdown file using the Write or Edit tool.
### Step 7: Verify
+**Count files written:**
+
```bash
find docs/user-docs -name "*.md" -type f | wc -l
```
+**Public-safety scan** — every page generated under `guides/deploying/` must pass these greps with zero hits. If any hit, the page is leaking ops-internal detail and must be rewritten:
+
+```bash
+# Tokens that indicate private-repo or operator-fleet patterns
+grep -rE 'sshpass|tailnet|ts\.net|ssh -i [^ ]+ [^ ]+@' docs/user-docs/guides/deploying/
+
+# Real IPs (allow only loopback and the documented agent subnet 172.28.0.0/16)
+grep -rE '\b(10|192\.168|172\.(1[6-9]|2[0-9]|3[01]))\.[0-9]+\.[0-9]+' docs/user-docs/guides/deploying/ \
+ | grep -vE '172\.28\.0\.[0-9]+/16'
+
+# Instance directory references that hint at private-repo structure
+grep -rE 'instances/[a-z0-9_-]+/' docs/user-docs/guides/deploying/
+```
+
Confirm the expected number of files were created/updated. Report the final count.
## Completion Checklist
-- [ ] All sections in target structure have corresponding files
-- [ ] Every doc follows the dual-audience template (How It Works + For Agents)
+- [ ] All sections in target structure have corresponding files (including `guides/deploying/` spokes)
+- [ ] Every feature doc follows the dual-audience template (How It Works + For Agents)
+- [ ] Every operational doc under `guides/deploying/` follows the operational template (When to Run → Pre-flight → Procedure → Verify → Rollback)
- [ ] No redundant explanations across docs (MECE verified)
- [ ] All API references link to Swagger (`/docs`) rather than duplicating schemas
-- [ ] No real credentials, internal URLs, or PII in any doc
+- [ ] Every key in `.env.example` is annotated in the relevant deploy spoke as: dev-compose / prod-compose-only / overlay-only / requires-compose-edit (no key promised to work in a compose that doesn't forward it)
+- [ ] Reusable snippets (the "use restart" rule, "rebuild platform services only" rule, six-probe verification, thresholds table, alpine `cp` backup pattern) are used verbatim, not paraphrased, across deploy spokes
+- [ ] No real credentials, internal URLs, host IPs, or PII in any doc
+- [ ] Public-safety greps from Step 7 return zero hits across `guides/deploying/`
- [ ] README.md index is complete and links are valid
- [ ] Changes reviewed by user before writing
diff --git a/.claude/skills/test-runner-ui-e2e/SKILL.md b/.claude/skills/test-runner-ui-e2e/SKILL.md
new file mode 100644
index 000000000..04b202dd4
--- /dev/null
+++ b/.claude/skills/test-runner-ui-e2e/SKILL.md
@@ -0,0 +1,300 @@
+---
+name: test-runner-ui-e2e
+description: Run the Playwright frontend e2e suite against a live Trinity stack, analyze failures, and update visual regression snapshots. Use after UI changes, before merging a PR with the `ui` label, or when the `frontend-e2e` CI workflow fails.
+allowed-tools: [Bash, Read, Write, Edit, Grep, Glob]
+user-invocable: true
+argument-hint: "[] [--update-snapshots] [--headed] [--ui]"
+automation: manual
+---
+
+# Test Runner — UI E2E
+
+Run the Playwright e2e suite that lives in `src/frontend/e2e/` against a live Trinity stack. Frontend equivalent of `/test-runner` (which covers backend pytest only).
+
+## When to Use
+
+- After any change in `src/frontend/src/` to confirm the UI still works end-to-end
+- Before merging a PR carrying the `ui` label (the same PRs that trigger `frontend-e2e` in CI)
+- When the `frontend-e2e` CI job fails — to reproduce locally and diagnose
+- After a design-system migration to update visual regression snapshots
+- When adding new specs to `e2e/` to validate them locally before pushing
+
+## Usage
+
+```
+/test-runner-ui-e2e # Run the full suite against http://localhost
+/test-runner-ui-e2e smoke # Run specs matching "smoke"
+/test-runner-ui-e2e --update-snapshots # Update visual regression baselines
+/test-runner-ui-e2e --headed # Run with a visible browser
+/test-runner-ui-e2e --ui # Open Playwright's interactive UI
+```
+
+## Arguments
+
+| Arg | Description |
+|---|---|
+| `` | Pattern passed to `playwright test --grep` |
+| `--update-snapshots` | Regenerate `e2e/**/*-snapshots/*.png` baselines after intentional UI changes |
+| `--headed` | Run with a visible Chromium window (debugging) |
+| `--ui` | Open Playwright's interactive test UI (debugging) |
+
+## Process
+
+### Step 1: Prerequisites check
+
+Before running tests, verify the stack is up. The e2e workflow assumes Trinity is reachable at the configured `baseURL` (default `http://localhost`).
+
+```bash
+# 1. Trinity backend health
+curl -fsS http://localhost:8000/health
+
+# 2. Trinity frontend (Vite dev or nginx) on port 80
+curl -sI http://localhost/ | head -1
+
+# 3. Containers
+docker ps --filter 'name=trinity' --format 'table {{.Names}}\t{{.Status}}'
+```
+
+If backend is down or returning 404, **diagnose and recover** (see "Common failure modes" below) before trying tests. **Do not** skip this step — every failure I've seen has a stack-state precondition.
+
+### Step 2: Resolve admin password
+
+The Playwright auth setup needs `ADMIN_PASSWORD`:
+
+```bash
+ADMIN_PASSWORD=$(grep '^ADMIN_PASSWORD=' .env | cut -d= -f2)
+test -n "$ADMIN_PASSWORD" || { echo "ADMIN_PASSWORD not set in .env"; exit 1; }
+```
+
+Confirm it actually works by hitting the login endpoint directly first:
+
+```bash
+curl -s -X POST http://localhost:8000/api/token \
+ --data-urlencode "username=admin" --data-urlencode "password=$ADMIN_PASSWORD" \
+ | head -c 200
+```
+
+If that returns `{"detail":"Not Found"}` the backend is dead. If it returns `Invalid username or password`, the password is wrong. If it returns `{"access_token": "..."}`, you're good to run tests.
+
+### Step 3: Run Playwright
+
+From `src/frontend/`:
+
+```bash
+cd src/frontend
+rm -rf e2e/.auth/admin.json e2e/test-results 2>/dev/null
+
+# Translate the skill args into npm scripts
+SCRIPT="test:e2e"
+[[ "$ARGS" == *"--update-snapshots"* ]] && SCRIPT="test:e2e:update"
+[[ "$ARGS" == *"--headed"* ]] && SCRIPT="test:e2e:headed"
+[[ "$ARGS" == *"--ui"* ]] && SCRIPT="test:e2e:ui"
+
+ADMIN_PASSWORD="$ADMIN_PASSWORD" npm run "$SCRIPT" -- ${FILTER:+--grep "$FILTER"}
+```
+
+### Step 4: Parse results
+
+A passing run looks like:
+
+```
+Running 5 tests using 4 workers
+ ✓ 1 [setup] › e2e/auth.setup.js:8:6 › authenticate as admin (2.5s)
+ ✓ 2 [chromium] › e2e/smoke.spec.js:25:7 › smoke › templates page loads (842ms)
+ ...
+ 5 passed (5.1s)
+```
+
+Failing runs include attachments (screenshot + video + trace) under `e2e/test-results//`. The most useful artifact is `error-context.md` which has the page snapshot at the moment of failure.
+
+### Step 5: Generate report
+
+```
+## E2E Test Results
+
+**Status**: PASSED / FAILED
+**Duration**: Xs
+**Specs**: X passed, Y failed
+**Browser**: chromium
+
+### Failures (if any)
+- spec name:
+ → File: e2e/:
+ → Page state:
+
+### HTML report
+e2e/playwright-report/index.html
+
+### Recommendations
+- ...
+```
+
+If snapshots updated: list the new/changed `*.png` paths so the user knows to commit them.
+
+### Step 6: Reminder about the `ui` label
+
+If the run was triggered for a PR-bound change, remind the user:
+
+> Add the `ui` label to your PR so CI runs the same suite (`frontend-e2e.yml`).
+
+## Common failure modes (learned 2026-04-29)
+
+These patterns recur. Diagnose by checking the page snapshot in `e2e/test-results//error-context.md`.
+
+### A. Stack is down before tests run
+
+**Symptom**: `auth.setup` fails with `net::ERR_CONNECTION_REFUSED at http://localhost/`.
+
+**Cause**: `trinity-frontend` or `trinity-backend` container exited.
+
+**Diagnose**:
+```bash
+docker ps --filter 'name=trinity' --format 'table {{.Names}}\t{{.Status}}'
+docker logs trinity-backend --tail 20
+```
+
+**Recover**:
+```bash
+docker rm -f trinity-backend 2>/dev/null
+docker compose up -d backend
+```
+
+If port 8000 is "already allocated" by `com.docker.backend` after `docker rm -f`, you have the **Docker Desktop port-zombie bug**. The only reliable fix is restarting Docker Desktop:
+```bash
+osascript -e 'quit app "Docker Desktop"'
+# wait for daemon stop, then reopen
+open -a "Docker Desktop"
+```
+
+### B. Stuck on /setup wizard (CI fresh-DB scenario)
+
+**Symptom**: Page snapshot shows `` rather than the login form.
+
+**Cause**: Fresh Trinity DB. Migration #19 (`setup_completed_backfill`) runs *before* `_ensure_admin_user`, finds no admin, skips backfill. Admin is then created from `ADMIN_PASSWORD` env var, but `setup_completed` stays `false` — the frontend redirects all routes to `/setup`.
+
+**Recover**:
+```bash
+docker exec trinity-backend python3 -c "from database import db; db.set_setting('setup_completed', 'true')"
+```
+
+This is exactly what the CI workflow's "Skip first-time setup wizard" step does.
+
+### C. "Invalid username or password"
+
+**Symptom**: Page snapshot shows the "Access Denied" panel with that exact text.
+
+**Causes** (in order of likelihood):
+1. `ADMIN_PASSWORD` env in your shell ≠ `ADMIN_PASSWORD` in the running backend container (backend was started before `.env` was edited)
+2. Username changed (default is `admin`, but `ADMIN_USERNAME` env var can override)
+
+**Diagnose**:
+```bash
+docker exec trinity-backend printenv ADMIN_USERNAME
+docker exec trinity-backend printenv ADMIN_PASSWORD | head -c 5 # first 5 chars to compare
+diff <(echo "$ADMIN_PASSWORD") <(docker exec trinity-backend printenv ADMIN_PASSWORD)
+```
+
+**Recover**: restart backend so it picks up `.env`:
+```bash
+docker compose restart backend
+```
+
+### D. Submit button found but disabled
+
+**Symptom**: `locator.click: ... element is not enabled`. Click retries 50+ times.
+
+**Cause**: Form has a `disabled` attribute on submit because a precondition isn't met (empty input, validation failure, loading state).
+
+**Diagnose**: read the page snapshot in `error-context.md` — the button's class string usually reveals which form (`bg-blue-600` = login, `bg-indigo-600` = setup wizard, etc.).
+
+**Fix**: usually a selector or precondition issue in the spec, not a stack issue.
+
+### E. Auth setup passes but smoke specs land on /login
+
+**Symptom**: `setup` ✓ but downstream specs fail asserting on dashboard nav links.
+
+**Cause**: `e2e/.auth/admin.json` (storageState) is empty (36 bytes). The auth setup saved state before `localStorage.setItem('token', ...)` completed.
+
+**Diagnose**:
+```bash
+cat src/frontend/e2e/.auth/admin.json | python3 -c "
+import json, sys
+d = json.load(sys.stdin)
+print('cookies:', len(d.get('cookies',[])))
+for o in d.get('origins',[]):
+ print(' localStorage entries:', len(o.get('localStorage',[])))
+"
+```
+
+**Fix**: `auth.setup.js` must `expect.poll(() => page.evaluate(() => localStorage.getItem('token'))).not.toBeNull()` before `storageState({ path: ... })`. If you're on a branch where this isn't yet fixed, cherry-pick from PR #579.
+
+### F. Visual regression baseline mismatch
+
+**Symptom**: `expected ... received ...` with a screenshot diff under `e2e/test-results//-diff.png`.
+
+**Causes**:
+1. Intentional UI change — run `--update-snapshots` and commit the new PNGs
+2. Unintentional regression — review the diff image; fix the code
+
+**Recover (intentional)**:
+```bash
+ADMIN_PASSWORD=$ADMIN_PASSWORD npm run test:e2e:update
+git add src/frontend/e2e/**/*-snapshots/
+```
+
+## Coverage today
+
+What the suite currently exercises:
+- `auth.setup.js` — admin login flow, JWT persistence
+- `smoke.spec.js` — top-nav rendering on `/` and that `/agents`, `/operating-room`, `/templates` load
+
+What's **not** covered (use this to scope new specs):
+- Settings, Health (`/monitoring`), Keys (`/api-keys`)
+- Any agent detail page (`/agents/:name`)
+- Form submission, modals, terminal, file manager
+- WebSocket events / real-time updates
+- Visual regression on the design system
+- Mobile / responsive viewports
+- Cross-browser (Chromium only by default)
+
+When adding a spec, prefer **visual regression** for any PR that touches design-system tokens — that's where the harness pays the highest dividend.
+
+## Output
+
+```
+## E2E Test Results
+
+**Status**: PASSED
+**Duration**: 5.1s
+**Specs**: 5 passed
+**Browser**: chromium
+
+### Report
+HTML: src/frontend/e2e/playwright-report/index.html
+Trace (on failure only): e2e/test-results//trace.zip
+```
+
+Or on failure:
+
+```
+## E2E Test Results
+
+**Status**: FAILED
+**Specs**: 1 failed, 4 did not run
+**Failure category**: B (stuck on /setup wizard)
+
+### Failed
+- [setup] authenticate as admin (auth.setup.js:8)
+ → Page is /setup wizard, not /login
+ → Run: docker exec trinity-backend python3 -c "from database import db; db.set_setting('setup_completed','true')"
+
+### Recovery
+Re-run after applying the fix above.
+```
+
+## Implementation notes
+
+- This skill does NOT spawn a sub-agent — it's a thin orchestration over `npm run test:e2e`. The diagnostic logic is small enough to live in the skill itself.
+- For the failure-classification logic, prefer reading `error-context.md` (Playwright auto-generates it on failure) over parsing CLI output. The page snapshot YAML is the most reliable signal for distinguishing failure modes.
+- This skill is the local complement to the `frontend-e2e.yml` CI workflow. CI runs the same `npm run test:e2e` against a fresh dockerized stack on a GitHub-hosted runner — see `docs/memory/feature-flows/...` for the deploy-to-dev story.
+- The CI workflow is path-gated by the `ui` label (not by file paths) — adding `ui` to a PR is the trigger. Mention this to users who run this skill locally before pushing.
diff --git a/.claude/skills/validate-architecture/SKILL.md b/.claude/skills/validate-architecture/SKILL.md
index e9b8b5b5e..42f62fcd6 100644
--- a/.claude/skills/validate-architecture/SKILL.md
+++ b/.claude/skills/validate-architecture/SKILL.md
@@ -130,6 +130,22 @@ architecture.md:L — Process Engine marked "OUT OF SCOPE" but routers/proces
Suggested edit: either remove the OUT OF SCOPE tag (if the module is in fact live), or remove the routers (if it is truly dormant).
```
+### Step 2c: Filter Stale Citations
+
+Before generating the report, validate every `file:line` citation produced in Steps 2a and 2b against the current working tree. The skill is sometimes run on a snapshot taken just before a large deletion PR lands; without this filter, the report (and any auto-created issue) cites paths that no longer exist.
+
+For each cited path (always quote `"$path"` — citations may contain spaces or shell metacharacters):
+
+```bash
+git ls-files --error-unmatch "$path" >/dev/null 2>&1 && echo exists || echo dropped
+```
+
+- **If `dropped`**: remove the citation from the violation. Do not retain "ghost" line numbers from a previous tree.
+- **After filtering**, if an invariant's violation list is empty, downgrade its status from `FAIL` to `PASS (after stale-citation filter)` and record the dropped citation count in the report so the reader can see what was removed.
+- **If the only violations cited paths that no longer exist**, do not propagate this invariant to Step 4's issue-creation trigger.
+
+This step exists because of issue #479: a 2026-04-24 run cited 11 paths that were deleted by commit e901108 (#430) the same day. None of those P1-critical citations were real on `main`, but the unfiltered report produced a `priority-p1` issue against `main`.
+
### Step 3: Generate Report
Output two sections:
@@ -162,9 +178,9 @@ Output two sections:
- architecture.md:L — "" marked out-of-scope but . Suggested edit: .
```
-### Step 4: Create Issue if Critical
+### Step 4: Create or Update Issue if Critical
-Create a GitHub issue when any of these fire:
+Create or update a GitHub issue when any of these fire (after the Step 2c stale-citation filter has run):
**P0-P1 invariants** (critical — break runtime or security):
- #1 Three-Layer Backend (layer violations cause maintenance debt)
@@ -176,19 +192,57 @@ Create a GitHub issue when any of these fire:
- D1 count mismatches with >25% divergence
- D2 any scope contradiction (dormant-but-live modules)
-If any fire, create issue:
+**Dedupe guard — required before any `gh issue create`:**
+
+Find any existing open Architecture Validation issue. If one exists, add findings as a comment; only create a new issue when none is open.
+
+```bash
+COMMIT_SHA=$(git rev-parse --short HEAD)
+
+# Find any open automated arch-validation issue
+EXISTING=$(gh issue list --repo abilityai/trinity \
+ --label "automated" --state open \
+ --search "\"Architecture Validation\" in:title" \
+ --json number --jq '.[0].number')
+```
+
+**Branch on `$EXISTING`. The two paths are mutually exclusive — execute exactly one.**
+
+**Path A — `$EXISTING` is non-empty (open issue found): COMMENT, then STOP.**
+
+```bash
+gh issue comment "$EXISTING" --repo abilityai/trinity --body "Re-run on \`$COMMIT_SHA\` ($(date -u +%Y-%m-%d)): [N] violations, [M] doc drift findings still present.
+
+### Updated Critical Invariant Violations (P0-P1)
+
+[List each P0-P1 violation with invariant number, file:line, description — Step 2c-filtered, no stale paths]
+
+### Updated Doc Drift — Suggested architecture.md Edits
+
+[List each suggested edit with line number and replacement text]
+
+---
+*Generated by scheduled /validate-architecture run*"
+```
+
+After commenting, **DO NOT** execute Path B. The skill workflow ends here for this run.
+
+**Path B — `$EXISTING` is empty (no open issue found): CREATE a new issue.**
+
+Only run this block when Path A did not run.
```bash
gh issue create \
--repo abilityai/trinity \
- --title "Architecture drift: [N] violations, [M] doc drift findings" \
+ --title "Architecture Validation: [N] critical violations found ($(date -u +%Y-%m-%d))" \
--body "## Automated Architecture Validation Report
**Date**: $(date -u +%Y-%m-%d)
+**Commit**: $COMMIT_SHA
### Critical Invariant Violations (P0-P1)
-[List each P0-P1 violation with invariant number, file:line, description]
+[List each P0-P1 violation with invariant number, file:line, description — Step 2c-filtered, no stale paths]
### Doc Drift — Suggested architecture.md Edits
@@ -198,12 +252,19 @@ gh issue create \
1. [Prioritized fix for each finding]
+### Tracking Notes
+
+- Future runs will comment on this issue rather than open a new one.
+- Close this issue once all cited invariants pass on \`main\`.
+
---
*Generated by scheduled /validate-architecture run*" \
--label "type-bug,priority-p1,automated"
```
-If nothing critical fires, skip issue creation — report only logged to execution history.
+**Concurrency caveat**: best-effort, not atomic. Two simultaneous runners could both create issues; the next run will comment on the first one found. Close any duplicate manually.
+
+If nothing critical fires after Step 2c's filter, skip issue creation — report only logged to execution history. **Do not create an issue solely on pre-filter results.**
## Outputs
diff --git a/.claude/skills/validate-config/SKILL.md b/.claude/skills/validate-config/SKILL.md
index 45ed43221..f05ec5b7b 100644
--- a/.claude/skills/validate-config/SKILL.md
+++ b/.claude/skills/validate-config/SKILL.md
@@ -133,9 +133,9 @@ Output a summary:
**Result: X issues found (Y must-fix, Z informational)**
```
-### Step 9: Create Issue if Critical
+### Step 9: Create or Update Issue if Critical
-If any critical (P0-P1) config issues were found, create a GitHub issue.
+If any critical (P0-P1) config issues were found, create or update a GitHub issue.
**P0-P1 criteria** (critical — cause startup failures or security issues):
- Docker-compose `${VAR}` with no .env.example entry (deployment fails)
@@ -145,12 +145,46 @@ If any critical (P0-P1) config issues were found, create a GitHub issue.
**Check**: Count "must-fix" issues from the report.
-**If must-fix issues > 0**, create issue:
+**If must-fix issues > 0**, find or create a tracking issue:
+
+**Dedupe guard — check for an existing open Config Validation issue before creating:**
+
+```bash
+EXISTING=$(gh issue list --repo abilityai/trinity \
+ --label "automated" --state open \
+ --search "\"Config Validation\" in:title" \
+ --json number --jq '.[0].number')
+```
+
+**Branch on `$EXISTING`. The two paths are mutually exclusive — execute exactly one.**
+
+**Path A — `$EXISTING` is non-empty (open issue found): COMMENT, then STOP.**
+
+```bash
+gh issue comment "$EXISTING" --repo abilityai/trinity --body "Re-run on $(date -u +%Y-%m-%d): [N] critical config issues still present.
+
+### Updated Critical Findings (P0-P1)
+
+[List each must-fix finding with var name, where it's used, and what's missing]
+
+### Recommended Actions
+
+1. [Prioritized fix — typically add to .env.example or remove dead config]
+
+---
+*Generated by scheduled /validate-config run*"
+```
+
+After commenting, **DO NOT** execute Path B. The skill workflow ends here for this run.
+
+**Path B — `$EXISTING` is empty (no open issue found): CREATE a new issue.**
+
+Only run this block when Path A did not run.
```bash
gh issue create \
--repo abilityai/trinity \
- --title "Config Validation: [N] critical config issues found" \
+ --title "Config Validation: [N] critical config issues found ($(date -u +%Y-%m-%d))" \
--body "## Automated Config Validation Report
**Date**: $(date -u +%Y-%m-%d)
@@ -164,11 +198,18 @@ gh issue create \
1. [Prioritized fix — typically add to .env.example or remove dead config]
+### Tracking Notes
+
+- Future runs will comment on this issue rather than open a new one.
+- Close this issue once all critical config issues are resolved.
+
---
*Generated by scheduled /validate-config run*" \
--label "type-bug,priority-p1,automated"
```
+**Concurrency caveat**: best-effort, not atomic. Two simultaneous runners could both create issues; the next run will comment on the first one found. Close any duplicate manually.
+
**If no must-fix issues** (only informational like unused vars), skip issue creation — report only.
## Outputs
diff --git a/.claude/skills/validate-pr/SKILL.md b/.claude/skills/validate-pr/SKILL.md
index 2487f8d58..973ad220f 100644
--- a/.claude/skills/validate-pr/SKILL.md
+++ b/.claude/skills/validate-pr/SKILL.md
@@ -29,6 +29,19 @@ Validate a pull request against the Trinity development methodology and generate
## Process
+### Quick Triage (30 seconds)
+
+Before deep validation, check the basics:
+1. PR has an issue link (`Fixes #N` or `Closes #N` in title or body)
+2. Priority label on the linked issue — P0/P1 get closer scrutiny
+3. PR size — if 50+ files changed, flag as potentially needing a split
+
+```bash
+gh pr view $PR_NUMBER --json title,body,labels | jq '{title: .title, body: .body[:300]}'
+```
+
+If the PR has no issue link, flag as ❌ CRITICAL immediately.
+
### Step 1: Fetch PR Information
```bash
@@ -51,6 +64,14 @@ Store this information for validation:
- Base and head branches
- Author
+#### 1.1 Base Branch Check
+Verify `baseRefName` is `dev` (unless this is a release-cut PR from `dev` → `main`):
+- [ ] PR targets `dev` (not `main` directly, unless it's a release PR)
+- If targeting `main`: flag as ❌ CRITICAL unless PR title/body indicates a release cut
+
+#### 1.2 PR Size Check
+- If `changedFiles >= 50`: flag as ⚠️ WARNING — "Large PR, consider splitting"
+
### Step 2: Validate Documentation Updates
#### 2.1 Commit Messages (REQUIRED)
@@ -72,7 +93,7 @@ Check if PR references a GitHub Issue (e.g., "Closes #17", "Fixes #23").
**Validation**:
- [ ] PR references related issue(s) in description
- [ ] Issue has correct priority label (priority-p0/p1/p2/p3)
-- [ ] Issue has correct type label (type-feature/bug/refactor)
+- [ ] Issue has correct type label (type-feature/bug/refactor/docs)
#### 2.3 Requirements Update (CONDITIONAL)
Check if `docs/memory/requirements.md` is in the changed files list.
@@ -227,6 +248,8 @@ Create the report in this format:
| Category | Status | Notes |
|----------|--------|-------|
| Commit Messages | ✅/❌ | [details] |
+| Base Branch | ✅/❌ | targets dev (or release cut to main) |
+| PR Size | ✅/⚠️ | [file count] |
| Roadmap | ✅/❌/➖ | [details or N/A] |
| Requirements | ✅/❌/➖ | [details or N/A] |
| Architecture | ✅/❌/➖ | [details or N/A] |
@@ -305,10 +328,20 @@ Please address these items and request re-review.
## Related
-- `docs/DEVELOPMENT_WORKFLOW.md` - Development cycle
+- `docs/DEVELOPMENT_WORKFLOW.md` - Development cycle and reviewer pipeline
- `docs/memory/feature-flows.md` - Feature flow index
-- `.claude/agents/feature-flow-analyzer.md` - Flow format specification
-- `/security-check` - Security validation details
+- `/review` - Complementary code review (SQL safety, race conditions, auth, scope drift, test gaps)
+- `/cso --diff` - Deep security audit (required for P0/P1 features)
+
+### Review Pipeline by PR Type
+
+| PR Type | `/review` | `/validate-pr` | `/cso --diff` |
+|---------|-----------|----------------|----------------|
+| Feature (P0/P1) | Required | Required | Required |
+| Feature (P2/P3) | Required | Required | Recommended |
+| Bug fix | Required | Required | Skip (unless auth/security) |
+| Refactor | Required | Required | Skip |
+| Docs only | Skip | Required | Skip |
## Completion Checklist
diff --git a/.claude/skills/validate-schema/SKILL.md b/.claude/skills/validate-schema/SKILL.md
index 3bef6a6b6..ec43607ac 100644
--- a/.claude/skills/validate-schema/SKILL.md
+++ b/.claude/skills/validate-schema/SKILL.md
@@ -104,9 +104,9 @@ Output a summary:
**Result: X issues found (Y critical, Z informational)**
```
-### Step 8: Create Issue if Critical
+### Step 8: Create or Update Issue if Critical
-If any critical (P0-P1) drift was found, create a GitHub issue.
+If any critical (P0-P1) drift was found, create or update a GitHub issue.
**P0-P1 criteria** (critical — cause runtime errors or data loss):
- Tables in schema.py with no migration (new deployments fail)
@@ -116,12 +116,46 @@ If any critical (P0-P1) drift was found, create a GitHub issue.
**Check**: Count critical issues from the report.
-**If critical issues > 0**, create issue:
+**If critical issues > 0**, find or create a tracking issue:
+
+**Dedupe guard — check for an existing open Schema Validation issue before creating:**
+
+```bash
+EXISTING=$(gh issue list --repo abilityai/trinity \
+ --label "automated" --state open \
+ --search "\"Schema Validation\" in:title" \
+ --json number --jq '.[0].number')
+```
+
+**Branch on `$EXISTING`. The two paths are mutually exclusive — execute exactly one.**
+
+**Path A — `$EXISTING` is non-empty (open issue found): COMMENT, then STOP.**
+
+```bash
+gh issue comment "$EXISTING" --repo abilityai/trinity --body "Re-run on $(date -u +%Y-%m-%d): [N] critical schema drift issues still present.
+
+### Updated Critical Findings (P0-P1)
+
+[List each critical finding with table name, issue type, and specific mismatch]
+
+### Recommended Actions
+
+1. [Prioritized fix for each — typically add migration or update schema.py]
+
+---
+*Generated by scheduled /validate-schema run*"
+```
+
+After commenting, **DO NOT** execute Path B. The skill workflow ends here for this run.
+
+**Path B — `$EXISTING` is empty (no open issue found): CREATE a new issue.**
+
+Only run this block when Path A did not run.
```bash
gh issue create \
--repo abilityai/trinity \
- --title "Schema Validation: [N] critical drift issues found" \
+ --title "Schema Validation: [N] critical drift issues found ($(date -u +%Y-%m-%d))" \
--body "## Automated Schema Validation Report
**Date**: $(date -u +%Y-%m-%d)
@@ -135,11 +169,18 @@ gh issue create \
1. [Prioritized fix for each — typically add migration or update schema.py]
+### Tracking Notes
+
+- Future runs will comment on this issue rather than open a new one.
+- Close this issue once all critical schema drift issues are resolved.
+
---
*Generated by scheduled /validate-schema run*" \
--label "type-bug,priority-p1,automated"
```
+**Concurrency caveat**: best-effort, not atomic. Two simultaneous runners could both create issues; the next run will comment on the first one found. Close any duplicate manually.
+
**If no critical issues** (only informational like doc drift), skip issue creation — report only.
## Outputs
diff --git a/.env.example b/.env.example
index d791f2471..d320ca7b5 100644
--- a/.env.example
+++ b/.env.example
@@ -45,6 +45,16 @@ EMAIL_PROVIDER=console
# Get from: https://resend.com/api-keys
RESEND_API_KEY=
+# SendGrid API (when EMAIL_PROVIDER=sendgrid)
+# Get from: https://app.sendgrid.com/settings/api_keys
+SENDGRID_API_KEY=
+
+# SMTP provider (when EMAIL_PROVIDER=smtp)
+SMTP_HOST=
+SMTP_PORT=587
+SMTP_USER=
+SMTP_PASSWORD=
+
# From address for verification emails
SMTP_FROM=noreply@your-domain.com
@@ -52,7 +62,7 @@ SMTP_FROM=noreply@your-domain.com
# CORS ORIGINS
# ===========================================
-# Additional CORS origins (comma-separated)
+# Additional CORS origins (comma-separated) [PROD: wire through docker-compose.prod.yml]
# Add your production domains here
EXTRA_CORS_ORIGINS=https://your-domain.com,http://your-domain.com
@@ -71,6 +81,7 @@ SLACK_CLIENT_ID=
SLACK_CLIENT_SECRET=
# Slack Signing Secret (for verifying Slack webhook requests)
# Get from: https://api.slack.com/apps → Basic Information → Signing Secret
+# [PROD: also set in docker-compose.prod.yml environment block]
SLACK_SIGNING_SECRET=
# GitHub OAuth (for GitHub MCP)
@@ -83,9 +94,10 @@ GITHUB_CLIENT_SECRET=
# This will be auto-uploaded to Redis on startup for local development
GITHUB_PAT=
-# Self-hosted git support (#387)
+# Self-hosted git support (#387) [OVERLAY: forwarded by docker-compose.gitea.yml only]
# Optional overrides — default to github.com / api.github.com (standard GitHub).
# Set both when targeting GitHub Enterprise Server, Gitea, or a dev harness.
+# To activate: use `docker compose -f docker-compose.yml -f docker-compose.gitea.yml up -d`
# TRINITY_GIT_BASE_URL=https://git.example.com
# TRINITY_GIT_API_BASE=https://git.example.com/api/v1
@@ -108,9 +120,14 @@ GEMINI_API_KEY=
# ===========================================
REDIS_URL=redis://redis:6379
-AUDIT_URL=http://audit-logger:8001
BACKEND_URL=http://localhost:8000
+# Public-facing frontend URL — REQUIRED in production
+# Used for: OAuth post-auth redirects (Slack, etc.), SSH host auto-detection,
+# public chat link generation
+# Example: https://trinity.your-domain.com
+FRONTEND_URL=
+
# ===========================================
# REDIS SECURITY (Optional but recommended for production)
# ===========================================
@@ -124,14 +141,18 @@ REDIS_PASSWORD=
# PUBLIC ACCESS CONFIGURATION (Optional)
# ===========================================
-# External URL for public chat links (PUB-002)
+# External URL for public chat links (PUB-002) [PROD: also set in docker-compose.prod.yml]
# Set this when you want to share public agent links with users outside VPN
-# This is the public-facing domain (e.g., https://public.abilityai.dev)
+# This is the public-facing domain (e.g., https://public.your-domain.com)
# Used by: public chat links, Telegram webhooks, Slack OAuth, Nevermined payments
# When set, enables "Copy External Link" button in PublicLinksPanel
# Leave empty if all users have VPN access
PUBLIC_CHAT_URL=
+# Frontend URL (for email links, OAuth callbacks) [PROD: set in docker-compose.prod.yml]
+# Leave empty to auto-detect from request headers
+FRONTEND_URL=
+
# ===========================================
# CLOUDFLARE TUNNEL (Optional - public access)
# ===========================================
@@ -164,6 +185,7 @@ TUNNEL_TOKEN=
# ===========================================
# SSH host for agent SSH access (MCP tool get_agent_ssh_access)
+# [DEV: not forwarded by docker-compose.yml; add manually if needed]
# Auto-detected from FRONTEND_URL domain in production, or:
# - Set explicitly for custom setups (e.g., Tailscale IP, public IP)
# - Leave empty to use auto-detection
diff --git a/.github/workflows/deploy-dev.yml b/.github/workflows/deploy-dev.yml
new file mode 100644
index 000000000..1418b09b0
--- /dev/null
+++ b/.github/workflows/deploy-dev.yml
@@ -0,0 +1,51 @@
+name: Deploy to Dev
+
+on:
+ push:
+ branches: [dev]
+
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+
+ steps:
+ - name: Connect to Tailscale
+ uses: tailscale/github-action@v2
+ with:
+ authkey: ${{ secrets.TAILSCALE_AUTH_KEY }}
+
+ - name: Deploy
+ uses: appleboy/ssh-action@v1
+ with:
+ host: 100.96.144.19
+ username: trinity
+ key: ${{ secrets.DEV_SSH_KEY }}
+ command_timeout: 25m
+ script: |
+ set -e
+ cd ~/trinity
+
+ echo "=== Pull ==="
+ git fetch origin dev
+ git checkout dev
+ git pull origin dev
+ echo "Version: $(git log -1 --oneline)"
+
+ echo "=== Build ==="
+ sudo docker compose -f docker-compose.prod.yml build --no-cache backend frontend mcp-server scheduler
+
+ echo "=== Restart ==="
+ sudo docker compose -f docker-compose.prod.yml up -d backend frontend mcp-server scheduler
+
+ echo "=== Health ==="
+ sleep 10
+ curl -sf http://localhost:8000/health && echo "Backend: OK"
+ SCHED=$(sudo docker inspect trinity-scheduler --format='{{.State.Health.Status}}')
+ echo "Scheduler: $SCHED"
+ [ "$SCHED" = "healthy" ] || echo "WARNING: scheduler not healthy yet"
+
+ echo "=== Error check ==="
+ sudo docker logs trinity-backend --tail 50 2>&1 | grep -iE 'error|exception|failed' | head -10 || echo "No errors"
+
+ echo "=== Done ==="
diff --git a/.github/workflows/frontend-build.yml b/.github/workflows/frontend-build.yml
new file mode 100644
index 000000000..79214ac00
--- /dev/null
+++ b/.github/workflows/frontend-build.yml
@@ -0,0 +1,35 @@
+name: frontend-build
+
+on:
+ pull_request:
+ paths:
+ - 'src/frontend/**'
+ - '.github/workflows/frontend-build.yml'
+ push:
+ branches: [main, dev]
+ paths:
+ - 'src/frontend/**'
+ - '.github/workflows/frontend-build.yml'
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ working-directory: src/frontend
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: actions/setup-node@v4
+ with:
+ node-version: '20'
+ cache: 'npm'
+ cache-dependency-path: src/frontend/package-lock.json
+
+ - run: npm ci
+
+ - name: Verify design tokens
+ run: npm run check:tokens
+
+ - name: Build
+ run: npm run build
diff --git a/.github/workflows/frontend-e2e.yml b/.github/workflows/frontend-e2e.yml
new file mode 100644
index 000000000..cb6bae316
--- /dev/null
+++ b/.github/workflows/frontend-e2e.yml
@@ -0,0 +1,106 @@
+name: frontend-e2e
+
+# Runs only when a PR carries the `ui` label. Adds a browser test layer for
+# UI changes; gated to avoid spending ~5 min of CI on every backend PR.
+# Trigger manually by adding the `ui` label to a PR.
+
+on:
+ pull_request:
+ types: [opened, synchronize, reopened, labeled]
+
+jobs:
+ e2e:
+ if: contains(github.event.pull_request.labels.*.name, 'ui')
+ runs-on: ubuntu-latest
+ timeout-minutes: 25
+ defaults:
+ run:
+ working-directory: src/frontend
+
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: actions/setup-node@v4
+ with:
+ node-version: '20'
+ cache: 'npm'
+ cache-dependency-path: src/frontend/package-lock.json
+
+ - name: Install npm dependencies
+ run: npm ci
+
+ - name: Cache Playwright browsers
+ uses: actions/cache@v4
+ with:
+ path: ~/.cache/ms-playwright
+ key: playwright-${{ runner.os }}-${{ hashFiles('src/frontend/package-lock.json') }}
+
+ - name: Install Playwright browser
+ run: npx playwright install --with-deps chromium
+
+ - name: Start Trinity stack
+ working-directory: ${{ github.workspace }}
+ env:
+ # Fallback password meets OWASP ASVS 2.1 complexity (upper/lower/digit/special)
+ # so the backend creates the admin user without warnings.
+ ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD || 'CiTestPassword!1' }}
+ ANTHROPIC_API_KEY: ${{ secrets.E2E_ANTHROPIC_API_KEY || 'placeholder' }}
+ run: |
+ # Generate the minimum .env needed to boot. Real secrets are not
+ # required for e2e — login uses ADMIN_PASSWORD only.
+ cp .env.example .env
+ echo "ADMIN_PASSWORD=$ADMIN_PASSWORD" >> .env
+ echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" >> .env
+ echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env
+ ./scripts/deploy/start.sh
+
+ - name: Wait for backend health
+ run: |
+ for i in {1..60}; do
+ if curl -fsS http://localhost:8000/health >/dev/null 2>&1; then
+ echo "backend healthy"; exit 0
+ fi
+ sleep 2
+ done
+ echo "backend never became healthy"; exit 1
+
+ - name: Skip first-time setup wizard
+ working-directory: ${{ github.workspace }}
+ run: |
+ # On a fresh DB the admin user is bootstrapped from ADMIN_PASSWORD,
+ # but `setup_completed` stays false (the backfill migration runs
+ # before admin creation). Without this flag the frontend redirects
+ # every route to /setup. Mark setup complete directly.
+ docker exec trinity-backend python3 -c "from database import db; db.set_setting('setup_completed', 'true'); print('setup_completed=true')"
+
+ - name: Run Playwright smoke tests
+ env:
+ ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD || 'CiTestPassword!1' }}
+ E2E_BASE_URL: http://localhost
+ # Only @smoke-tagged specs run in CI. @visual / @interactive specs
+ # are local-only until their flake/baseline story is sorted (#596).
+ run: npm run test:e2e:smoke
+
+ - name: Collect Trinity logs on failure
+ if: failure()
+ working-directory: ${{ github.workspace }}
+ run: |
+ docker compose logs --no-color --tail=200 backend frontend mcp-server > trinity-logs.txt || true
+
+ - name: Upload Playwright report
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: playwright-report
+ path: src/frontend/e2e/playwright-report/
+ retention-days: 14
+
+ - name: Upload failure artifacts
+ if: failure()
+ uses: actions/upload-artifact@v4
+ with:
+ name: e2e-failure-artifacts
+ path: |
+ src/frontend/e2e/test-results/
+ trinity-logs.txt
+ retention-days: 14
diff --git a/README.md b/README.md
index 7ffe1aaed..b30aebb67 100644
--- a/README.md
+++ b/README.md
@@ -45,6 +45,8 @@
| **Build Custom** | 6-12 months, $500K+ engineering | Deploy in minutes |
| **Frameworks** | No observability, no fleet management | Real-time monitoring, scheduling, audit trails |
+> **Security**: Trinity received a **Grade A (Excellent)** in an independent web application penetration test by [UnderDefense](https://underdefense.com) (April 2026). All critical and high findings from the initial assessment were fully remediated. See the [attestation letter](docs/security/UnderDefense-Web-Pentest-Attestation-Apr-2026.pdf).
+
---
## Getting Started — Deploy an Agent in 3 Minutes
@@ -614,6 +616,7 @@ EMAIL_PROVIDER=console # Use 'resend' or 'smtp' for production
- [Testing Guide](docs/TESTING_GUIDE.md) — Testing approach and standards
- [Contributing Guide](CONTRIBUTING.md) — How to contribute (PRs, code standards)
- [Known Issues](docs/KNOWN_ISSUES.md) — Current limitations and workarounds
+- [Security Attestation](docs/security/UnderDefense-Web-Pentest-Attestation-Apr-2026.pdf) — Web pentest by UnderDefense (Apr 2026, Grade A — Excellent)
## Development
diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml
index eca89201a..9af35ef12 100644
--- a/docker-compose.prod.yml
+++ b/docker-compose.prod.yml
@@ -40,8 +40,6 @@ services:
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-} # For Gemini-powered agents
- GEMINI_API_KEY=${GEMINI_API_KEY:-} # For platform image generation (IMG-001)
- GITHUB_PAT=${GITHUB_PAT}
- - AUTH0_DOMAIN=${AUTH0_DOMAIN}
- - AUTH0_ALLOWED_DOMAIN=${AUTH0_ALLOWED_DOMAIN:-ability.ai}
# Host paths for volumes (used when creating agent containers)
- HOST_TEMPLATES_PATH=${HOST_TEMPLATES_PATH:-${PWD}/config/agent-templates}
# OpenTelemetry Configuration (Optional)
@@ -94,9 +92,6 @@ services:
dockerfile: ../../docker/frontend/Dockerfile.prod
args:
- VITE_API_URL=${VITE_API_URL:-http://localhost:8000}
- - VITE_AUTH0_DOMAIN=${AUTH0_DOMAIN:-dev-10tz4lo7hcoijxav.us.auth0.com}
- - VITE_AUTH0_CLIENT_ID=${VITE_AUTH0_CLIENT_ID:-bFeIEm4WAwaalgSnxsfS1V6vd4gOk0li}
- - VITE_AUTH0_ALLOWED_DOMAIN=${AUTH0_ALLOWED_DOMAIN:-ability.ai}
image: trinity-frontend:prod
container_name: trinity-frontend
restart: unless-stopped
diff --git a/docker-compose.yml b/docker-compose.yml
index 60ba2a967..5aab34f6b 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -41,6 +41,15 @@ services:
- CREDENTIAL_ENCRYPTION_KEY=${CREDENTIAL_ENCRYPTION_KEY:-}
# Internal API shared secret (C-003) - for scheduler/agent communication
- INTERNAL_API_SECRET=${INTERNAL_API_SECRET:-}
+ # Public access / OAuth callbacks
+ - PUBLIC_CHAT_URL=${PUBLIC_CHAT_URL:-}
+ - FRONTEND_URL=${FRONTEND_URL:-}
+ # CORS (comma-separated extra origins)
+ - EXTRA_CORS_ORIGINS=${EXTRA_CORS_ORIGINS:-}
+ # Slack channel adapter
+ - SLACK_SIGNING_SECRET=${SLACK_SIGNING_SECRET:-}
+ # SSH access host override
+ - SSH_HOST=${SSH_HOST:-}
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./src/backend:/app
diff --git a/docker/base-image/agent_server/models.py b/docker/base-image/agent_server/models.py
index a93dcad74..af5c2a285 100644
--- a/docker/base-image/agent_server/models.py
+++ b/docker/base-image/agent_server/models.py
@@ -206,6 +206,7 @@ class ParallelTaskRequest(BaseModel):
max_turns: Optional[int] = None # Maximum agentic turns (--max-turns) for runaway prevention
execution_id: Optional[str] = None # Database execution ID (used for process registry if provided)
resume_session_id: Optional[str] = None # Claude Code session ID for --resume (EXEC-023)
+ images: Optional[List[Dict[str, str]]] = None # Vision images: [{"media_type": "image/jpeg", "data": ""}]
class ParallelTaskResponse(BaseModel):
diff --git a/docker/base-image/agent_server/routers/chat.py b/docker/base-image/agent_server/routers/chat.py
index fc290b63b..bb49e7fbd 100644
--- a/docker/base-image/agent_server/routers/chat.py
+++ b/docker/base-image/agent_server/routers/chat.py
@@ -128,7 +128,8 @@ async def execute_task(request: ParallelTaskRequest):
timeout_seconds=request.timeout_seconds or 900, # Default 15 minutes for research tasks
max_turns=request.max_turns,
execution_id=request.execution_id, # Use provided ID for process registry (enables termination)
- resume_session_id=request.resume_session_id # Resume previous session (EXEC-023)
+ resume_session_id=request.resume_session_id, # Resume previous session (EXEC-023)
+ images=request.images, # Vision images from channel adapters (#562)
)
logger.info(f"[Task] Task {session_id} completed successfully")
@@ -257,7 +258,10 @@ async def terminate_execution(execution_id: str):
This allows Claude Code to finish its current operation gracefully.
"""
registry = get_process_registry()
- result = registry.terminate(execution_id)
+ # registry.terminate() does up to 7s of synchronous process.wait() (SIGINT grace + SIGKILL grace);
+ # run in the default executor so the event loop stays responsive to concurrent /health probes.
+ loop = asyncio.get_running_loop()
+ result = await loop.run_in_executor(None, registry.terminate, execution_id)
if result["success"]:
logger.info(f"[Terminate] Execution {execution_id} terminated successfully")
diff --git a/docker/base-image/agent_server/routers/files.py b/docker/base-image/agent_server/routers/files.py
index b0c91fd2a..2dac59b87 100644
--- a/docker/base-image/agent_server/routers/files.py
+++ b/docker/base-image/agent_server/routers/files.py
@@ -164,14 +164,25 @@ async def download_file(path: str):
".mcp.json.template",
]
-# Paths that cannot be edited (subset of PROTECTED_PATHS)
-# CLAUDE.md and .mcp.json ARE editable since users need to modify them
+# Paths that cannot be edited via the file-write endpoint.
+#
+# .mcp.json and .mcp.json.template were historically editable here because
+# "users need to modify them" — but raw editing of either is RCE-by-config:
+# tool `command:` fields run as the agent process. Owners modify MCP servers
+# at agent-creation time via the template, or via the platform-internal
+# /api/credentials/update flow which regenerates .mcp.json from the template
+# with envsubst (no arbitrary content). See #590 (AISEC-C2).
+#
+# CLAUDE.md is intentionally NOT here — owners do edit their agent's
+# instructions directly.
EDIT_PROTECTED_PATHS = [
".trinity",
".git",
".gitignore",
".env",
+ ".mcp.json",
".mcp.json.template",
+ ".credentials.enc",
]
diff --git a/docker/base-image/agent_server/services/claude_code.py b/docker/base-image/agent_server/services/claude_code.py
index ac00df1f7..b5d0f9589 100644
--- a/docker/base-image/agent_server/services/claude_code.py
+++ b/docker/base-image/agent_server/services/claude_code.py
@@ -140,16 +140,18 @@ async def execute_headless(
timeout_seconds: int = 900,
max_turns: Optional[int] = None,
execution_id: Optional[str] = None,
- resume_session_id: Optional[str] = None
+ resume_session_id: Optional[str] = None,
+ images: Optional[List[Dict]] = None,
) -> Tuple[str, List[ExecutionLogEntry], ExecutionMetadata, str]:
"""Execute Claude Code in headless mode for parallel tasks.
Args:
resume_session_id: Optional session ID to resume (EXEC-023)
+ images: Optional list of vision images: [{"media_type": str, "data": base64_str}] (#562)
"""
return await execute_headless_task(
prompt, model, allowed_tools, system_prompt, timeout_seconds,
- max_turns, execution_id, resume_session_id
+ max_turns, execution_id, resume_session_id, images=images,
)
@@ -220,6 +222,8 @@ def parse_stream_json_output(output: str) -> tuple[str, List[ExecutionLogEntry],
message_content = msg.get("message", {}).get("content", [])
for content_block in message_content:
+ if not isinstance(content_block, dict):
+ continue # stream-json content arrays can contain plain strings
block_type = content_block.get("type")
if block_type == "tool_use":
@@ -397,6 +401,8 @@ def process_stream_line(line: str, execution_log: List[ExecutionLogEntry], metad
logger.debug(f"Processing {msg_type} message with {len(message_content)} content blocks")
for content_block in message_content:
+ if not isinstance(content_block, dict):
+ continue # stream-json content arrays can contain plain strings
block_type = content_block.get("type")
if block_type == "tool_use":
@@ -686,8 +692,13 @@ def read_subprocess_output():
f"[Chat] Outer timeout on session {execution_id} "
f"— killing process group as last resort"
)
- _terminate_process_group(process, graceful_timeout=2, pgid=process_pgid)
- _safe_close_pipes(process)
+ # _terminate_process_group does up to 4s of process.wait() (SIGTERM grace + SIGKILL grace);
+ # off-load to the executor so the event loop stays responsive while we tear down.
+ await loop.run_in_executor(
+ None,
+ lambda: _terminate_process_group(process, graceful_timeout=2, pgid=process_pgid),
+ )
+ await loop.run_in_executor(None, _safe_close_pipes, process)
raise HTTPException(
status_code=504,
detail=f"Chat execution timed out after {timeout_seconds} seconds"
@@ -935,6 +946,7 @@ def _classify_signal_exit(
def _classify_empty_result(
metadata: Optional['ExecutionMetadata'] = None,
raw_message_count: int = 0,
+ raw_messages: Optional[List[Dict]] = None,
) -> Optional[Tuple[int, str]]:
"""Classify a clean (return_code == 0) exit that produced no result message.
@@ -956,6 +968,10 @@ def _classify_empty_result(
nullability could be a Claude format quirk; both-None is a strong
signal that the terminal ``result`` message never arrived.
+ When the result line is lost, metadata.tool_count / num_turns are also
+ None (populated only by that line). Derive honest counts from
+ raw_messages when available so the 502 detail is accurate. (#531)
+
Returns ``(status_code, detail)`` for empty-result exits, or ``None``
if metadata looks well-formed (caller proceeds with the normal
response-building path).
@@ -965,8 +981,18 @@ def _classify_empty_result(
if metadata.cost_usd is not None or metadata.duration_ms is not None:
return None
+ # tool_count is accumulated per-message during parsing (line ~1467), so
+ # it's reliable even when the result line is lost. num_turns is populated
+ # only by the result line — fall back to counting assistant messages in
+ # raw_messages when it's None. (#531)
tool_count = metadata.tool_count or 0
- num_turns = metadata.num_turns or 0
+ if metadata.num_turns is not None:
+ num_turns = metadata.num_turns
+ elif raw_messages:
+ num_turns = sum(1 for m in raw_messages if m.get("type") == "assistant")
+ else:
+ num_turns = 0
+
detail = (
f"Execution completed without a result message after {tool_count} tool calls "
f"/ {num_turns} turns (raw_messages={raw_message_count}). "
@@ -992,7 +1018,8 @@ async def execute_headless_task(
timeout_seconds: int = 900,
max_turns: Optional[int] = None,
execution_id: Optional[str] = None,
- resume_session_id: Optional[str] = None
+ resume_session_id: Optional[str] = None,
+ images: Optional[List[Dict]] = None,
) -> tuple[str, List[ExecutionLogEntry], ExecutionMetadata, str]:
"""
Execute Claude Code in headless mode for parallel task execution.
@@ -1077,6 +1104,12 @@ async def execute_headless_task(
cmd.extend(["--disallowedTools", ",".join(disallowed_tools)])
logger.info(f"[Headless Task] Guardrails disallow tools: {disallowed_tools}")
+ # #562: when images are present, use stream-json stdin format so images
+ # are delivered as proper vision content blocks, not base64 text strings.
+ if images:
+ cmd.extend(["--input-format", "stream-json"])
+ logger.info(f"[Headless Task] {len(images)} image(s) — switching to stream-json input")
+
# Add system prompt if specified
if system_prompt:
cmd.extend(["--append-system-prompt", system_prompt])
@@ -1129,10 +1162,6 @@ async def execute_headless_task(
"pgid": process_pgid,
})
- # Write prompt to stdin and close it
- process.stdin.write(prompt)
- process.stdin.close()
-
# Issue #285: Event to signal auth failure detected in stderr
# When set, stdout loop should stop and process should be killed
auth_abort_event = threading.Event()
@@ -1239,14 +1268,46 @@ def _run_stdout():
pass
def read_subprocess_output_with_timeout():
- """Runs in thread pool. Waits for subprocess with bounded timeout,
- then drains reader threads (killing process-group stragglers if
- they hold pipes open — Issue #407)."""
+ """Runs in thread pool. Writes stdin, starts reader threads, and
+ waits for subprocess with bounded timeout, then drains reader
+ threads (killing process-group stragglers if they hold pipes
+ open — Issue #407).
+
+ Stdin is written here (not in the async coroutine) so that:
+ 1. Large payloads (e.g. base64 images) do not block the event loop.
+ 2. Reader threads are active before the write, preventing pipe-
+ buffer deadlock if claude writes stdout before stdin is closed.
+ """
stderr_thread = threading.Thread(target=read_stderr, daemon=True)
stdout_thread = threading.Thread(target=_run_stdout, daemon=True)
stderr_thread.start()
stdout_thread.start()
+ # Build and write stdin payload. For vision tasks use stream-json
+ # format so images arrive as proper content blocks (#562).
+ if images:
+ content_blocks: List[Dict] = [
+ {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": img["media_type"],
+ "data": img["data"],
+ },
+ }
+ for img in images
+ ]
+ content_blocks.append({"type": "text", "text": prompt})
+ stdin_payload = (
+ json.dumps({"type": "user", "message": {"role": "user", "content": content_blocks}})
+ + "\n"
+ )
+ else:
+ stdin_payload = prompt
+
+ process.stdin.write(stdin_payload)
+ process.stdin.close()
+
# Bounded wait on the subprocess itself. If claude hangs, we
# never wedge the executor thread for more than timeout_seconds.
try:
@@ -1293,8 +1354,13 @@ def read_subprocess_output_with_timeout():
f"[Headless Task] Outer timeout on task {task_session_id} "
f"— killing process group as last resort"
)
- _terminate_process_group(process, graceful_timeout=2, pgid=process_pgid)
- _safe_close_pipes(process)
+ # _terminate_process_group does up to 4s of process.wait() (SIGTERM grace + SIGKILL grace);
+ # off-load to the executor so the event loop stays responsive while we tear down.
+ await loop.run_in_executor(
+ None,
+ lambda: _terminate_process_group(process, graceful_timeout=2, pgid=process_pgid),
+ )
+ await loop.run_in_executor(None, _safe_close_pipes, process)
raise HTTPException(
status_code=504,
detail=f"Task execution timed out after {timeout_seconds} seconds"
@@ -1411,7 +1477,7 @@ def read_subprocess_output_with_timeout():
# the agent-server log misleadingly claims "completed successfully".
# Surface this as 502 so backend records it as FAILED with a useful
# diagnostic rather than dispatching an empty 200.
- empty_result = _classify_empty_result(metadata, raw_message_count=len(raw_messages))
+ empty_result = _classify_empty_result(metadata, raw_message_count=len(raw_messages), raw_messages=raw_messages)
if empty_result is not None:
status_code, detail = empty_result
logger.error(f"[Headless Task] {detail}")
diff --git a/docker/base-image/agent_server/services/gemini_runtime.py b/docker/base-image/agent_server/services/gemini_runtime.py
index 81152a65c..d1c6e354a 100644
--- a/docker/base-image/agent_server/services/gemini_runtime.py
+++ b/docker/base-image/agent_server/services/gemini_runtime.py
@@ -506,7 +506,8 @@ async def execute_headless(
timeout_seconds: int = 900,
max_turns: Optional[int] = None,
execution_id: Optional[str] = None,
- resume_session_id: Optional[str] = None
+ resume_session_id: Optional[str] = None,
+ images: Optional[List[Dict]] = None,
) -> Tuple[str, List[ExecutionLogEntry], ExecutionMetadata, str]:
"""
Execute Gemini CLI in headless mode for parallel tasks.
diff --git a/docker/base-image/agent_server/services/runtime_adapter.py b/docker/base-image/agent_server/services/runtime_adapter.py
index fb14baffd..4e7cce8e1 100644
--- a/docker/base-image/agent_server/services/runtime_adapter.py
+++ b/docker/base-image/agent_server/services/runtime_adapter.py
@@ -109,7 +109,8 @@ async def execute_headless(
timeout_seconds: int = 900,
max_turns: Optional[int] = None,
execution_id: Optional[str] = None,
- resume_session_id: Optional[str] = None
+ resume_session_id: Optional[str] = None,
+ images: Optional[List[Dict]] = None,
) -> Tuple[str, List[ExecutionLogEntry], ExecutionMetadata, str]:
"""
Execute a stateless task in headless mode (no conversation context).
diff --git a/docker/base-image/agent_server/utils/subprocess_pgroup.py b/docker/base-image/agent_server/utils/subprocess_pgroup.py
index b42d6c74e..e27949297 100644
--- a/docker/base-image/agent_server/utils/subprocess_pgroup.py
+++ b/docker/base-image/agent_server/utils/subprocess_pgroup.py
@@ -137,14 +137,23 @@ def drain_reader_threads(
process: subprocess.Popen,
*threads: Optional[threading.Thread],
grace: int = 5,
+ post_kill_grace: int = 30,
pgid: Optional[int] = None,
) -> None:
"""Join subprocess reader threads with a bounded timeout.
If any thread is still alive after ``grace`` seconds, a surviving
- process-group member is holding a pipe write end open. Kill the
- group (via ``pgid`` if provided, else looked up) and force-close
- our pipe FDs so readline() returns and threads exit.
+ process-group member is holding a pipe write end open. Kill the group
+ (via ``pgid`` if provided, else looked up), then wait up to
+ ``post_kill_grace`` seconds for the reader to drain naturally before
+ force-closing pipes as a last resort.
+
+ Ordering matters: once all writers (claude + hook grandchildren) are
+ dead the kernel will EOF the read end as soon as the buffered bytes are
+ consumed. If we close our read FD first the reader raises
+ ``ValueError: I/O operation on closed file`` and the kernel buffer —
+ including the final ``{"type":"result"}`` JSON line — is discarded
+ silently. Waiting for natural drain preserves that data. (#531)
Callers that have already reaped the parent via ``process.wait()``
must pass ``pgid`` — after reaping, the pid is gone and we'd
@@ -159,21 +168,46 @@ def drain_reader_threads(
return
logger.warning(
- "[Subprocess] Reader thread(s) stuck after process exit "
- "(pid=%s, stuck_count=%s) — killing process group and closing pipes to unwind",
- process.pid, len(stuck),
+ "[Subprocess] Reader thread(s) still busy after process exit "
+ "(pid=%s, stuck_count=%s) — killing process group, then waiting "
+ "%ss for natural drain",
+ process.pid, len(stuck), post_kill_grace,
)
terminate_process_group(process, graceful_timeout=1, pgid=pgid)
- safe_close_pipes(process)
+
+ # Grandchildren are gone → kernel will EOF the pipe as the last write
+ # FD is reaped → reader's readline() returns '' once it drains the
+ # buffered tail (including the final result JSON line on long tasks).
for t in stuck:
+ t.join(timeout=post_kill_grace)
+
+ still_stuck = [t for t in stuck if t.is_alive()]
+ if not still_stuck:
+ logger.info(
+ "[Subprocess] Reader thread(s) drained naturally after "
+ "grandchild termination (pid=%s)",
+ process.pid,
+ )
+ return
+
+ # Genuine wedge — reader did not return even after grandchildren died
+ # and the kernel should have EOF'd. Force-close and accept data loss.
+ logger.error(
+ "[Subprocess] Reader thread(s) still stuck after %ss post-kill "
+ "grace — force-closing pipes; some buffered data may be lost "
+ "(pid=%s, stuck_count=%s)",
+ post_kill_grace, process.pid, len(still_stuck),
+ )
+ safe_close_pipes(process)
+ for t in still_stuck:
t.join(timeout=2)
- still_alive = [t for t in stuck if t.is_alive()]
- if still_alive:
+ leaked = [t for t in still_stuck if t.is_alive()]
+ if leaked:
logger.error(
"[Subprocess] %s reader thread(s) leaked for pid=%s after "
- "close+killpg; continuing anyway",
- len(still_alive), process.pid,
+ "force-close; continuing anyway",
+ len(leaked), process.pid,
)
diff --git a/docker/frontend/Dockerfile.prod b/docker/frontend/Dockerfile.prod
index 66a329ff1..67a3a34ca 100644
--- a/docker/frontend/Dockerfile.prod
+++ b/docker/frontend/Dockerfile.prod
@@ -17,15 +17,9 @@ COPY . .
# Build arguments (injected at build time)
ARG VITE_API_URL=http://localhost:8000
-ARG VITE_AUTH0_DOMAIN=dev-10tz4lo7hcoijxav.us.auth0.com
-ARG VITE_AUTH0_CLIENT_ID=bFeIEm4WAwaalgSnxsfS1V6vd4gOk0li
-ARG VITE_AUTH0_ALLOWED_DOMAIN=ability.ai
# Set environment variables for Vite build
ENV VITE_API_URL=${VITE_API_URL}
-ENV VITE_AUTH0_DOMAIN=${VITE_AUTH0_DOMAIN}
-ENV VITE_AUTH0_CLIENT_ID=${VITE_AUTH0_CLIENT_ID}
-ENV VITE_AUTH0_ALLOWED_DOMAIN=${VITE_AUTH0_ALLOWED_DOMAIN}
# Build the application
RUN npm run build
diff --git a/docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md b/docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md
index b7c966bcc..caf268291 100644
--- a/docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md
+++ b/docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md
@@ -125,35 +125,50 @@ OTHER_VAR=value
### 5. `.gitignore` (Required)
-Must exclude secrets, instance-specific files, and large content:
+Must exclude secrets, instance-specific files, and large content.
+
+> **Source of truth**: this list mirrors `_GITIGNORE_PATTERNS` in
+> `src/backend/services/git_service.py`. The Python constant is the
+> source of truth — the platform appends every entry to the agent's
+> `.gitignore` on init and again on every Push. A unit test
+> (`tests/unit/test_github_init_gitignore.py::test_doc_and_constant_in_sync`)
+> asserts the two stay in sync.
```gitignore
+# Shell init / history (instance-specific)
+.bash_logout
+.bashrc
+.profile
+.bash_history
+.sudo_as_admin_successful
+
# Credentials - NEVER COMMIT
-.mcp.json
.env
+.env.*
+.mcp.json
+credentials.json
*.pem
*.key
-credentials.json
-
-# Large generated content - DO NOT COMMIT
-content/
# Instance-specific directories - DO NOT COMMIT
+.cache/
+.local/
.npm/
.ssh/
.trinity/
-.cache/
-# Instance-specific files - DO NOT COMMIT
-.claude.json
-.claude.json.backup
-.sudo_as_admin_successful
+# Large generated content - DO NOT COMMIT
+content/
# Claude Code - commit commands/skills/agents, exclude runtime data
+.claude.json
+.claude.json.backup
.claude/projects/
.claude/statsig/
.claude/todos/
.claude/debug/
+.claude/sessions/
+.claude/shell-snapshots/
# Keep: .claude/commands/, .claude/skills/, .claude/agents/, settings.local.json
# Temporary files
@@ -179,6 +194,8 @@ content/
- ❌ `.claude/statsig/` - Analytics
- ❌ `.claude/todos/` - Temporary todo lists
- ❌ `.claude/debug/` - Debug logs
+- ❌ `.claude/sessions/` - Per-session state
+- ❌ `.claude/shell-snapshots/` - Shell environment snapshots
**Note on Skills**: Skills in templates are seeded to the **Platform Skills Library** on first deployment, then managed centrally. See [Platform Skills](#platform-skills).
diff --git a/docs/drafts/amazing-file-outbound-phase1-execution.md b/docs/drafts/amazing-file-outbound-phase1-execution.md
new file mode 100644
index 000000000..1d3198831
--- /dev/null
+++ b/docs/drafts/amazing-file-outbound-phase1-execution.md
@@ -0,0 +1,69 @@
+# Amazing File Outbound — Phase 1 Execution Checklist
+
+**Date**: 2026-04-24
+**Branch**: `feature/295-files-outbound-sharing`
+**Parent commit**: `e47c688 feat(files): FILES-001 outbound file sharing MVP (Steps 1-6)`
+**Source plan**: [amazing-file-outbound-production-readiness.md](amazing-file-outbound-production-readiness.md) §3 CRITICAL
+
+This is the narrow "what to execute right now" checklist. Each item is fix-and-ship quality; no architectural debate.
+
+## Execution order (~2 hours total)
+
+- [ ] **C1 — Docs pass** (~45 min)
+ - `docs/memory/requirements.md` — mark FILES-001 Implemented with date + summary
+ - `docs/memory/architecture.md` — bump MCP tool count (62 → 73+), add `agent_shared_files` to schema section, add the 4 new endpoints to API table
+ - `docs/memory/feature-flows.md` — add index entry
+ - `docs/memory/feature-flows/file-sharing-outbound.md` — **new** full feature-flow doc (UI → store → router → service → DB → download)
+
+- [ ] **C2 — Filename length cap** (~2 min)
+ - `src/backend/models.py` — `Field(max_length=255)` on `ShareFileRequest.filename` and `ShareFileMcpRequest.filename`
+
+- [ ] **C3 — Disk-space pre-check** (~10 min)
+ - `src/backend/services/agent_shared_files_service.py` — `shutil.disk_usage` check before `open().write()`; reject with 507 Insufficient Storage if free space below threshold (default 500 MB configurable)
+
+- [ ] **C4 — Cleanup sweep (Step 7)** (~30 min)
+ - `src/backend/db/agent_shared_files.py` — new method `delete_expired_and_revoked() -> list[stored_filename]` (returns disk paths to unlink)
+ - `src/backend/database.py` — facade forward
+ - `src/backend/services/cleanup_service.py` — call on existing 5-min tick; unlink disk files; log summary per sweep
+
+- [ ] **C5 — Separate rate-limit bucket** (~15 min)
+ - `src/backend/routers/files.py` — replace `check_public_link_rate_limit` with new `check_file_download_rate_limit` helper using key `file_downloads:{ip}`
+ - Declared in same module or a small helpers file; same limits (30/min) but separate bucket so download traffic doesn't starve public chat's rate-limit quota
+
+- [ ] **C6 — HEAD handler** (~10 min)
+ - `src/backend/routers/files.py` — add `@router.head("/{file_id}")` that returns same headers as GET but no body; reuse the same auth + expiration + revocation checks
+
+- [ ] **C7 — Tighten list/revoke to owner+admin** (~5 min)
+ - `src/backend/routers/agent_files.py` — `GET /api/agents/{name}/shared-files` change `can_user_access_agent` → `can_user_share_agent`
+ - `DELETE .../shared-files/{file_id}` already uses `can_user_share_agent` — verify and no-op if so
+ - No API shape change; only access gate tightening
+
+- [ ] **C8 — Agent prompt nudge** (~10 min)
+ - Target file: `src/backend/services/platform_prompt_service.py` (or the system-wide prompt file — confirm during execution)
+ - Add 2 lines: "When sharing files with users, write the file to `/home/developer/public/` and call the `share_file` MCP tool to get a download URL. Return the URL as-is to the user."
+
+- [ ] **C9 — Close #295** (~2 min)
+ - `gh issue comment 295` with a link to this feature-flow doc + the merged PR
+ - `gh issue close 295`
+
+## Pre-execution step (required by user request)
+
+- [ ] Read updated project docs (requirements.md, architecture.md, feature-flows.md) via `read-docs` skill — main's recent commits touched these, need fresh context before touching them in C1.
+
+## Post-execution verification
+
+- [ ] All edited Python files parse (`ast.parse`)
+- [ ] `pytest tests/unit/test_agent_shared_files_migration.py tests/unit/test_file_sharing_mixin.py tests/unit/test_public_folder_mount_match.py` still green (33/33)
+- [ ] Backend + MCP + frontend still healthy after restart
+- [ ] Repeat the 37-scenario live regression from the earlier thorough audit
+- [ ] Verify agent filesystem delete + cleanup sweep both purge disk files
+- [ ] Verify HEAD returns same headers as GET with empty body
+- [ ] Verify shared user gets 403 on list/revoke
+
+## Commit strategy
+
+One commit per C-item, conventional message. Squash at PR-time. Branch stays `feature/295-files-outbound-sharing`. No push to remote until all 9 done + verified.
+
+## Out-of-scope (filed as GitHub issues after Phase 1)
+
+See [amazing-file-outbound-production-readiness.md](amazing-file-outbound-production-readiness.md) §4 for the full 12-item LATER list (G1–G12).
diff --git a/docs/drafts/amazing-file-outbound-production-readiness.md b/docs/drafts/amazing-file-outbound-production-readiness.md
new file mode 100644
index 000000000..428853244
--- /dev/null
+++ b/docs/drafts/amazing-file-outbound-production-readiness.md
@@ -0,0 +1,210 @@
+# Amazing File Outbound — Production Readiness Plan
+
+**Status**: Draft
+**Created**: 2026-04-24
+**Owner**: @pavshulin
+**Scope**: Take the MVP outbound file sharing feature (Steps 1–6 of [amazing-file-outbound.md](amazing-file-outbound.md)) from "working on local" to "safe to ship".
+
+> Feature is **live and verified end-to-end on real Slack traffic** (see final commit of Step 6). This doc is the punch-list to get it from MVP to v1.0.
+
+---
+
+## 1. Scope
+
+The MVP shipped Steps 1–6:
+
+1. Schema + migration (`agent_shared_files`)
+2. Per-agent opt-in toggle + publish volume
+3. Internal share endpoint with path / MIME / size / quota validation
+4. Public download endpoint with token auth + policy gate + audit
+5. `share_file` MCP tool with same-agent defense
+6. UI panel — toggle, list, revoke, copy URL
+
+Plus two bugs found during live test:
+- Agent-delete didn't cascade-delete rows/files/volume → **fixed** (added explicit cleanup in delete handler since SQLite FKs aren't enforced at runtime)
+- URL format used `/files/{id}` path which wasn't in the `/api/*` proxy allowlist on either Vite or prod nginx → **fixed** (switched to `/api/files/{id}?sig=...`; `download_token` alias kept for backward compat)
+- Credential sanitizer redacted `?download_token=...` query params from agent responses → **fixed** (renamed to `?sig=...` which is outside the sanitizer's sensitive-key patterns)
+
+Live regression: **37 of 37 assertions pass + 33/33 pytest unit tests green + real Slack round-trip observed**.
+
+---
+
+## 2. What we are NOT doing (deferred)
+
+| | Why |
+|---|-----|
+| One-time download links | Deferred in Step 3; schema columns retained for later |
+| Slack/Discord/Twitter unfurl-bot defense (crawler UA allowlist) | Only needed with one-time links |
+| Inbound file uploads (user → agent via URL) | Different feature (see #364) |
+| MCP tool count sync in architecture.md | Separate drift issue |
+| FK `PRAGMA foreign_keys=ON` platform-wide | Pre-existing platform pattern, separate issue (G11) |
+
+---
+
+## 3. Triage — CRITICAL (must fix before production)
+
+9 items, ~2 hours total. All fix-and-ship quality, no architectural debate.
+
+| # | Item | Why critical | Effort |
+|---|------|--------------|--------|
+| **C1** | Docs pass: `requirements.md` (mark FILES-001 Implemented), `architecture.md` (tool count 62→73, add `agent_shared_files` to schema section, add endpoints to API table), `feature-flows.md` index entry, new `feature-flows/file-sharing-outbound.md` | Trinity's Rules of Engagement §1 + §4 require requirements/docs updates for new features. Not ship-blocking in code but ship-blocking in SDLC. | 45 min |
+| **C2** | **Filename length cap** — `Field(max_length=255)` on `ShareFileRequest.filename` and `ShareFileMcpRequest.filename` | Prevents 10KB filename from agent or attacker. Trivial footgun removal. | 2 min |
+| **C3** | **Disk-space pre-check** before writing `/data/agent-files/{id}` | `/data` is shared with the SQLite DB. If disk fills, whole backend crashes — not just file sharing. Use `shutil.disk_usage` with a configurable min-free threshold (default 500 MB). | 10 min |
+| **C4** | **Step 7 cleanup sweep** in `cleanup_service.py` — purge expired + old-revoked shares on 5-min tick | Without it, one month of traffic = gigabytes of dead files. Disk pressure → C3 fire. Delete disk file first, then DB row. Audit a summary per sweep. | 30 min |
+| **C5** | **Separate rate-limit bucket** for `/api/files/*` | Current code shares the `public_link_lookups:{ip}` bucket with public chat. Heavy download traffic exhausts rate limit for every other public endpoint on that IP — real multi-tenant issue. | 15 min |
+| **C6** | **HEAD handler** on download endpoint | Some link-previewers probe with HEAD. Currently 405. Return same headers as GET but without body. | 10 min |
+| **C7** | **Tighten list/revoke endpoints** to owner + admin (currently any shared user can see URLs and revoke) | Access-model mismatch with `share_file` (owner-only). Shared user could harvest URLs or revoke owner's shares. Change `can_user_access_agent` → `can_user_share_agent` in both. | 5 min |
+| **C8** | **Agent prompt nudge** — add guidance about `/home/developer/public/` + `share_file` tool to the system-wide Trinity prompt (`platform_prompt_service.py`) | Every new agent will otherwise need the user to discover the tool. Near-zero-cost DX win. | 10 min |
+| **C9** | **Close #295** with comment linking to this implementation | Housekeeping — prevents duplicate work by anyone else picking up the backlog. | 2 min |
+
+### Things deliberately NOT in CRITICAL
+
+- **Multi-agent stress test** — verified live through Slack; V1 traffic won't stress this.
+- **pytest for Steps 3/4/5/6 endpoints** — critical path manually verified (37 assertions + live trace); file as P2 (G5).
+- **Memory streaming during extract** — 100 MB peak per share is fine at our concurrency; fix when we see the problem (G1).
+- **Directory sharding** — only matters past ~10k files (G2).
+- **OTel explicit spans** — Claude Code already emits OTel from agent side (G10).
+
+---
+
+## 4. Triage — LATER (GitHub issues)
+
+Each is scoped small enough for independent pickup.
+
+| # | Proposed title | Priority | Labels |
+|---|----------------|----------|--------|
+| **G1** | `refactor(file-sharing): stream tar extraction to cap peak memory at 64 KB per share` | P2 | type-refactor, performance |
+| **G2** | `perf(file-sharing): shard /data/agent-files/ by UUID prefix` | P3 | type-refactor, performance |
+| **G3** | `feat(file-sharing): add platform-wide storage quota with setting` | P2 | type-feature, security |
+| **G4** | `feat(audit): index file_share_download events by file_id for fast lookup` | P3 | type-feature |
+| **G5** | `test(file-sharing): pytest coverage for share / download / list / revoke endpoints` | P2 | type-test |
+| **G6** | `security(file-sharing): sanitize download URLs from stored chat_messages/schedule_executions to prevent token reuse` | P2 | security |
+| **G7** | `feat(file-sharing): add one-time download links with link-previewer UA defense` (deferred from MVP) | P3 | type-feature |
+| **G8** | `refactor(file-sharing): remove legacy ?download_token= alias once clients migrate` | P3 | type-refactor |
+| **G9** | `ui(file-sharing): pagination, download trend chart, clipboard fallback for non-HTTPS contexts` | P3 | type-feature, ui |
+| **G10** | `observability(file-sharing): OTel explicit span + metrics for shares/downloads` | P3 | type-feature |
+| **G11** | `bug(platform): enable PRAGMA foreign_keys=ON per SQLite connection` (platform-wide, affects all FK declarations) | P2 | type-bug, security |
+| **G12** | `refactor(models): unify ShareFileRequest (internal) + ShareFileMcpRequest (MCP) — near-duplicate` | P3 | type-refactor |
+
+---
+
+## 5. Initial audit — findings (self-review)
+
+### HIGH priority (addressed in CRITICAL above)
+
+| # | Finding | Location | Resolution |
+|---|---------|----------|------------|
+| H1 | `sig` value ends up in stored chat_messages/schedule_executions rows | download tokens live in persisted agent transcripts for 7 days | Known limitation; filed as G6 |
+| H2 | No HEAD handler → 405 on probes | `routers/files.py:79` | **C6** |
+| H3 | No disk-full guard before write | `services/agent_shared_files_service.py:264` | **C3** |
+| H4 | Filename length unbounded | `models.py` `ShareFileMcpRequest` / `ShareFileRequest` | **C2** |
+
+### MEDIUM priority
+
+| # | Finding | Resolution |
+|---|---------|------------|
+| M1 | Shared IP rate-limit bucket with public chat | **C5** |
+| M2 | List endpoint visible to shared users (not just owner) | **C7** |
+| M3 | Memory: full file into bytearray + extracted bytes | G1 |
+| M4 | Flat `/data/agent-files/` directory | G2 |
+| M5 | Audit event logs target_id but not file_id in searchable column | G4 |
+| M6 | No pytest for Steps 3/4/5/6 endpoints | G5 |
+
+### LOW priority
+
+| # | Finding | Resolution |
+|---|---------|------------|
+| L1 | `one_time` / `consumed_at` columns retained but unused | Documented in schema + design doc; addressed when one-time lands (G7) |
+| L2 | `ShareFileRequest` / `ShareFileMcpRequest` near-duplicate models | G12 |
+| L3 | Log line could be more structured for Vector | Cosmetic |
+| L4 | No concurrency test on 50MB shares | Monitor prod |
+| L5 | Legacy `?download_token=` alias | G8 |
+
+### Non-issues (verified safe by audit)
+
+- Constant-time token compare with `secrets.compare_digest`
+- Path-traversal rejection (absolute, `..`, `\`)
+- MIME blocklist for PE/ELF/Mach-O/shebang
+- Filesystem isolation — backend never mounts agent workspace; reads only via `docker get_archive` at agent-named paths
+- `Content-Disposition: attachment` prevents inline HTML XSS
+- FK `ON UPDATE/DELETE CASCADE` declared (defense-in-depth — not runtime-enforced due to platform pattern, but explicit cascade is in `rename_agent()` and delete handler)
+- Cleanup on agent delete — rows, on-disk files, volume (verified by live test)
+
+---
+
+## 6. Execution plan — today
+
+```
+Step 1 (5 min) Save this doc
+Step 2 (~2 hrs) Execute C1–C9 in order
+Step 3 (20 min) File G1–G12 as GitHub issues, link from this doc
+Step 4 (15 min) Re-run the 37-scenario regression to confirm no regressions from C2–C8
+Step 5 (5 min) Commit with conventional message; mark FILES-001 Implemented
+```
+
+---
+
+## 7. Section B — Inventory of everything that changed across Steps 1–6
+
+### New files
+| File | Approx lines | Purpose |
+|------|-------------|---------|
+| `src/backend/db/agent_shared_files.py` | 120 | DB ops class |
+| `src/backend/db/agent_settings/file_sharing.py` | 58 | Per-agent toggle mixin |
+| `src/backend/services/agent_service/file_sharing.py` | 130 | Toggle service + mount check |
+| `src/backend/services/agent_shared_files_service.py` | 280 | Share orchestrator (validate/extract/persist) |
+| `src/backend/routers/files.py` | 170 | Public download endpoint |
+| `src/mcp-server/src/tools/files.ts` | 130 | `share_file` MCP tool |
+| `src/frontend/src/components/FileSharingPanel.vue` | 210 | UI panel |
+| `tests/unit/test_agent_shared_files_migration.py` | 220 | Schema + migration tests (12 tests) |
+| `tests/unit/test_file_sharing_mixin.py` | 170 | DB mixin tests (12 tests) |
+| `tests/unit/test_public_folder_mount_match.py` | 140 | Mount match helper tests (9 tests) |
+| `docs/drafts/amazing-file-outbound.md` | 300+ | Design doc (canonical through Step 6) |
+| `docs/drafts/amazing-file-outbound-production-readiness.md` | *(this doc)* | Production readiness plan |
+
+### Modified files
+- `src/backend/db/schema.py` (table + column + 3 indexes)
+- `src/backend/db/migrations.py` (`_migrate_agent_shared_files`)
+- `src/backend/db/agent_settings/__init__.py`, `db/agents.py` (mixin registration)
+- `src/backend/db/agent_settings/metadata.py` (rename_agent cascade)
+- `src/backend/db/public_links.py` (`validate_agent_session` cross-link validator)
+- `src/backend/database.py` (facade forwards — ~10 methods)
+- `src/backend/services/agent_service/__init__.py`, `lifecycle.py`, `crud.py`, `helpers.py` (mount volume, start/stop flow)
+- `src/backend/services/docker_utils.py` (`container_get_archive`)
+- `src/backend/routers/agent_files.py` (toggle + share + list + revoke endpoints)
+- `src/backend/routers/internal.py` (internal share endpoint)
+- `src/backend/routers/agents.py` (delete handler cleanup)
+- `src/backend/routers/files.py` (download endpoint — new file too)
+- `src/backend/main.py` (register router)
+- `src/backend/models.py` (5 new Pydantic models)
+- `src/mcp-server/src/client.ts`, `server.ts` (MCP wiring)
+- `src/frontend/src/stores/agents.js` (4 new actions)
+- `src/frontend/src/components/SharingPanel.vue` (embed FileSharingPanel)
+- `tests/registry.json` (3 entries)
+- `.claude/settings.json` (permission allowlist — created)
+
+---
+
+## 8. Open decisions for review
+
+1. **Docs-first or code-first in Phase 1?** Plan has docs as C1 since SDLC requires. If you'd prefer shipping code, docs can move to end — happy to reorder.
+2. **Agent prompt nudge placement**: system-wide Trinity prompt (affects all agents, instant rollout) vs. per-agent CLAUDE.md (more discoverable but requires rebuilds). Proposed: system-wide.
+3. **Legacy `?download_token=` alias**: keep for one release cycle then remove (file G8)? Or remove now since only URLs that had tokens redacted would be using it and those URLs are already broken?
+4. **Platform-wide storage cap (G3)**: default value? Proposing 10 GB for single-host dev/small-team deployments.
+
+---
+
+## 9. Decision log
+
+| Date | Decision |
+|------|----------|
+| 2026-04-24 | Draft created after completing Step 6 + live Slack round-trip. |
+| 2026-04-24 | Triaged 9 items as CRITICAL (ship-blockers) and 12 as LATER (GitHub issues). |
+
+---
+
+## 10. References
+
+- Design doc: [amazing-file-outbound.md](amazing-file-outbound.md)
+- Related issues: `#295` (FILES-001, to close), `#364` (inbound web chat files, unrelated track)
+- Security posture: see `amazing-file-outbound.md` §6 — all 17 threat-model items addressed or documented.
diff --git a/docs/drafts/amazing-file-outbound.md b/docs/drafts/amazing-file-outbound.md
new file mode 100644
index 000000000..59a89789d
--- /dev/null
+++ b/docs/drafts/amazing-file-outbound.md
@@ -0,0 +1,600 @@
+# Amazing File Outbound — Design Context
+
+**Status**: Draft — MVP-in-progress
+**Created**: 2026-04-21
+**Owner**: @pavshulin
+**Scope**: Outbound file sharing (agent → user) via Trinity-hosted public URL
+
+> This is the running context doc for the feature. MVP defaults are locked (see §4). Open questions are tracked in §9 and resolved as we build/test. Update this file as decisions change.
+
+---
+
+## 1. Origin & Motivation
+
+Agents generate artifacts (CSVs, reports, images, PDFs, exports, code bundles) but today have no first-class way to hand them to a user. Current workarounds:
+
+- Fenced code blocks in chat → ugly, truncated, not downloadable.
+- `#222` inbound Slack files (user → agent) — already shipped, wrong direction.
+- `#282` outbound Slack-only (extract code block, upload to Slack via V2 API) — only handles code blocks, only Slack, size-limited.
+- Base64 data URIs in chat — fragile, terrible UX.
+
+The gap — **generic outbound file delivery that works across Slack, Telegram, public chat, and email** — was scoped as `#295` (FILES-001: Platform file storage + one-time download links) but never started.
+
+### The proposal (user, 2026-04-21)
+
+> Build file sharing on top of the public-link mechanism we already have. The agent writes a file to a volume, Trinity mints a URL (same style as `/chat/{token}`), and later we wire that URL into Slack so the agent can "post a file" in Slack by simply posting the URL.
+
+Direction: **outbound-only first** (agent → user). Inbound (user → agent uploads via a URL) is explicitly out of scope for Phase 1.
+
+---
+
+## 2. What Already Exists (reuse surface)
+
+| Capability | Where | Reusable for this feature? |
+|------------|-------|---------------------------|
+| Token-scoped public URLs | `agent_public_links` + `routers/public_links.py` + `routers/public.py` | ✅ Exact pattern to clone |
+| `secrets.token_urlsafe(24)` token generator | `routers/public_links.py:66` | ✅ |
+| External URL building | `_build_external_url()` in `routers/public_links.py` | ✅ |
+| IP rate limiting with trusted-proxy handling | `check_public_link_rate_limit()` in `routers/public.py` | ✅ |
+| Per-agent Docker volumes with ownership fix | `agent_shared_folder_config` + `services/agent_service/crud.py` (alpine chown 1000:1000) | ✅ Mirror this pattern for publish volume |
+| Container recreation on volume change | `check_shared_folder_mounts_match()` + `recreate_container_with_updated_config()` | ✅ |
+| Unified channel access gate (require_email / open_access) | `routers/public.py`, `message_router.py` (#311) | ✅ Gate downloads the same way |
+| MCP tool scaffolding | `src/mcp-server/src/tools/*.ts` | ✅ |
+| Platform audit trail | `PlatformAuditService` (SEC-001) | ✅ Emit `FILE_SHARE_CREATE`, `FILE_SHARE_DOWNLOAD` |
+| Slack adapter send path | `slack_adapter.send_response()` | ✅ Phase 2 hook point |
+
+**Bottom line**: every primitive we need is already in the codebase. This feature is mostly wiring.
+
+---
+
+## 3. Related Issues
+
+| # | State | Relationship |
+|---|-------|--------------|
+| **#295** | OPEN P1 | FILES-001. Same goal, different implementation. This doc effectively supersedes it. **Action**: post a comment on #295 linking to this draft when Phase 1 lands; update #295's body to reference the implementation path, then close when MVP ships. |
+| **#354** | CLOSED 2026-04-16 | File upload for external channels — Telegram Phase 1 shipped; Slack + public-link inbound parts dropped without successor issue. Tangential (inbound). |
+| **#364** | OPEN P2 | Web chat file upload (inbound). Orthogonal. |
+| **#222** | CLOSED 2026-03-31 | Slack inbound file sharing. Orthogonal. |
+| **#282** | CLOSED 2026-04-08 | Slack outbound via code-block extraction + native Slack upload. Phase 3 question: retire for files > threshold, keep for small code blocks. |
+| **#237** | CLOSED | Slack file download bug fix. Context. |
+
+### 3a. #295 cross-reference — what we adopt, what we change
+
+| Aspect | #295 | This MVP | Decision |
+|--------|------|----------|----------|
+| MCP tool shape | `upload_file(file_path, filename, content_type?, expires_in?, one_time?)` | `share_file(filename, display_name?, expires_in?, one_time?)` | **Adopt** `expires_in` and `one_time` from #295 (see §5.5). |
+| Storage layout | Flat `/data/files/{uuid}` | Per-agent `/data/agent-public/{name}/` | **Diverge** — per-agent is better for quotas, cleanup, and tenant isolation; mirrors shared-folders. |
+| Token | HMAC-SHA256 signed, stateless | Random `token_urlsafe(32)` stored in DB | **Diverge** — DB tokens make revocation trivial and match public-chat precedent; revisit if download QPS is ever a concern. |
+| Default lifecycle | One-time, 24h | Reusable, 7d (agent can opt into one-time per call) | **Diverge** — user preference; one-time available via flag. |
+| Access model | "Token is the auth" (anonymous) | Token + inherit agent channel-access policy (`require_email` / `open_access`) | **Diverge (stricter)** — if owner email-gated the agent, file URLs run the same gate; no side door. Behavior identical to #295 when `open_access=true`. |
+| File size cap | 100 MB | 50 MB (setting-configurable) | Minor. |
+| Per-agent quota | Not specified | 500 MB (setting-configurable) | **Added.** |
+| Listing / delete endpoints | Yes | Yes | Aligned. |
+| Executable blocklist, streamed, outside web root | Yes | Yes | Aligned. |
+
+---
+
+## 4. Locked MVP Defaults
+
+| Question | Decision | Rationale |
+|----------|----------|-----------|
+| **Q1** Which link? | Inherit behavior of existing public link (`agent_public_links`). Access policy (`require_email`, `open_access`) is read from agent ownership, same source of truth as public chat. | User answered 2026-04-21: "yes inherit behaviour". |
+| **Q2** Who creates? | Outbound only (agent → user). Inbound deferred. | User answered 2026-04-21. |
+| **Q3** Storage | Per-agent Docker-managed volume `agent-{name}-public` mounted **only** into the agent at `/home/developer/public/` (same pattern as `shared-folders`). On `share_file`, backend uses Docker SDK `get_archive()` to stream the named file out, extracts it, and stores at `/data/agent-files/{file_id}` under the existing `trinity-data` mount. Download endpoint serves from `/data/agent-files/{file_id}`. | **Zero docker-compose changes in dev or prod.** Backend never sees the agent's filesystem directly — only files the agent explicitly names via MCP (tightest blast radius). Matches existing `get_archive` usage in credential_service, agent_files, audit MCP tool. |
+| **Q4** Link lifecycle | Reusable (not one-time). 7-day expiration after last download. Manually revocable from UI. | Matches user mental model of "here's a link to my report"; one-time surprises users and conflicts with Slack unfurl races. |
+| **Q5** Size limits | 50 MB per file; 500 MB per agent total; oldest auto-expires first on quota breach. | Fits typical report/export sizes, well under Slack's 1 GB workspace ceiling. |
+| **Q6** Email gating | Inherit agent's channel-access policy. If `require_email=true`, download requires a valid `session_token` (same one the public chat uses). If `open_access=true`, anonymous allowed. | One source of truth; no side-door around owner's policy. |
+| **Q7** Slack integration | Phase 2 only. MVP ships without Slack-specific handling. Agents return the URL as plain text in chat; Slack's native unfurl will preview it. | Don't block MVP on channel-specific work; decide replace-vs-complement of #282 later based on testing. |
+
+---
+
+## 5. Architecture (MVP)
+
+### 5.1 Data flow (happy path)
+
+```
+Agent container Backend User
+--------------- ------- ----
+1. Agent writes file to
+ /home/developer/public/report.csv
+ (Docker-managed volume
+ agent-{name}-public, mounted
+ ONLY into the agent)
+
+2. Agent calls MCP tool
+ share_file("report.csv")
+ │
+ ▼
+ MCP server → POST /api/internal/agent-files/share
+ (Header: X-Internal-Secret, Agent-scoped MCP key identifies agent)
+ │
+ ▼
+3. Backend validates:
+ - Path relative, no traversal, under /home/developer/public/
+ - docker.containers.get(agent-{name}).get_archive(path)
+ - Streams tar, extracts single file to /data/agent-files/{file_id}
+ - Size ≤ 50 MB (checked as bytes stream)
+ - Agent quota ≤ 500 MB
+ - python-magic MIME detection on extracted bytes
+ - Not in blocklist (PE/ELF/Mach-O)
+
+ Inserts into agent_shared_files
+ (file_id = uuid, download_token = token_urlsafe(32))
+
+ Returns {file_id, url, expires_at}
+ ▼
+4. Agent includes URL in
+ its response to user:
+ "Here's your report: https://trinity.example.com/files/{file_id}?t={token}"
+
+ 5. User clicks URL
+ │
+ ▼
+ GET /api/files/{file_id}?t={token}
+ - Validate token (constant-time compare)
+ - Check expiration / revocation / consumption
+ - Check agent's access policy:
+ • open_access=true → allow
+ • require_email=true → check session_token query param
+ against public_link_verifications
+ • else → allow (owner's link is assumed public by default)
+ - Rate limit by IP
+ - Stream /data/agent-files/{file_id} with:
+ • Content-Disposition: attachment; filename="..."
+ • X-Content-Type-Options: nosniff
+ • Content-Type: {server-detected MIME}
+ - Emit FILE_SHARE_DOWNLOAD audit event
+ - Update last_downloaded_at, download_count
+ ▼
+ User saves file
+```
+
+### 5.2 Components to add
+
+| Layer | File | Notes |
+|-------|------|-------|
+| Schema | `src/backend/db/schema.py` | New `agent_shared_files` table + indexes |
+| Migration | `src/backend/db/migrations.py` | Versioned migration |
+| DB ops | `src/backend/db/agent_shared_files.py` | `AgentSharedFilesOperations` class (Invariant #2) |
+| Service | `src/backend/services/agent_shared_files_service.py` | Path validation, MIME detection, quota enforcement |
+| Internal router (agent-auth) | `src/backend/routers/internal.py` (extend) | `POST /api/internal/agent-files/share` |
+| Public router (token-auth) | `src/backend/routers/files.py` (new) | `GET /api/files/{id}` with streaming response |
+| Admin router (JWT) | `src/backend/routers/agent_files.py` (extend) | `GET /api/agents/{name}/shared-files`, `DELETE /api/agents/{name}/shared-files/{id}` |
+| MCP tool | `src/mcp-server/src/tools/agents.ts` (extend) | `share_file` tool |
+| Volume mgmt | `src/backend/services/agent_service/crud.py` + `lifecycle.py` | Create/attach Docker-managed volume `agent-{name}-public` on agent start when feature is enabled (mirrors `shared-folders` expose pattern). Backend does NOT mount this volume. |
+| Settings toggle | `agent_ownership` column `file_sharing_enabled` (default 0) | Owner opts in per-agent |
+| UI | `src/frontend/src/components/SharingPanel.vue` or new `FileSharingPanel.vue` | List/revoke shared files, toggle feature on/off |
+| Cleanup | `src/backend/services/cleanup_service.py` | Add sweep for expired rows + disk deletion |
+
+### 5.3 URL structure
+
+```
+External: https://{public_chat_url}/files/{file_id}?t={download_token}
+ &s={session_token} ← only when agent requires email
+Internal: http://localhost:8000/api/files/{file_id}?t={download_token}
+```
+
+Under `/files/` (frontend route proxying to backend) — **not** under `/api/agents/{name}/...` — because users don't have an agent name in their hand. Matches public chat's `/chat/{token}` pattern.
+
+### 5.4 DB schema (draft)
+
+```sql
+CREATE TABLE IF NOT EXISTS agent_shared_files (
+ id TEXT PRIMARY KEY, -- uuid
+ agent_name TEXT NOT NULL,
+ filename TEXT NOT NULL, -- original display name for download
+ stored_filename TEXT NOT NULL, -- UUID filename under /data/agent-files/
+ size_bytes INTEGER NOT NULL,
+ mime_type TEXT, -- python-magic detected
+ download_token TEXT UNIQUE NOT NULL, -- secrets.token_urlsafe(32)
+ created_by TEXT NOT NULL, -- agent_name (or user_id if admin created)
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL, -- created_at + expires_in (default 7d)
+ revoked_at TEXT, -- set if manually revoked
+ one_time INTEGER DEFAULT 0, -- 1 = invalidate after first download
+ consumed_at TEXT, -- set on first successful GET when one_time=1
+ download_count INTEGER DEFAULT 0,
+ last_downloaded_at TEXT,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
+ ON DELETE CASCADE ON UPDATE CASCADE
+);
+-- ON UPDATE CASCADE is belt-and-suspenders: rename_agent() in db/agent_settings/metadata.py
+-- also explicitly UPDATEs this table (same pattern as the other 16 tables it cascades through).
+
+CREATE INDEX idx_agent_files_agent ON agent_shared_files(agent_name);
+CREATE INDEX idx_agent_files_token ON agent_shared_files(download_token);
+CREATE INDEX idx_agent_files_expires ON agent_shared_files(expires_at) WHERE revoked_at IS NULL;
+```
+
+Plus one column on `agent_ownership`:
+
+```sql
+ALTER TABLE agent_ownership ADD COLUMN file_sharing_enabled INTEGER DEFAULT 0;
+```
+
+### 5.5 MCP tool contract
+
+```typescript
+share_file({
+ filename: string, // Relative path in /home/developer/public/
+ display_name?: string, // Override download filename (default: basename(filename))
+ expires_in?: number, // Seconds until expiration. Default 604800 (7d). Min 60s, max 604800s.
+}): {
+ file_id: string,
+ url: string,
+ expires_at: string,
+ size_bytes: number,
+ mime_type: string,
+}
+```
+
+> **Deferred (2026-04-24)**: `one_time: boolean` is omitted from the MVP MCP tool + request/response and from the download endpoint's consumption path. The schema columns (`one_time`, `consumed_at`) are retained so the feature can be added back without a migration. See §9 OQ9 and the Decision Log.
+
+Errors: `FILE_NOT_FOUND`, `PATH_TRAVERSAL`, `SIZE_LIMIT_EXCEEDED`, `QUOTA_EXCEEDED`, `FEATURE_DISABLED`, `MIME_BLOCKED`, `INVALID_EXPIRATION`.
+
+---
+
+## 6. Security Review (OWASP-targeted)
+
+| # | Threat | Mitigation in MVP |
+|---|--------|-------------------|
+| S1 | Path traversal — agent tries `share_file("../.env")` | Reject absolute paths. Normalize and require the resolved path to begin with `/home/developer/public/`. Reject any path containing `..` segments after normalization. `get_archive` call happens inside the agent container's own context — even if path validation slipped, the agent can only name files it already has read access to, not backend files. |
+| S2 | Credential leak via backend reach | Backend never mounts the agent workspace; it extracts only the single file the agent explicitly names via MCP. The extracted file is the only artifact the backend ever touches. |
+| S3 | Predictable tokens | `secrets.token_urlsafe(32)` (192-bit entropy). Constant-time compare on download. |
+| S4 | Link leaked to public archives (Slack indexers, bots) | 7-day expiration; revocable; audit log of every download with IP + UA. |
+| S5 | Slack unfurl pre-consumes `one_time=true` link | N/A for MVP (one-time deferred). When re-added: detect `User-Agent: Slackbot-LinkExpanding` (and equivalents for other crawlers) and **return the file without marking `consumed_at`**. Only non-crawler GETs consume. |
+| S6 | XSS via agent-uploaded HTML served inline | Force `Content-Disposition: attachment`; `X-Content-Type-Options: nosniff`; CSP `default-src 'none'`; consider serving from cookieless subdomain (post-MVP). |
+| S7 | Filename header injection (CRLF) | Strip control chars; RFC 6266 quoting; fallback `file-{id}.bin` if sanitization fails. |
+| S8 | MIME spoofing | python-magic server-side detection; reject PE/ELF/Mach-O; serve with detected MIME, not agent-claimed. |
+| S9 | Storage DoS (runaway agent fills disk) | Per-file 50 MB cap, per-agent 500 MB quota, global cap via setting. Oldest expires first on quota breach. Cleanup every 60s. |
+| S10 | Enumeration / brute force | 192-bit tokens (infeasible). IP rate limit reuses `check_public_link_rate_limit` (30/min). |
+| S11 | Cross-tenant download | `file_id` alone addresses files; agent_name is looked up from DB row, never accepted from URL. |
+| S12 | SSRF via URL-based file sharing | Not a vector — agent writes to local volume only. No URL-fetch code path. |
+| S13 | Audit gaps | `FILE_SHARE_CREATE` (agent, file_id, size, MIME, filename) + `FILE_SHARE_DOWNLOAD` (IP, UA, success). |
+| S14 | Access-policy bypass (require_email agent, file URL skips it) | Download endpoint runs the same gate helpers as `/api/public/chat/{token}`. Unified gate. |
+| S15 | Agent impersonation via MCP | `share_file` takes no `agent_name` param; backend reads it from the MCP key's agent scope. Agent A cannot share from Agent B's volume. |
+| S16 | Log leakage of content | Only metadata logged (file_id, size, MIME, requester); never content. |
+| S17 | One-time race (two GETs arriving simultaneously on `one_time=true` link) | N/A for MVP (one-time deferred). When re-added: atomic `UPDATE agent_shared_files SET consumed_at=? WHERE id=? AND consumed_at IS NULL RETURNING id` — first winner streams, losers 410 Gone. |
+
+### Architectural invariants checked
+
+- ✅ **#1 Three-layer backend**: router → service → db operations
+- ✅ **#2 Class-per-domain DB ops**: new `AgentSharedFilesOperations`
+- ✅ **#3 Schema in `db/schema.py`, versioned migration**: yes
+- ✅ **#4 Router registration order**: new `/api/files/{id}` is top-level, no conflict with `/api/agents/{name}/...`
+- ✅ **#8 Auth pattern**: internal endpoint uses `X-Internal-Secret` + MCP agent scope; admin endpoints use `Depends(get_current_user)`; public download is token-scoped (matches public chat precedent)
+- ✅ **#11 Docker as source of truth**: metadata in SQLite, Docker volume is storage only
+- ✅ **#13 MCP/backend/agent three surfaces**: MCP TypeScript + backend router both added
+- ✅ **#14 Pydantic models centralized**: add `ShareFileRequest`, `ShareFileResponse` to `models.py`
+
+---
+
+## 7. Phased Rollout
+
+### Phase 1 — MVP (this iteration)
+
+Ordered by dependency. Each step is independently testable before moving to the next.
+
+#### Step 1 — Schema + migration (~30 min)
+
+**Add**:
+- `agent_shared_files` table (see §5.4) in `src/backend/db/schema.py` (add to `TABLES` dict; index declarations into the indexes list).
+- `agent_ownership.file_sharing_enabled INTEGER DEFAULT 0` column.
+- Versioned migration in `src/backend/db/migrations.py` (next migration number after current head).
+
+**Verify locally**:
+```bash
+./scripts/deploy/start.sh # or: docker compose restart backend
+docker compose logs backend | grep -i migration # confirms migration ran
+sqlite3 ~/trinity-data/trinity.db ".schema agent_shared_files"
+sqlite3 ~/trinity-data/trinity.db "SELECT file_sharing_enabled FROM agent_ownership LIMIT 1"
+```
+
+**Gate**: schema present, `file_sharing_enabled` column default 0 on existing agents. No runtime impact.
+
+---
+
+#### Step 2 — Publish volume + opt-in toggle (~1.5 h)
+
+**Add**:
+- `PUT /api/agents/{name}/file-sharing` — body `{enabled: bool}`. Owner-only (use `can_user_share_agent`). Writes `agent_ownership.file_sharing_enabled`, returns `{restart_required: true}`.
+- `GET /api/agents/{name}/file-sharing` — returns `{enabled, volume_attached, file_count, total_bytes, quota_bytes}`.
+- Extend `services/agent_service/crud.py` and `lifecycle.py`: when `file_sharing_enabled=1`, create Docker-managed volume `agent-{name}-public` and mount only into the agent at `/home/developer/public/` (rw). Apply alpine chown-1000 fix. **No backend-side mount.**
+- Extend `check_shared_folder_mounts_match()` (or add `check_public_folder_mount()`) so container recreates on toggle.
+- Ensure `agents.py` DELETE handler removes the `agent-{name}-public` volume alongside `agent-{name}-shared`.
+
+**Verify locally** (no compose rebuild needed — backend code changes auto-reload):
+```bash
+TOKEN=$(curl -s -X POST http://localhost:8000/api/token -d "username=admin&password=$ADMIN_PASSWORD" | jq -r .access_token)
+curl -s -X POST http://localhost:8000/api/agents -H "Authorization: Bearer $TOKEN" \
+ -H "Content-Type: application/json" -d '{"name":"filetest","template":"local:default"}'
+
+curl -s -X PUT http://localhost:8000/api/agents/filetest/file-sharing \
+ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"enabled": true}'
+
+curl -s -X POST http://localhost:8000/api/agents/filetest/stop -H "Authorization: Bearer $TOKEN"
+curl -s -X POST http://localhost:8000/api/agents/filetest/start -H "Authorization: Bearer $TOKEN"
+
+docker volume ls | grep agent-filetest-public # volume exists
+docker exec agent-filetest ls -la /home/developer/public # mounted rw, owned by 1000
+docker exec agent-filetest sh -c 'echo hi > /home/developer/public/x.txt' # writable by agent
+# Backend MUST NOT have /data/agent-public/ — this is intentional:
+docker exec trinity-backend ls /data/agent-public 2>&1 # expect: No such file or directory
+```
+
+**Gate**: owner can toggle, agent has `/home/developer/public/` (writable), backend has **no direct view** of it.
+
+---
+
+#### Step 3 — Backend `share` endpoint (~2 h)
+
+**Add**:
+- `src/backend/db/agent_shared_files.py` — `AgentSharedFilesOperations` class (Invariant #2): `create`, `get_by_id`, `get_by_token`, `list_by_agent`, `mark_downloaded`, `consume_one_time_atomic`, `revoke`, `delete_expired`, `total_bytes_for_agent`.
+- `src/backend/services/agent_shared_files_service.py`:
+ - `validate_publish_path(filename)` — reject absolute paths; normalize; reject `..` segments after normalization; final form must start with `/home/developer/public/` when combined.
+ - `extract_from_agent(agent_name, filename) -> bytes` — uses Docker SDK `get_archive` to pull the tar, extracts a single regular file (rejects symlinks/dirs/devices), caps bytes read at 50 MB + 1 to detect oversize early.
+ - `detect_mime(bytes) -> str` — python-magic (already installed via #354).
+ - `check_mime_blocklist(mime)` — reject PE/ELF/Mach-O/shell scripts.
+ - `enforce_quota(agent_name, new_bytes)` — sum `size_bytes` where `revoked_at IS NULL AND (consumed_at IS NULL OR expires_at > now())`.
+ - `create_share(agent_name, filename, display_name, expires_in, one_time)` — orchestrator: validate path → extract → MIME detect → quota → write to `/data/agent-files/{file_id}` → DB insert → return `(file_id, url, expires_at, size_bytes, mime_type, one_time)`.
+- `src/backend/routers/internal.py` — add `POST /api/internal/agent-files/share` gated by `X-Internal-Secret` + agent-scoped MCP key (agent_name from context; body contains filename/options only).
+- Pydantic `ShareFileRequest` / `ShareFileResponse` in `src/backend/models.py`.
+
+**Verify locally**:
+```bash
+# Drop a file in the agent's publish volume
+docker exec agent-filetest sh -c 'printf "a,b\n1,2\n" > /home/developer/public/test.csv'
+
+# Agent-scoped call (X-Internal-Secret + agent MCP key)
+AGENT_KEY=$(docker exec agent-filetest printenv TRINITY_MCP_API_KEY)
+INTERNAL_SECRET=$(docker compose exec backend printenv INTERNAL_API_SECRET | tr -d '\r')
+curl -s -X POST http://localhost:8000/api/internal/agent-files/share \
+ -H "X-Internal-Secret: $INTERNAL_SECRET" \
+ -H "Authorization: Bearer $AGENT_KEY" \
+ -H "Content-Type: application/json" \
+ -d '{"filename":"test.csv","expires_in":604800,"one_time":false}'
+# → {"file_id":"...","url":"http://localhost/files/.../?t=...","expires_at":"...",...}
+
+# Abuse cases
+curl ... -d '{"filename":"/etc/passwd"}' # 400 PATH_TRAVERSAL (absolute)
+curl ... -d '{"filename":"../.env"}' # 400 PATH_TRAVERSAL (escape)
+curl ... -d '{"filename":"nonexistent.csv"}' # 404 FILE_NOT_FOUND
+ln -s /etc/passwd /home/developer/public/link.txt # via docker exec
+curl ... -d '{"filename":"link.txt"}' # 400 NOT_REGULAR_FILE
+
+# Quota test
+docker exec agent-filetest sh -c 'dd if=/dev/zero of=/home/developer/public/big.bin bs=1M count=51 2>/dev/null'
+curl ... -d '{"filename":"big.bin"}' # 413 SIZE_LIMIT_EXCEEDED
+
+# Verify DB + on-disk artifact
+sqlite3 ~/trinity-data/trinity.db "SELECT id,filename,size_bytes,mime_type,one_time FROM agent_shared_files"
+docker compose exec backend ls -la /data/agent-files/
+```
+
+**Gate**: can register a valid file → get URL; all abuse cases return correct HTTP code.
+
+---
+
+#### Step 4 — Public download endpoint (~1.5 h)
+
+**Add**:
+- `src/backend/routers/files.py` — new top-level router. `GET /api/files/{file_id}` with `?t={token}&s={session_token}` query.
+- Constant-time token compare (`secrets.compare_digest`).
+- Policy gate: if agent has `require_email=1`, require `session_token` matching a valid row in `public_link_verifications` (reuse `_agent_requires_email` helper from `routers/public.py`).
+- `StreamingResponse` with chunked read from `/data/agent-files/{stored_filename}`.
+- Headers: `Content-Disposition: attachment; filename="sanitized"`, `X-Content-Type-Options: nosniff`, `Content-Type: {detected_mime}`, `Cache-Control: private, no-store`.
+- Atomic consume for `one_time=true` via DB RETURNING.
+- Rate-limit by IP (reuse `check_public_link_rate_limit`).
+- Emit `FILE_SHARE_DOWNLOAD` audit event.
+- Register router in `main.py` **before** any `/api/agents/{name}` catch-alls (Invariant #4).
+
+**Verify locally**:
+```bash
+# Happy path — open URL from Step 3 in browser: should download test.csv
+# Or via curl
+curl -v "http://localhost:8000/api/files/?t=" -o /tmp/downloaded.csv
+diff /tmp/downloaded.csv <(docker exec agent-filetest cat /home/developer/public/test.csv)
+
+# Wrong token → 404
+curl -v "http://localhost:8000/api/files/?t=WRONG"
+
+# Revoked → 410
+sqlite3 ~/trinity-data/trinity.db "UPDATE agent_shared_files SET revoked_at=datetime('now') WHERE id=''"
+curl -v "http://localhost:8000/api/files/?t="
+
+# Expired → 410
+sqlite3 ~/trinity-data/trinity.db "UPDATE agent_shared_files SET expires_at=datetime('now','-1 hour') WHERE id=''"
+curl -v "http://localhost:8000/api/files/?t="
+
+# One-time consumed twice → second request 410
+curl "http://localhost:8000/api/files/?t=" # 200
+curl "http://localhost:8000/api/files/?t=" # 410
+
+# Slackbot unfurl simulation (should NOT consume)
+curl -H "User-Agent: Slackbot-LinkExpanding 1.0" "http://localhost:8000/api/files/?t="
+sqlite3 ~/trinity-data/trinity.db "SELECT consumed_at FROM agent_shared_files WHERE id=''"
+# → still NULL; real GET after still works
+
+# Email-gated agent (if agent has require_email=1): download without session → 403
+# With valid session token from public-chat verify flow → 200
+
+# Audit
+curl -s -H "Authorization: Bearer $TOKEN" \
+ "http://localhost:8000/api/audit-log?event_type=file_share" | jq
+```
+
+**Gate**: all HTTP codes correct, byte-identical download, headers set, audit rows written.
+
+---
+
+#### Step 5 — MCP `share_file` tool (~1 h)
+
+**Add**:
+- `src/mcp-server/src/tools/agents.ts` — new `share_file` tool. Wraps `POST /api/internal/agent-files/share`.
+- Schema matches §5.5. Pulls `agent_name` from `McpAuthContext.agentName` (enforce agent-scoped key).
+- Rebuild MCP server.
+
+**Verify locally**:
+```bash
+docker compose build mcp-server && docker compose up -d mcp-server
+# From a running agent's Claude Code session (SSH or terminal tab):
+claude -p 'Call the share_file MCP tool with filename="test.csv" and return the URL'
+# Or test via MCP HTTP directly:
+AGENT_KEY=$(docker exec agent-filetest printenv TRINITY_MCP_API_KEY)
+curl -s http://localhost:8080/mcp/tools/call \
+ -H "Authorization: Bearer $AGENT_KEY" \
+ -H "Content-Type: application/json" \
+ -d '{"name":"share_file","arguments":{"filename":"test.csv"}}'
+```
+
+**Gate**: MCP tool returns valid URL; cross-agent attempt (Agent A key with Agent B filename) is refused by internal endpoint.
+
+---
+
+#### Step 6 — UI: toggle + file list (~3 h)
+
+**Add**:
+- `src/frontend/src/components/FileSharingPanel.vue` — toggle, quota bar, table of active shared files (filename, size, created, expires, download count, Copy URL, Revoke).
+- Embed in `SharingPanel.vue` (next to Public Links section) when `agent.can_share`.
+- `stores/agents.js` — `getFileSharing`, `updateFileSharing`, `listSharedFiles`, `revokeSharedFile`.
+
+**Verify locally** (browser):
+- Navigate to `http://localhost/agents/filetest` → Sharing tab → "File Sharing" section appears.
+- Toggle on, restart banner shows, restart agent.
+- Have agent create a file + share via MCP (reuse Step 5 session).
+- File appears in UI; copy URL, download works.
+- Click Revoke; file row marked revoked; URL returns 410.
+
+**Gate**: owner can drive the full lifecycle from UI without touching CLI.
+
+---
+
+#### Step 7 — Cleanup task (~45 min)
+
+**Add**:
+- Extend `src/backend/services/cleanup_service.py`: new sweep `_cleanup_expired_shared_files()` on the existing 5-min tick.
+ - Delete disk file at `/data/agent-files/{stored_filename}` when `expires_at < now` OR `(one_time=1 AND consumed_at IS NOT NULL)` OR `revoked_at IS NOT NULL AND revoked_at < now - 24h`.
+ - Then `DELETE FROM agent_shared_files WHERE id=?`.
+ - Emit audit event per cleanup batch.
+
+**Verify locally**:
+```bash
+# Age a row
+sqlite3 ~/trinity-data/trinity.db "UPDATE agent_shared_files SET expires_at=datetime('now','-1 day') WHERE id=''"
+
+# Trigger cleanup manually (admin endpoint already exists)
+curl -s -X POST http://localhost:8000/api/monitoring/cleanup-trigger -H "Authorization: Bearer $TOKEN"
+
+# Verify row gone + on-disk artifact gone
+sqlite3 ~/trinity-data/trinity.db "SELECT id FROM agent_shared_files WHERE id=''" # empty
+docker compose exec backend ls /data/agent-files/ # no such file
+```
+
+**Gate**: expired/consumed/revoked files are swept within one cleanup cycle.
+
+---
+
+### Phase 2 — Slack integration
+
+### Phase 2 — Slack integration
+- Slack adapter: when agent response body contains a Trinity file URL, optionally enrich the Slack message (e.g., explicit "📎 Download: ")
+- Decision: retire `#282` code-block extraction for files > N KB, or keep both (decide after testing).
+
+### Phase 1 — effort summary
+
+| Step | Rough effort |
+|------|--------------|
+| 1. Schema + migration | 30 min |
+| 2. Volume + toggle | 1.5 h |
+| 3. Backend share endpoint | 2 h |
+| 4. Public download endpoint | 1.5 h |
+| 5. MCP `share_file` tool | 1 h |
+| 6. UI | 3 h |
+| 7. Cleanup task | 45 min |
+| **Total** | **~10 h** across 1–2 sessions |
+
+### Phase 3 — Other channels
+- Telegram: same URL-in-text pattern (Telegram unfurls too)
+- Public chat: render download chip instead of raw URL
+- Email (future)
+
+### Phase 4 — Extensions (backlog)
+- One-time download mode
+- Password-protected links
+- Per-file access policy override (instead of agent-wide)
+- Cookieless subdomain for XSS defense-in-depth
+- User-initiated inbound (combine with #364)
+- Retire/unify with #295
+
+---
+
+## 8. Testing Plan (local)
+
+Golden path:
+1. Create agent with `file_sharing_enabled=true`.
+2. Have agent write `/home/developer/public/test.csv`.
+3. Agent calls `share_file("test.csv")` via MCP.
+4. Receive URL; open in browser; verify download + correct filename + correct MIME.
+5. Re-download; verify counter increments.
+6. Revoke via UI; verify 410 Gone.
+7. Wait past expiration (or manually age); verify cleanup removes disk + row.
+
+Abuse paths:
+- `share_file("../../etc/passwd")` → rejected
+- `share_file("/absolute/path")` → rejected
+- `share_file("../.env")` → rejected (and volume isolation means even if it got through, the path isn't reachable)
+- 51 MB file → rejected on size cap
+- 11 × 50 MB files → 11th rejected on quota
+- File containing PE/ELF header → rejected on MIME
+- Direct GET with wrong token → 404 (no enumeration signal)
+- Direct GET with expired token → 410
+- Agent with `require_email=true`, GET without session token → 403
+
+Cross-agent:
+- Agent A cannot call `share_file` with a path that resolves into agent B's volume (validated by MCP scope + backend path check).
+
+UI:
+- Owner sees list of shared files, can revoke, sees per-file download count.
+
+---
+
+## 9. Open Questions (tracked for resolution during/after MVP)
+
+| # | Question | Status |
+|---|----------|--------|
+| OQ1 | Should we retire `#282` (Slack code-block → native Slack upload) once URL-posting works, or keep both paths? | Defer to Phase 2. |
+| OQ2 | Do we want a cookieless subdomain for download serving (defense-in-depth against HTML XSS)? | Defer; non-blocker for MVP. |
+| OQ3 | Should revoking the agent's public chat link cascade-revoke file links, or are they independent? | **Independent** — file-sharing is a separate per-agent toggle. Revisit if coupling is ever requested. |
+| OQ4 | How does `trinity-system` agent use this (if at all)? | Defer. |
+| OQ5 | Quota failure UX — silent oldest-eviction or hard error to agent? | MVP: **hard error**, agent decides what to do. |
+| OQ6 | Is 7 days the right default expiration, or per-agent-configurable? | MVP: 7 days hardcoded; add setting in Phase 4 if requested. |
+| OQ7 | Do we need a download-page interstitial (filename, size, "click to download") for better UX + Slack-unfurl behavior? | Defer; direct stream for MVP. |
+| OQ8 | Should we support overwriting an existing share (stable URL that gets new content)? | No for MVP — each `share_file` mints a new URL. Agents can revoke the old one explicitly. |
+| OQ9 | One-time download links | **Deferred** (2026-04-24). Schema columns (`one_time`, `consumed_at`) retained; API / service / docs surface removed. Re-add when we have a use case driving it (e.g. sharing sensitive artifacts). |
+
+---
+
+## 10. Decision Log
+
+| Date | Decision | Rationale |
+|------|----------|-----------|
+| 2026-04-21 | Build atop public-link pattern, not a standalone platform-files service. | User preference; reuses hardened primitives; inherits access-control model. |
+| 2026-04-21 | Outbound-only for Phase 1. | User scope-cut; inbound via #364 later. |
+| 2026-04-21 | Per-agent publish Docker volume bind-mounted to backend. | Eliminates `docker cp` cost; filesystem-isolates from agent secrets. |
+| 2026-04-23 | **Revised**: per-agent Docker-managed publish volume, **not** bind-mounted to backend. Backend uses Docker SDK `get_archive` on `share_file` to extract and store at `/data/agent-files/{id}` under the existing `trinity-data` mount. | Zero docker-compose changes in dev or prod (no compose drift, no ops-side filesystem prep). Backend only touches files the agent explicitly names via MCP — tightest blast radius. Trade-off: ~<1s extraction cost per share, acceptable at 50 MB cap. |
+| 2026-04-23 | Use `agent_name` as the identifier (matches every other table); FK declared with `ON DELETE CASCADE ON UPDATE CASCADE`; also added explicit `UPDATE agent_shared_files` line to `rename_agent()` in `db/agent_settings/metadata.py`. | Consistency with 16 existing tables. Surrogate ID would have made this one table the outlier and added a JOIN to every list/filter query. `ON UPDATE CASCADE` + explicit `rename_agent` update is defense-in-depth so a future maintainer can't break it. |
+| 2026-04-24 | One-time download links deferred. Stripped `one_time` / `consumed_at` from the MCP tool contract, request/response models, service params, and DB `create()`. Schema columns retained. Step 4 removes `consume_one_time_atomic` DB method and `is_crawler_user_agent` helper from scope. | Simplifies the first working slice; no real use case on the table today. Bringing it back later is a ~40-line change because the columns are already there. |
+| 2026-04-21 | Reusable 7-day links, not one-time. | User mental model; avoids Slack unfurl race. |
+| 2026-04-21 | Email-gating inherits agent's channel policy. | Single source of truth. |
+| 2026-04-21 | Defer Slack-specific integration to Phase 2. | Don't block MVP on channel bikeshedding. |
+
+---
+
+## 11. References
+
+- #295 — FILES-001 (original request, to be superseded)
+- #354 — Inbound file upload (closed partially)
+- `docs/memory/feature-flows/public-agent-links.md` — URL + access-policy pattern we're cloning
+- `docs/memory/feature-flows/agent-shared-folders.md` — Docker volume pattern we're cloning
+- `docs/memory/feature-flows/unified-channel-access-control.md` — Gating model
+- `docs/memory/architecture.md` — Architectural invariants checked
diff --git a/docs/memory/architecture.md b/docs/memory/architecture.md
index 5f2e8d47c..0c8b18a4b 100644
--- a/docs/memory/architecture.md
+++ b/docs/memory/architecture.md
@@ -96,7 +96,8 @@ Each agent runs as an isolated Docker container with standardized interfaces for
*Core Agent:*
- `agents.py` - Core CRUD, start/stop, logs, stats, queue, activities, terminal (642 lines)
- `agent_config.py` - Per-agent settings: autonomy, read-only, resources, capabilities, capacity, timeout, api-key
-- `agent_files.py` - Files, info, playbooks, permissions, metrics, shared folders
+- `agent_files.py` - Files, info, playbooks, permissions, metrics, shared folders, file-sharing toggle + list/revoke (FILES-001)
+- `files.py` - Public download endpoint for outbound agent file sharing (FILES-001)
- `agent_rename.py` - Rename endpoint (RENAME-001)
- `agent_ssh.py` - SSH access endpoint
- `credentials.py` - Credential injection/export/import (CRED-002 simplified system)
@@ -170,9 +171,9 @@ Each agent runs as an isolated Docker container with standardized interfaces for
*Execution & Scheduling:*
- `task_execution_service.py` - Unified task execution lifecycle (slot mgmt, activity tracking, sanitization) (EXEC-024)
-- `slot_service.py` - Parallel execution slot management with dynamic TTL (CAPACITY-001)
-- `backlog_service.py` - Persistent SQLite-backed FIFO backlog for async tasks at capacity (BACKLOG-001)
-- `execution_queue.py` - Redis-based execution queueing
+- `capacity_manager.py` - **Unified capacity facade (#428, CAPACITY-CONSOLIDATE).** Single public API for admit/release/status across `/chat` (`max_concurrent=max_parallel_tasks`, `queue_in_memory` policy) and `/task` (`queue_persistent` policy). Composes `slot_service.py` and `backlog_service.py` internally; owns the in-memory overflow store (Redis LIST, depth 3). Replaces the prior three-class pyramid (`SlotService` + `ExecutionQueue` + `BacklogService`); `ExecutionQueue` deleted, the other two are now private internals.
+- `slot_service.py` - Internal: atomic N-ary capacity counter (Redis ZSET) with dynamic per-agent TTL (CAPACITY-001). Used only by `CapacityManager`.
+- `backlog_service.py` - Internal: persistent SQLite-backed FIFO overflow store with drain-on-release (BACKLOG-001). Used only by `CapacityManager`.
- `scheduler_service.py` - APScheduler-based scheduling service
- `cleanup_service.py` - Active watchdog reconciliation + passive stale recovery for executions, activities, and slots (CLEANUP-001, #129)
@@ -199,6 +200,7 @@ Each agent runs as an isolated Docker container with standardized interfaces for
- `slack_service.py` - Slack API client (OAuth, messaging, verification) (SLACK-001)
- `nevermined_payment_service.py` - x402 payment verification and settlement (NVM-001)
- `proactive_message_service.py` - Agent-to-user proactive messaging with rate limiting and audit (#321)
+- `agent_shared_files_service.py` - Outbound file sharing: path validation, MIME blocklist, quota, Docker `get_archive` extraction, URL building (FILES-001)
**Channel Adapters (`adapters/`)** — Pluggable external messaging (SLACK-002):
@@ -322,6 +324,7 @@ Each agent runs as an isolated Docker container with standardized interfaces for
| `docs.ts` (1) | `get_agent_requirements` | Agent documentation |
| `channels.ts` (2) | `list_channel_groups`, `send_group_message` | Channel group discovery and proactive group messaging (#349) |
| `messages.ts` (1) | `send_message` | Proactive user messaging by verified email (#321) |
+| `files.ts` (1) | `share_file` | Outbound file sharing — publish file from `/home/developer/public/` and return download URL (FILES-001) |
### Vector Log Aggregator (`config/vector.yaml`)
@@ -357,12 +360,14 @@ docker exec trinity-vector sh -c "tail -50 /data/logs/agents.json" | jq .
**Internal Server:** `agent-server.py`
- FastAPI app on port 8000
- `/api/chat` - Claude Code execution (messages persisted to database)
-- `/api/health` - Health check
+- `/health` - Health check
- `/api/credentials/update` - Hot-reload credentials
- `/api/chat/session` - Context window stats
- `/api/files` - List workspace files (recursive tree structure)
- `/api/files/download` - Download file content (100MB limit)
+**Template-supplied pre-check** (optional, SCHED-COND-001): if the template ships an executable `~/.trinity/pre-check` file, the backend's internal endpoint `POST /api/internal/agents/{name}/pre-check` runs it via `docker exec` before the scheduler fires a cron-triggered chat. The hook is **language-agnostic** — interpreter is selected by the file's shebang line (Python, bash, node, compiled binary, …); Trinity does not invoke `python3` for it. The hook's stdout becomes the chat message; empty stdout + exit 0 records a skipped execution. No HTTP endpoint is exposed on the agent-server for this — the primitive is the same `execute_command_in_container` already used by `services/git_service.py` (persistent-state allowlist), `ssh_service.py`, and the agent terminal.
+
**Persistent Chat:**
- All chat messages automatically saved to SQLite (`chat_sessions`, `chat_messages`)
- Sessions survive container restarts/deletions
@@ -395,8 +400,9 @@ Services that run continuously in the backend process:
| **Operator Queue Sync** | `operator_queue_service.py` | Polls running agents every 5s, reads `~/.trinity/operator-queue.json`, syncs to DB, writes responses back. (OPS-001) |
| **Sync Health Service** | `sync_health_service.py` | Polls git-enabled agents every 60s, upserts `agent_sync_state`, emits `sync_failing` operator-queue entries when consecutive_failures ≥ 3. (#389 S1) |
| **Monitoring Service** | `monitoring_service.py` | Fleet-wide health checks on configurable interval. (MON-001) |
-| **Scheduler Service** | `scheduler_service.py` | APScheduler-based cron job execution. Async fire-and-forget with DB polling for status. |
-| **Backlog Maintenance** | `backlog_service.py` | Expires stale queued tasks (>24h) and drains orphans after restart. Runs every 60s. (BACKLOG-001) |
+| **Scheduler Service** | `scheduler_service.py` | APScheduler-based cron job execution. Async fire-and-forget with DB polling for status. On each cron-triggered fire, optionally invokes the agent's executable `~/.trinity/pre-check` (interpreter chosen by shebang) via the backend's `POST /api/internal/agents/{name}/pre-check` (which `docker exec`s into the agent container). Empty stdout + exit 0 records a skipped execution and does not invoke Claude (SCHED-COND-001, #454). |
+| **Capacity Maintenance** | `capacity_manager.py` | Calls `CapacityManager.run_maintenance()` every 60s — expires stale queued tasks (>24h) and drains orphans after restart. (BACKLOG-001 / CAPACITY-CONSOLIDATE #428) |
+| **Audit Retention** | `audit_retention_service.py` | Daily APScheduler job at 04:15 UTC that DELETEs `audit_log` rows past the retention window. Configured via `AUDIT_LOG_RETENTION_DAYS` (default 365, floored at 365 — the `audit_log_no_delete` trigger refuses younger rows). Pruning ages out hash-chain history past the cutoff by design. (#552) |
The **agent server** also runs a 15-min `auto_sync` heartbeat loop (gated
by `GIT_SYNC_AUTO` env var; default-on for non-source-mode GitHub-template
@@ -502,6 +508,11 @@ picks up on its next poll. (#389 S1a)
| PUT | `/api/agents/{name}/timeout` | Set execution timeout (60-7200s, default 900s = 15min) |
| GET | `/api/agents/{name}/guardrails` | Get per-agent guardrails config (NEW: 2026-04-15) |
| PUT | `/api/agents/{name}/guardrails` | Set per-agent guardrails overrides (GUARD-001) |
+| GET | `/api/agents/{name}/file-sharing` | Get outbound file-sharing status + quota (NEW: 2026-04-24, FILES-001) |
+| PUT | `/api/agents/{name}/file-sharing` | Enable/disable outbound file sharing (owner-only; returns `restart_required`) |
+| POST | `/api/agents/{name}/shared-files` | Mint a download URL for a file in the publish dir (owner/admin or agent-scoped key; used by `share_file` MCP tool) |
+| GET | `/api/agents/{name}/shared-files` | List active (non-revoked, non-expired) shared files with download counts |
+| DELETE | `/api/agents/{name}/shared-files/{file_id}` | Revoke a shared file (owner-only; idempotent) |
**Note**: Route ordering is critical. `/context-stats` and `/autonomy-status` must be defined BEFORE `/{name}` catch-all route to avoid 404 errors.
@@ -609,7 +620,7 @@ picks up on its next poll. (#389 S1a)
| DELETE | `/api/mcp/keys/{id}` | Delete API key |
| GET | `/oauth/{provider}/authorize` | Start OAuth |
| GET | `/oauth/{provider}/callback` | OAuth callback |
-| GET | `/api/health` | Health check |
+| GET | `/health` | Health check (unauthenticated, top-level — no `/api/` prefix) |
### Fleet Sync Audit (#390 / S6)
@@ -676,6 +687,15 @@ export, enable/disable toggle. Issue #20 can be closed.
| GET | `/api/nevermined/settlement-failures` | Admin | Failed settlements |
| POST | `/api/nevermined/retry-settlement/{log_id}` | Admin | Retry settlement |
+### Outbound File Sharing (FILES-001, NEW: 2026-04-24)
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| GET | `/api/files/{file_id}` | Token (`?sig=`) | Public download. 401 on bad/missing sig, 404 on unknown id, 410 on revoked/expired, `Content-Disposition: attachment; filename="..."`, `X-Content-Type-Options: nosniff`, rate-limited per IP, audit event `file_share_download` |
+| POST | `/api/internal/agent-files/share` | `X-Internal-Secret` | Agent-server path — mint a download URL (used by agent-server direct calls, not the MCP tool) |
+
+Storage: `/data/agent-files/{file_id}` under the existing `trinity-data` volume (no compose changes). Agent writes to `/home/developer/public/` (Docker volume `agent-{name}-public`); backend uses Docker SDK `get_archive` to extract the named file on demand — never mounts the agent workspace.
+
### Platform Settings (3 endpoints)
| Method | Path | Description |
@@ -994,6 +1014,43 @@ CREATE INDEX idx_shared_folders_consume ON agent_shared_folder_config(consume_en
- Container recreation on restart when mount config changes
- Volume ownership automatically fixed to UID 1000
+**agent_shared_files:** (FILES-001 — Outbound File Sharing, NEW: 2026-04-24)
+```sql
+CREATE TABLE agent_shared_files (
+ id TEXT PRIMARY KEY, -- UUID
+ agent_name TEXT NOT NULL,
+ filename TEXT NOT NULL, -- Display name in download
+ stored_filename TEXT NOT NULL, -- UUID filename under /data/agent-files/
+ size_bytes INTEGER NOT NULL,
+ mime_type TEXT, -- python-magic detected
+ download_token TEXT UNIQUE NOT NULL, -- secrets.token_urlsafe(32), 192-bit
+ created_by TEXT NOT NULL, -- Agent name (or user for admin-created)
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL, -- Default 7d
+ revoked_at TEXT, -- Set when manually revoked
+ one_time INTEGER DEFAULT 0, -- Deferred: one-time link mode (column retained for future)
+ consumed_at TEXT, -- Deferred
+ download_count INTEGER DEFAULT 0,
+ last_downloaded_at TEXT,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
+ ON DELETE CASCADE ON UPDATE CASCADE
+);
+CREATE INDEX idx_agent_files_agent ON agent_shared_files(agent_name);
+CREATE INDEX idx_agent_files_token ON agent_shared_files(download_token);
+CREATE INDEX idx_agent_files_expires ON agent_shared_files(expires_at) WHERE revoked_at IS NULL;
+-- Also: agent_ownership.file_sharing_enabled INTEGER DEFAULT 0
+```
+
+**Outbound File Sharing Features:**
+- Per-agent opt-in via `agent_ownership.file_sharing_enabled`
+- Publish dir is a Docker volume `agent-{name}-public` mounted at `/home/developer/public/` inside the agent
+- Backend stores extracted bytes at `/data/agent-files/{file_id}` (under existing `trinity-data` volume — no compose changes)
+- Agent extracts via Docker SDK `get_archive` on demand — backend never mounts the agent workspace (filesystem-isolated blast radius)
+- Query param is `?sig={token}` (NOT `?download_token=`) to avoid the credential sanitizer's `.*TOKEN.*` pattern redacting it in agent transcripts
+- URL format: `{public_chat_url}/api/files/{file_id}?sig={token}` — uses existing `/api/*` proxy rules on Vite dev + prod nginx
+- FK has `ON UPDATE CASCADE` + `ON DELETE CASCADE` (aspirational — platform doesn't `PRAGMA foreign_keys=ON`; the agent delete handler + `rename_agent()` manually cascade as is the platform convention)
+- Manually cascaded in: `routers/agents.py` delete handler (rows + on-disk files + volume), `db/agent_settings/metadata.py:rename_agent` (updates `agent_name` in 17 tables)
+
**agent_event_subscriptions:** (EVT-001 - Agent Event Pub/Sub)
```sql
CREATE TABLE agent_event_subscriptions (
@@ -1448,9 +1505,17 @@ Agent starts → If .credentials.enc exists → Decrypt → Write files
Internal endpoints (`/api/internal/`) used by the scheduler and agent containers require shared-secret authentication via `X-Internal-Secret` header. Falls back to `SECRET_KEY` if `INTERNAL_API_SECRET` env var is not set.
-### WebSocket Security (C-002)
+### WebSocket Security (C-002, #550)
+
+The `/ws` endpoint uses **single-use opaque tickets** instead of a JWT in the URL. Browser flow:
+
+1. Authenticated client `POST /api/ws/ticket` (JWT in `Authorization` header) → backend mints a 32-byte urlsafe ticket, stores it in Redis with a 30s TTL, and returns it.
+2. Client connects to `/ws?ticket=`. Backend atomically `GETDEL`s the Redis key (Redis 6.2+) — single-use — resolves it to the authenticated subject, and only then accepts the WebSocket.
+3. Reconnects re-mint a fresh ticket; the JWT never enters the WebSocket URL.
+
+This closes the JWT-leak surface flagged by the April 2026 remediation pentest (finding 3.2.1): nginx access logs, browser history, and upstream proxies no longer see the JWT. CSWSH is mitigated because the ticket endpoint requires the JWT in an `Authorization` header — a malicious page can't mint a ticket on the victim's behalf without an explicit cross-origin request, which CORS rejects. Implementation lives in `services/ws_ticket_service.py` + `routers/ws_tickets.py`.
-The `/ws` endpoint requires JWT authentication. Token provided via `?token=` query parameter or as first message (`Bearer `, 5s timeout). Unauthenticated connections are rejected. The `/ws/events` endpoint requires MCP API key authentication (unchanged).
+The `/ws/events` endpoint still uses `?token=trinity_mcp_xxx` (MCP API key) for compatibility with documented external scripts (`websocat`, `wscat`); MCP keys are scoped, named, and revocable so the leak surface is bounded relative to a JWT.
**Reconnect replay (RELIABILITY-003, #306):** Both `/ws` and `/ws/events` accept an optional `?last-event-id=` query param. The value is regex-gated (`^\d+-\d+$`) by `validate_last_event_id()` in `services/event_bus.py` before reaching `XRANGE`; malformed input is ignored (no catchup). Catchup is capped at `REPLAY_GAP_LIMIT=5000` entries — a larger gap returns `{"type": "resync_required", "reason": "gap_too_large"}` instead of an unbounded `XRANGE`. Authorization (`accessible_agents` for `/ws/events`) is re-applied on replay, not just on live fan-out.
diff --git a/docs/memory/feature-flows.md b/docs/memory/feature-flows.md
index ce1f606b5..46d7212ff 100644
--- a/docs/memory/feature-flows.md
+++ b/docs/memory/feature-flows.md
@@ -11,12 +11,21 @@
| Date | ID | Feature | Flow |
|------|-----|---------|------|
+| 2026-04-29 | #584 | feat(slack): UI + API to change Slack DM-default agent — `set_slack_dm_default()` DB method (single-tx clear-then-set), `PUT /api/agents/{name}/slack/channel/dm-default` (owner-only, audit-logged), "Make default" button + tooltip in `SlackChannelPanel.vue`, unbind refuses 409 when target is DM default with siblings remaining | [slack-channel-routing.md](feature-flows/slack-channel-routing.md) |
+| 2026-04-30 | #598 | sec: AISEC-C2 Layer 2 — restored `.mcp.json` post-deploy editing via structure validation (`services.mcp_validator`). Closed schema, command/transport allowlists, SSRF guard for http/sse, reserved env-ref blocklist, literal-secret detection. 88 unit tests + 22 integration tests. UI placeholder updated; `trinity` server name reserved. | [credential-injection.md](feature-flows/credential-injection.md) |
+| 2026-04-30 | #590 | sec: AISEC-C2 Layer 1 — backend `ALLOWED_CREDENTIAL_PATHS` tightened; backend `update_agent_file_logic` adds defense-in-depth deny check before proxy; agent-server `EDIT_PROTECTED_PATHS` adds `.mcp.json` and `.credentials.enc`. | [credential-injection.md](feature-flows/credential-injection.md), [file-browser.md](feature-flows/file-browser.md) |
+| 2026-04-30 | #364 | Web chat file upload — drag-drop/picker in ChatPanel and PublicChat; base64 JSON encoding; shared upload_service; images via vision blocks, non-images via Docker put_archive | [web-chat-file-upload.md](feature-flows/web-chat-file-upload.md) |
+| 2026-04-27 | #539 | fix: public chat context duplication — `build_public_chat_context()` now called before `add_public_chat_message(role="user")`, preventing current message appearing twice in every agent prompt | [public-agent-links.md](feature-flows/public-agent-links.md) |
+| 2026-04-26 | #428 | CapacityManager facade — single public surface for capacity (admit / release / overflow policy / status / reclaim) replacing ExecutionQueue + SlotService + BacklogService trio | [capacity-management.md](feature-flows/capacity-management.md) |
| 2026-04-26 | #516, #520 | Agent error classification — `_classify_signal_exit()` (504 for SIGINT/SIGKILL/SIGTERM, was misread as 503 auth) and `_classify_empty_result()` (502 when clean exit drops the final result message, was silent 200 + watchdog reap) | [parallel-headless-execution.md](feature-flows/parallel-headless-execution.md), [task-execution-service.md](feature-flows/task-execution-service.md) |
| 2026-04-26 | #498 | Sync `/task` long-poll on backlog — sync parallel calls at capacity now spill to BACKLOG-001 (same backlog as async) and long-poll the open HTTP connection until terminal status (cap `2 × effective_timeout`); new `services/sync_waiter.py` owns the in-process registry + event/poll-fallback wait helper | [persistent-task-backlog.md](feature-flows/persistent-task-backlog.md), [parallel-headless-execution.md](feature-flows/parallel-headless-execution.md) |
+| 2026-04-24 | FILES-001 (#295) | Outbound file sharing — per-agent opt-in publish volume, `share_file` MCP tool, public download URL (`?sig=` token), UI panel with toggle/list/revoke. Agents publish files to `/home/developer/public/`; backend extracts via Docker SDK `get_archive` on demand; URL format `/api/files/{id}?sig={token}` | [file-sharing-outbound.md](feature-flows/file-sharing-outbound.md) |
| 2026-04-24 | WEBHOOK-001 (#291) | Webhook triggers — token-authenticated public URL fires schedule executions | [webhook-triggers.md](feature-flows/webhook-triggers.md) |
+| 2026-04-25 | #496 | Backlog drain spawn fix — repair `_spawn_drain` lazy import after #95 renamed `_execute_task_background` → `_run_async_task_with_persistence`; AST-based regression tests pin the contract | [persistent-task-backlog.md](feature-flows/persistent-task-backlog.md) |
| 2026-04-25 | #487 | Telegram file upload Phase 2 — workspace delivery hardening: NFKC sanitizer with collision dedup, spec injection format `[File uploaded by {uploader}]: {name} ({size}) saved to {path}`, all-writes-failed channel error + abort. Same code path benefits Slack inbound. | [telegram-integration.md](feature-flows/telegram-integration.md), [slack-file-sharing.md](feature-flows/slack-file-sharing.md) |
| 2026-04-23 | #476 | SQLite lexicographic cutoff bug fix — new `iso_cutoff(hours)` helper replaces `datetime('now', ...)` in 15 sites across rate-limit / dashboard / schedules; `max_retries` default flipped `1 → 0`; `cleanup_old_rate_limit_events` wired into `CleanupService` (phase 6, hourly) | [subscription-auto-switch.md](feature-flows/subscription-auto-switch.md), [cleanup-service.md](feature-flows/cleanup-service.md), [scheduler-service.md](feature-flows/scheduler-service.md) |
| 2026-04-22 | #458 | `.gitignore` init fix — `initialize_git_in_container` now appends missing patterns instead of truncate-and-write; adds `.env`, `.env.*`, `.mcp.json` to the default list and runs for both `/home/developer` and legacy `/home/developer/workspace` (stops credential leak on first GitHub sync) | [github-repo-initialization.md](feature-flows/github-repo-initialization.md) |
+| 2026-04-22 | SCHED-COND-001 (#454) | Conditional schedule pre-check — backend `docker exec`s the template's executable `~/.trinity/pre-check` (language-agnostic, interpreter from shebang) before scheduler fires a cron chat; empty stdout + exit 0 records a skipped execution; fail-open; reuses `ExecutionStatus.SKIPPED` (no schema change, no HTTP edge from scheduler to agent) | [scheduler-pre-check.md](feature-flows/scheduler-pre-check.md) |
| 2026-04-21 | RELIABILITY-003 (#306) | WebSocket event bus on Redis Streams — replaces in-process broadcast with XADD/XREAD, adds reconnect replay via `?last-event-id=`, 3-failure eviction, MAXLEN trim (tunable) | [websocket-event-bus.md](feature-flows/websocket-event-bus.md) |
| 2026-04-20 | #420 | Scheduler sync loop fix — `update_schedule_run_times` no longer bumps `updated_at`, stopping the self-triggering re-register of every schedule per tick | [scheduler-service.md](feature-flows/scheduler-service.md) |
| 2026-04-20 | #418 | Inter-agent timeout honors per-agent `execution_timeout_seconds` — removed 600s hardcoded defaults in MCP `chat_with_agent`/`fan_out` tools and fan-out service; HTTP client ceiling bumped to platform max (7200s) | [fan-out.md](feature-flows/fan-out.md), [mcp-orchestration.md](feature-flows/mcp-orchestration.md), [parallel-headless-execution.md](feature-flows/parallel-headless-execution.md) |
@@ -145,6 +154,7 @@
| Parallel Headless Execution | [parallel-headless-execution.md](feature-flows/parallel-headless-execution.md) | Stateless parallel task execution via POST /task |
| Parallel Capacity | [parallel-capacity.md](feature-flows/parallel-capacity.md) | Per-agent parallel execution slot tracking |
| Persistent Task Backlog | [persistent-task-backlog.md](feature-flows/persistent-task-backlog.md) | SQLite-backed FIFO backlog for async tasks at capacity (BACKLOG-001) |
+| Capacity Management | [capacity-management.md](feature-flows/capacity-management.md) | Unified facade for per-agent execution capacity (#428) |
| Task Execution Service | [task-execution-service.md](feature-flows/task-execution-service.md) | Unified execution lifecycle for all task callers (EXEC-024) |
| Business Validation | [business-validation.md](feature-flows/business-validation.md) | Post-execution auditor verifies task completion (VALIDATE-001) |
| Fan-Out | [fan-out.md](feature-flows/fan-out.md) | Parallel task dispatch and result collection via semaphore (FANOUT-001) |
@@ -191,6 +201,7 @@
| Agent Permissions | [agent-permissions.md](feature-flows/agent-permissions.md) | Agent communication permissions |
| Agent Sharing | [agent-sharing.md](feature-flows/agent-sharing.md) | Cross-channel email allow-list (web/Slack/Telegram) with access policy and pending requests |
| Agent Shared Folders | [agent-shared-folders.md](feature-flows/agent-shared-folders.md) | File collaboration via shared volumes |
+| Outbound File Sharing | [file-sharing-outbound.md](feature-flows/file-sharing-outbound.md) | Agents publish files to public download URLs (FILES-001) |
| Agent Tags & System Views | [agent-tags.md](feature-flows/agent-tags.md) | Tagging and saved filters (ORG-001) |
| Tag Clouds | [tag-clouds.md](feature-flows/tag-clouds.md) | Visual grouping on Dashboard |
@@ -304,6 +315,12 @@
| Agents Page UI | [agents-page-ui-improvements.md](feature-flows/agents-page-ui-improvements.md) | Horizontal row tiles with success rate bars, filtering, responsive breakpoints |
| Alerts Page | [alerts-page.md](feature-flows/alerts-page.md) | Removed in #430 (process engine deletion; cost alerts were PE-only) |
+### File Management
+
+| Flow | Document | Description |
+|------|----------|-------------|
+| Web Chat File Upload | [web-chat-file-upload.md](feature-flows/web-chat-file-upload.md) | Drag-drop/picker for authenticated and public chat; shared upload_service (#364) |
+
### Chat & Sessions
| Flow | Document | Description |
diff --git a/docs/memory/feature-flows/capacity-management.md b/docs/memory/feature-flows/capacity-management.md
new file mode 100644
index 000000000..495164ca9
--- /dev/null
+++ b/docs/memory/feature-flows/capacity-management.md
@@ -0,0 +1,170 @@
+# Feature: Capacity Management (CapacityManager)
+
+## Overview
+
+Single public facade for per-agent execution capacity. Replaces the
+three-class pyramid `ExecutionQueue` + `SlotService` + `BacklogService`
+with one entry point: `services/capacity_manager.py`. Issue #428 (PR #527,
+Tier 2.5 of `docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md`).
+
+## Why one facade
+
+Three primitives had grown three different "is this agent free?" stories
+that had to stay in sync at every caller site. Routers were directly
+composing `ExecutionQueue.acquire` + `SlotService.acquire_slot` +
+`BacklogService.enqueue` and reasoning about which combination of return
+values meant "admitted vs queued vs reject vs 429." `CapacityManager`
+collapses that decision into a single `acquire(...)` call gated by an
+`overflow_policy` argument. `SlotService` and `BacklogService` survive as
+private internals (each has a distinct, well-tested job); `ExecutionQueue`
+is deleted — its N=1 count gate is now `SlotService`, and its in-memory
+FIFO is a Redis LIST owned inline by `CapacityManager`.
+
+## Public API
+
+All callers reach for `get_capacity_manager()` from
+`services/capacity_manager.py`. Full signatures live in that file (~480
+LOC); summary table:
+
+| Method | Purpose |
+|--------|---------|
+| `acquire(agent, exec_id, max_concurrent, *, overflow_policy, overflow_payload?, ...)` | Try to admit; on overflow, dispatch to chosen policy. Returns `AcquireResult{state, execution_id, queue_position?}`. Raises `CapacityFull` when at capacity AND overflow is unavailable/full. |
+| `release(agent, exec_id)` | Release a slot. In-memory queue is popped (bookkeeping); persistent backlog drains via internal slot-release callback. |
+| `release_if_matches(agent, exec_id)` | Watchdog-safe release: only releases if `exec_id` actually holds a slot. Returns `bool`. |
+| `get_status(agent, max_concurrent)` | `QueueStatus` for the `/api/agents/{name}/queue` endpoint (current + in-memory queue only; persistent backlog is exposed via executions endpoints). |
+| `get_all_states(agent_capacities)` | Bulk capacity meter for the dashboard. |
+| `get_slot_state(agent, max_concurrent)` | Per-slot detail for the agent_config router. |
+| `reclaim_stale(agent_timeouts?)` | Reclaim slots whose dynamic TTL has expired. Used by the cleanup watchdog. |
+| `force_release(agent)` | Emergency: clear ALL slots + the in-memory queue. Returns `ForceReleaseResult`. |
+| `clear_in_memory_queue(agent)` | Clear only the overflow queue (running executions untouched). |
+| `cancel_all_overflow(agent, reason)` | Cancel queued (in-memory + persistent) — used on agent deletion. Returns count of persistent cancellations. |
+| `run_maintenance(max_age_hours=24)` | Periodic: expire stale persistent rows + drain orphaned backlog. Called from `main.py` 60s loop. |
+
+`get_capacity_manager()` returns a process-wide singleton.
+`reset_capacity_manager()` exists for tests.
+
+## Overflow policies
+
+Three modes selected per call via the `overflow_policy` keyword:
+
+| Policy | Behavior at capacity | When to use |
+|--------|----------------------|-------------|
+| `reject` | Raises `CapacityFull(reason="rejected")`. | Internal callers that have already pre-acquired upstream — e.g., `TaskExecutionService` (the router admitted the slot; the service is just being defensive). |
+| `queue_in_memory` | LPUSH onto Redis LIST `agent:queue:{name}` bounded by `IN_MEMORY_DEPTH=3`. Returns `state="queued_in_memory"` with a 1-based `queue_position`. The caller still proceeds — the agent's Claude subprocess is the real serialization point; the queue exists for observability + crude rate limiting. Raises `CapacityFull(reason="in_memory_full")` at depth 3. | `/chat` (synchronous HTTP, short request, caller is blocked anyway). |
+| `queue_persistent` | Marks the pre-created `schedule_executions` row `status='queued'` with `queued_at` + `backlog_metadata`. Returns `state="queued_persistent"`. Caller should reply 202 Accepted; the drain reconstructs the request later. Raises `CapacityFull(reason="persistent_full")` if the backlog is at its configured depth. Requires `overflow_payload: PersistentTaskPayload`. | `/task` (async + sync long-poll variants). Restart-durable. |
+
+## End-to-end flow
+
+### `/chat` — short synchronous, in-memory queue
+
+`src/backend/routers/chat.py` (chat endpoint):
+
+1. Resolve `max_parallel_tasks` from agent ownership row.
+2. `await capacity.acquire(agent_name=..., execution_id=..., max_concurrent=N, overflow_policy="queue_in_memory", source=USER, message=...)`.
+3. On `state="admitted"` or `"queued_in_memory"`, proceed to call the agent container. (The in-memory queue position is informational; the agent serializes Claude subprocess execution itself.)
+4. On `CapacityFull(reason="in_memory_full")` → 429 to client.
+5. In `finally`: `await capacity.release(agent_name, execution_id)` — releases the slot AND pops the next in-memory bookkeeping entry.
+
+### `/task` async — at-capacity → backlog → drain on release
+
+`src/backend/routers/chat.py` (parallel task endpoint, async mode):
+
+1. Create `schedule_executions` row eagerly via `db.create_task_execution` so the caller has an `execution_id` to return.
+2. Build `PersistentTaskPayload(request, effective_timeout, user_id, ...)`.
+3. `await capacity.acquire(..., overflow_policy="queue_persistent", overflow_payload=payload)`.
+4. On `state="admitted"`: spawn the background task as usual.
+5. On `state="queued_persistent"`: return 202 with the existing `execution_id` — no work happens yet.
+6. On `CapacityFull(reason="persistent_full")` → 429 (backlog is also at depth).
+
+When ANY slot for that agent is released later (any caller, any policy), `CapacityManager._on_slot_released` fires (registered with `SlotService.register_on_release` in the constructor), which calls `BacklogService.drain_next(agent_name)`. The drain atomically claims one queued row and re-spawns the persisted request via `_run_async_task_with_persistence`. This is the path that survives a backend restart — the rows are durable; the orphan-drain in `run_maintenance()` resumes them on the next boot.
+
+The sync `/task` long-poll variant uses the same `queue_persistent` path and waits on `services/sync_waiter.py` for the eventual drain to flip the row to terminal state (#498).
+
+### Termination
+
+`src/backend/routers/chat.py` terminate endpoint calls
+`capacity.force_release(agent_name)` to clear all slots + the in-memory
+queue at once.
+
+## Storage map
+
+Keys/columns are intentionally unchanged from the predecessor classes so
+in-flight executions across the upgrade keep working.
+
+| Concern | Storage | Key / column |
+|---------|---------|--------------|
+| Active slot counter (per agent) | Redis ZSET | `agent:slots:{name}` (member = exec_id, score = unix ts) |
+| Per-slot metadata | Redis HASH | `agent:slot:{name}:{exec_id}` (auto-expires via dynamic TTL = `agent.timeout + 5min` buffer) |
+| In-memory overflow queue | Redis LIST | `agent:queue:{name}` (LPUSH new, RPOP oldest, depth ≤ `IN_MEMORY_DEPTH=3`) |
+| Persistent overflow backlog | SQLite | `schedule_executions` rows where `status='queued'` (driven by `queued_at` ASC for FIFO; `backlog_metadata` JSON holds the request to replay; partial index `idx_executions_queued`) |
+
+## Maintenance & recovery
+
+Two periodic loops keep capacity state honest:
+
+- **`main.py` 60s loop → `capacity.run_maintenance(max_age_hours=24)`** —
+ expires `status='queued'` rows older than 24h to FAILED, then calls
+ `_backlog.drain_orphans_all()` to resume any backlog rows that didn't
+ get a release callback (typically after a backend restart between
+ enqueue and drain).
+- **`services/cleanup_service.py` watchdog (5min tick)** — calls
+ `capacity.reclaim_stale(agent_timeouts={...})` to release slots whose
+ per-agent dynamic TTL has expired; uses `release_if_matches(agent, exec_id)` (TOCTOU-safe) when reconciling individual orphaned executions so it only releases slots the targeted execution actually holds.
+
+## What replaced what
+
+The CapacityManager facade is the only public surface. The earlier docs
+(`execution-queue.md`, `parallel-capacity.md`, `persistent-task-backlog.md`)
+now describe internal implementation details rather than independent caller
+APIs.
+
+| Old concept | CapacityManager equivalent |
+|-------------|----------------------------|
+| `ExecutionQueue.acquire(...)` (N=1 mutex + in-memory FIFO) | `acquire(..., overflow_policy="queue_in_memory")` with `max_concurrent=1` |
+| `ExecutionQueue.release(...)` | `release(...)` |
+| `ExecutionQueue.get_status(...)` | `get_status(...)` |
+| `ExecutionQueue.force_release(...)` | `force_release(...)` |
+| `SlotService.acquire_slot(...)` | `acquire(..., overflow_policy="reject")` |
+| `SlotService.release_slot(...)` | `release(...)` |
+| `SlotService.cleanup_stale_slots(...)` | `reclaim_stale(...)` |
+| `SlotService.get_slot_state(...)` / `get_all_slot_states(...)` | `get_slot_state(...)` / `get_all_states(...)` |
+| `SlotService.force_clear_slots(...)` | `force_release(...)` (combined with in-memory clear) |
+| `BacklogService.enqueue(...)` | `acquire(..., overflow_policy="queue_persistent", overflow_payload=...)` |
+| `BacklogService.drain_next(...)` | Internal: fired by SlotService release callback wired in `__init__` |
+| `BacklogService.expire_stale(...)` + `drain_orphans_all(...)` | `run_maintenance(...)` |
+| `BacklogService.cancel_all_backlog(...)` | `cancel_all_overflow(...)` (also clears in-memory) |
+| Manual wiring of `SlotService.register_on_release(BacklogService.drain_next)` in `main.py` | Done internally in `CapacityManager.__init__` |
+
+## Caller sites
+
+| Caller | Location | Policy |
+|--------|----------|--------|
+| `/chat` | `src/backend/routers/chat.py` | `queue_in_memory` |
+| `/task` async | `src/backend/routers/chat.py` | `queue_persistent` |
+| `/task` sync long-poll | `src/backend/routers/chat.py` (waits on `sync_waiter`) | `queue_persistent` |
+| Terminate endpoint | `src/backend/routers/chat.py` | `force_release` |
+| `TaskExecutionService` | `src/backend/services/task_execution_service.py` | `reject` (router pre-acquired) |
+| Cleanup watchdog | `src/backend/services/cleanup_service.py` | `reclaim_stale` + `release_if_matches` |
+| Maintenance tick | `src/backend/main.py` (60s loop) | `run_maintenance` |
+| `/api/agents/{name}/queue` | `src/backend/routers/agents.py` | `get_status` |
+| Dashboard capacity meter | agent_config router | `get_all_states`, `get_slot_state` |
+| Agent deletion | `src/backend/routers/agents.py` | `cancel_all_overflow` |
+
+## Issue references
+
+- **#428** — Tier 2.5 facade work (this flow). PR #527, branch `feature/428-capacity-manager`.
+- **CAPACITY-001** — `SlotService` (now internal). See [parallel-capacity.md](parallel-capacity.md).
+- **BACKLOG-001 (#260)** — `BacklogService` (now internal). See [persistent-task-backlog.md](persistent-task-backlog.md).
+- **EXEC-024** — `TaskExecutionService` consumer. See [task-execution-service.md](task-execution-service.md).
+- **TIMEOUT-001** — per-agent dynamic slot TTL.
+- `docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md` — Tier 2.5 (Simplification) plan; this issue is the cornerstone of "one capacity surface, three policies."
+
+## Related flows
+
+- [parallel-capacity.md](parallel-capacity.md) — `SlotService` internals (Redis ZSET counter, dynamic TTL). Now an internal-only module reachable through `CapacityManager`.
+- [persistent-task-backlog.md](persistent-task-backlog.md) — `BacklogService` internals (SQL row state machine, FIFO claim, drain). Now an internal-only module reachable through `CapacityManager`.
+- [execution-queue.md](execution-queue.md) — historical doc for the deleted `ExecutionQueue` class. Behavior preserved via `acquire(..., max_concurrent=1, overflow_policy="queue_in_memory")` + `release(...)`.
+- [task-execution-service.md](task-execution-service.md) — primary internal consumer (uses `reject` policy).
+- [parallel-headless-execution.md](parallel-headless-execution.md) — `/task` endpoint flow; the persistent path is the queue-overflow story.
+- [cleanup-service.md](cleanup-service.md) — uses `reclaim_stale` + `release_if_matches` for stale-slot recovery.
+- [execution-termination.md](execution-termination.md) — uses `force_release`.
diff --git a/docs/memory/feature-flows/cleanup-service.md b/docs/memory/feature-flows/cleanup-service.md
index 042a432fc..cb50be53f 100644
--- a/docs/memory/feature-flows/cleanup-service.md
+++ b/docs/memory/feature-flows/cleanup-service.md
@@ -1,5 +1,7 @@
# Feature: Cleanup Service (CLEANUP-001)
+> **Updated 2026-04-26 (#428):** Stale-slot reclaim and watchdog release now go through [`CapacityManager`](capacity-management.md) — `capacity.reclaim_stale(agent_timeouts)` replaces `slot_service.cleanup_stale_slots(...)` and `capacity.release_if_matches(agent, exec_id)` replaces the prior pair of `slot_service.release_slot` + `execution_queue.force_release_if_matches`. Recovery (`_recover_execution`) calls `capacity.release(...)`. `ExecutionQueue` is gone; the TOCTOU-safe match check lives on the new facade.
+
## Overview
Background service that periodically recovers stuck intermediate states. Includes active watchdog reconciliation (Issue #129) that checks agent process registries, recovers orphaned executions, auto-terminates timed-out executions, and releases capacity. Also marks stale executions, activities, and Redis slots as failed. Runs every 5 minutes with an immediate startup sweep.
diff --git a/docs/memory/feature-flows/credential-injection.md b/docs/memory/feature-flows/credential-injection.md
index 4cad471da..46a06dc00 100644
--- a/docs/memory/feature-flows/credential-injection.md
+++ b/docs/memory/feature-flows/credential-injection.md
@@ -432,7 +432,16 @@ async def decrypt_and_inject(request: InternalDecryptInjectRequest):
1. **Encryption Key**: AES-256-GCM key derived from `CREDENTIAL_ENCRYPTION_KEY` env var or JWT secret. The `GET /credentials/encryption-key` endpoint is admin-only (C-001, 2026-03-09).
2. **Agent Access Control**: All credential endpoints (`inject`, `export`, `import`, `status`) require `get_owned_agent_by_name` dependency — only agent owners and admins can manage credentials (M-006 upgraded to owner-only in #174, 2026-03-26).
-3. **File Path Allowlist** (pentest 3.2.6 / #183, 2026-03-27): The `inject` endpoint validates all file paths against `ALLOWED_CREDENTIAL_PATHS = {.env, .mcp.json, .mcp.json.template, .credentials.enc}`. Requests with any other path are rejected with HTTP 400. Enforced at both backend and agent layers (defense in depth).
+3. **File Path Allowlist + Content Validation** (pentest 3.2.6 / #183 + AISEC-C2 / #590 / #598):
+ - **Backend (user-facing) — path layer** — `ALLOWED_CREDENTIAL_PATHS = {.env, .credentials.enc, .mcp.json}`. `.mcp.json.template` stays blocked because the envsubst flow it feeds doesn't sanitize attacker JSON. Requests with any other path are rejected with HTTP 400.
+ - **Backend (user-facing) — content layer** (#598, Layer 2 of AISEC-C2 closure) — `.mcp.json` content is structure-validated by `services.mcp_validator.validate_mcp_config` before forwarding to the agent. Validates:
+ - Server names: `^[a-zA-Z0-9_-]{1,64}$`, `trinity` reserved (auto-injected entry — can't be clobbered)
+ - Stdio transport: `command` ∈ allowlist `{npx, uvx, python, python3, node, bun, deno, docker}`, no path separators, ASCII-only; args without shell metacharacters; no inline-exec flags (`-c`/`--eval`/`-p`/`eval`) as first positional
+ - http/sse transport: HTTPS only, no userinfo, hostname must NOT resolve to private/loopback/link-local (SSRF guard mirroring #179); header names from a small allowlist
+ - env values: `${VAR}` references with valid POSIX-shape names, NOT in `RESERVED_ENV_REFS` (PATH, LD_PRELOAD, PYTHONPATH, TRINITY_MCP_API_KEY, etc.); literal substrings checked for shell metachars and credential patterns from `guardrails-baseline.json`
+ - Closed schema: only `command/args/env/url/headers/type` keys allowed; only `mcpServers` at the root
+ - Bounded: 64KB content cap, 32 servers max, 64 args max, 4096-char env values
+ - **Agent-server (platform-internal)** — `ALLOWED_CREDENTIAL_PATHS = {.env, .mcp.json, .mcp.json.template, .credentials.enc}`. Intentionally broader so platform services (`template_service`, `credential_encryption`, `github_pat_propagation`) can write `.mcp.json` on behalf of the platform. Reachable only on the internal Docker network; cross-tenant exposure tracked separately in #589 (Redis no-auth).
4. **File Permissions**: All credential files written with 600 permissions (owner read/write only)
5. **Internal API**: `/api/internal/*` endpoints require `X-Internal-Secret` header (C-003, 2026-03-09)
6. **No Secret Logging**: Credential values never logged, only file names and counts
diff --git a/docs/memory/feature-flows/execution-queue.md b/docs/memory/feature-flows/execution-queue.md
index 6e46820b0..080a12001 100644
--- a/docs/memory/feature-flows/execution-queue.md
+++ b/docs/memory/feature-flows/execution-queue.md
@@ -1,5 +1,7 @@
# Feature: Execution Queue System
+> **⚠️ SUPERSEDED 2026-04-26 (#428):** `ExecutionQueue` has been deleted. Its responsibilities — N=1 serial gating and the in-memory FIFO overflow store — collapsed into the unified [`CapacityManager`](capacity-management.md) facade with `overflow_policy="queue_in_memory"`. The Redis LIST key (`agent:queue:{name}`) and the depth bound (3) are preserved for wire-format compatibility. New work should reference [`capacity-management.md`](capacity-management.md). The notes below are kept for historical context on the original race-condition fixes and observability behaviour.
+>
> **Updated**: 2026-03-21 - **Unified Capacity Tracking (#98)**: Chat executions (`/api/chat`) now acquire a capacity slot via `SlotService` in addition to using `ExecutionQueue`. This makes `SlotService` the single source of truth for agent load — the capacity meter reflects ALL execution types. The queue still enforces serial chat; the slot tracks resource usage. `force_release_agent_logic()` now also clears capacity slots. Termination also releases slots.
>
> **Previous (2026-03-18)** - **Execution Records for All /api/chat Calls (#96)**: Every call to `POST /api/agents/{name}/chat` now creates a `task_execution` DB record regardless of call source. `triggered_by` values: `"chat"` (UI), `"mcp"` (user via MCP), `"agent"` (agent-to-agent). The response always includes `execution.task_execution_id`. New headers accepted: `X-Via-MCP`, `X-MCP-Key-ID`, `X-MCP-Key-Name` for MCP attribution. **Default model changed** to `"sonnet"` in the Chat tab frontend (`AgentDetail.vue:511`) for subscription compatibility (#138).
diff --git a/docs/memory/feature-flows/execution-termination.md b/docs/memory/feature-flows/execution-termination.md
index e72fd9705..0a6f6815e 100644
--- a/docs/memory/feature-flows/execution-termination.md
+++ b/docs/memory/feature-flows/execution-termination.md
@@ -1,5 +1,7 @@
# Feature: Execution Termination
+> **Updated 2026-04-26 (#428):** Backend-side capacity cleanup after a successful terminate now calls [`CapacityManager.force_release(agent_name)`](capacity-management.md), which clears both the slot counter and the in-memory queue in one call. Replaces the prior two-step `execution_queue.force_release` + `slot_service.release_slot`.
+
## Overview
Execution Termination allows users to stop running Claude Code executions mid-flight by sending graceful signals (SIGINT) to the subprocess, with fallback to force kill (SIGKILL) if needed. This feature enables users to cancel long-running or stuck tasks without waiting for them to complete.
diff --git a/docs/memory/feature-flows/file-browser.md b/docs/memory/feature-flows/file-browser.md
index 904adfc92..91a80d7a9 100644
--- a/docs/memory/feature-flows/file-browser.md
+++ b/docs/memory/feature-flows/file-browser.md
@@ -363,8 +363,8 @@ The file browser feature uses a **thin router + service layer** architecture:
- `path` (query, required) - File path to update
- `body.content` (body, required) - New file content
-**Protected Paths**: Cannot edit `.trinity`, `.git`, `.gitignore`, `.env`, `.mcp.json.template`
-**Note**: `CLAUDE.md` and `.mcp.json` ARE editable since users need to modify them
+**Protected Paths** (#590, AISEC-C2): Cannot edit `.trinity`, `.git`, `.gitignore`, `.env`, `.mcp.json`, `.mcp.json.template`, `.credentials.enc`
+**Note**: `CLAUDE.md` IS editable (owners manage agent instructions). `.mcp.json` is no longer editable here — raw content defines executable tool commands; use the platform regenerate-from-template flow.
**Response**:
```json
@@ -542,17 +542,19 @@ PROTECTED_PATHS = [
- `path` (query, required) - File path to update
- `content` (body, required) - New file content
-**Edit-Protected Paths** (Line 169-175):
+**Edit-Protected Paths** (refreshed by #590 / AISEC-C2):
```python
-# CLAUDE.md and .mcp.json ARE editable since users need to modify them
EDIT_PROTECTED_PATHS = [
".trinity",
".git",
".gitignore",
".env",
- ".mcp.json.template",
+ ".mcp.json", # added #590 — raw injection = RCE-by-config
+ ".mcp.json.template", # ditto (envsubst doesn't sanitize attacker JSON)
+ ".credentials.enc", # added #590 — overwrite swaps encrypted backup
]
```
+**Defense in depth**: the backend `update_agent_file_logic` in `src/backend/services/agent_service/files.py` runs the same deny check (broader: also blocks `.ssh/*`, `.aws/*`, `/opt/trinity/*`, etc.) BEFORE proxying to the agent-server, so a future router/proxy gap can't bypass the agent-server's check.
**Business Logic**:
1. Validate path is within workspace
diff --git a/docs/memory/feature-flows/file-sharing-outbound.md b/docs/memory/feature-flows/file-sharing-outbound.md
new file mode 100644
index 000000000..70132b49c
--- /dev/null
+++ b/docs/memory/feature-flows/file-sharing-outbound.md
@@ -0,0 +1,312 @@
+# Feature: Outbound File Sharing (FILES-001)
+
+## Revision History
+
+| Date | Changes |
+|------|---------|
+| 2026-04-24 | Initial implementation (FILES-001 / #295). Steps 1–6 complete: schema, toggle + volume, internal share endpoint, public download, MCP tool, UI panel. |
+
+## Overview
+
+Agents publish files from their `/home/developer/public/` directory to a public download URL with a signed token and a 7-day default expiration. The URL works universally — web, Slack, Telegram, WhatsApp, email — replacing fragile per-channel upload patterns.
+
+## Requirement Reference
+
+- **Requirement**: §13.10 Outbound File Sharing (FILES-001)
+- **GitHub Issue**: #295
+- **Status**: ✅ Implemented 2026-04-24
+- **Pillar**: III (Persistent Memory / Delivery)
+
+## User Story
+
+As an agent user (web, Slack, Telegram, WhatsApp), I want the agent to produce a downloadable file I can retrieve from a URL, so that outputs like CSV reports, PDFs, exports, and generated assets don't need to be pasted as text or handled with per-channel upload APIs.
+
+## Entry Points
+
+- **MCP tool**: `share_file({ filename, display_name?, expires_in? })` — from inside any agent with file sharing enabled
+- **Owner UI**: Agent Detail → Sharing tab → File Sharing panel
+- **Owner API**: `POST /api/agents/{name}/shared-files`
+- **Agent-server path**: `POST /api/internal/agent-files/share` (agent-scoped internal call)
+- **Public download**: `GET /api/files/{file_id}?sig={token}`
+
+---
+
+## Architecture
+
+```
+┌─────────────── Agent container ───────────────┐
+│ │
+│ Claude Code runtime │
+│ │ │
+│ ▼ (MCP JSON-RPC) │
+│ trinity-mcp-server:8080 │
+│ │ │
+│ ▼ (Bearer: agent-scoped MCP key) │
+│ POST /api/agents/{name}/shared-files │
+│ │
+│ /home/developer/public/report.csv │
+│ (agent-{name}-public Docker volume, │
+│ mounted ONLY into the agent) │
+│ │
+└────────────────────────┬───────────────────────┘
+ │ Docker SDK get_archive
+ │ (backend never mounts
+ │ agent workspace)
+ ▼
+┌─────────────── Backend process ────────────────┐
+│ │
+│ services/agent_shared_files_service.py │
+│ ├── validate path (no abs, no .., no \) │
+│ ├── python-magic MIME + executable blocklist │
+│ ├── enforce per-agent quota │
+│ ├── shutil.disk_usage pre-check │
+│ └── write /data/agent-files/{file_id} │
+│ │
+│ db: insert into agent_shared_files │
+│ │
+└────────────────────────┬───────────────────────┘
+ │ response
+ ▼
+ URL: {public_chat_url}/api/files/{file_id}?sig={token}
+
+User clicks URL →
+ GET /api/files/{file_id}?sig={token}
+ ├── IP rate limit (file-download bucket)
+ ├── constant-time compare vs stored download_token
+ ├── revoked / expired checks
+ ├── agent require_email policy gate (via validate_agent_session)
+ ├── stream /data/agent-files/{file_id} (64 KB chunks)
+ ├── Content-Disposition: attachment (RFC 6266 UTF-8)
+ ├── X-Content-Type-Options: nosniff
+ ├── bump download_count + last_downloaded_at
+ └── audit: EXECUTION/file_share_download
+```
+
+---
+
+## Frontend Layer
+
+### Components
+
+| File | Line | Description |
+|------|------|-------------|
+| `src/frontend/src/components/FileSharingPanel.vue` | 1-210 | Toggle, restart-required banner, quota, table (filename/size/expires/downloads), Copy URL + Revoke buttons, empty state |
+| `src/frontend/src/components/SharingPanel.vue` | — | Embeds `` between Telegram/WhatsApp and Public Links sections |
+
+### State Management
+
+| Store method | File | Description |
+|--------------|------|-------------|
+| `getFileSharingStatus(name)` | `stores/agents.js` | GET `/api/agents/{name}/file-sharing` |
+| `setFileSharingStatus(name, enabled)` | `stores/agents.js` | PUT toggle |
+| `listSharedFiles(name)` | `stores/agents.js` | GET `/api/agents/{name}/shared-files` |
+| `revokeSharedFile(name, id)` | `stores/agents.js` | DELETE |
+
+No WebSocket updates yet — manual refresh on action (acceptable because volume is low and actions are local).
+
+---
+
+## Backend Layer
+
+### Architecture — three layers
+
+| Layer | File | Purpose |
+|-------|------|---------|
+| Router | `src/backend/routers/agent_files.py` (toggle + list/revoke) | GET/PUT `/file-sharing`, POST/GET/DELETE `/shared-files` |
+| Router | `src/backend/routers/files.py` (download) | GET `/api/files/{id}` |
+| Router | `src/backend/routers/internal.py` (share) | POST `/api/internal/agent-files/share` (agent-server path, `X-Internal-Secret` auth) |
+| Service | `src/backend/services/agent_shared_files_service.py` | `create_share()` orchestrator, path validation, MIME detection, quota, extraction |
+| Service | `src/backend/services/agent_service/file_sharing.py` | Toggle logic, `check_public_folder_mount_matches()` |
+| DB | `src/backend/db/agent_shared_files.py` | `AgentSharedFilesOperations` CRUD |
+| DB | `src/backend/db/agent_settings/file_sharing.py` | `FileSharingMixin` — per-agent toggle + volume name convention |
+
+### Endpoints
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| GET | `/api/agents/{name}/file-sharing` | JWT (access) | Status + quota bar data |
+| PUT | `/api/agents/{name}/file-sharing` | JWT (owner/admin) | Toggle; returns `restart_required: true` when config and mounts disagree |
+| POST | `/api/agents/{name}/shared-files` | JWT (owner/admin) OR agent-scoped MCP key (same agent) | Mint a download URL |
+| GET | `/api/agents/{name}/shared-files` | JWT (access) | List active shares |
+| DELETE | `/api/agents/{name}/shared-files/{file_id}` | JWT (owner/admin) | Revoke (idempotent) |
+| POST | `/api/internal/agent-files/share` | `X-Internal-Secret` | Agent-server direct path; takes `agent_name` in body |
+| GET | `/api/files/{file_id}` | Token (`?sig=`) + optional `session_token` when agent requires email | Public download |
+
+### MCP tool
+
+| Tool | File | Description |
+|------|------|-------------|
+| `share_file` | `src/mcp-server/src/tools/files.ts` | Agent-scoped. Body: `{ filename, display_name?, expires_in? }`. Returns: `{ file_id, url, expires_at, size_bytes, mime_type }` |
+
+### Key service behaviors
+
+| Behavior | Where | Notes |
+|----------|-------|-------|
+| Path validation | `validate_publish_path()` | Rejects absolute paths, `..` segments, backslashes. Resolves against `/home/developer/public/`. |
+| Extraction | `extract_from_agent()` | Docker SDK `get_archive`. Caps buffer at 50 MB + 4 KB tar overhead to prevent OOM. Rejects non-regular tar members (symlinks, dirs, devices). |
+| MIME detection | `detect_mime()` + `check_mime_blocklist()` | python-magic on first 4096 bytes. Blocklist: PE (`MZ`), ELF (`\x7fELF`), Mach-O (4 variants), `#!` shebang. |
+| Quota | `enforce_quota()` | Sum of non-revoked, non-expired `size_bytes` for the agent; default 500 MB. |
+| Token | `secrets.token_urlsafe(32)` | 192-bit entropy, stored in `download_token`. URL param name is `sig` (NOT `download_token`) to bypass the credential sanitizer's `.*TOKEN.*` pattern. |
+| URL build | `build_download_url()` | `{public_chat_url}/api/files/{id}?sig={token}` — uses existing `/api/*` proxy path on Vite dev + prod nginx, no new proxy rules needed. |
+
+---
+
+## Data Layer
+
+### Database Schema
+
+```sql
+CREATE TABLE agent_shared_files (
+ id TEXT PRIMARY KEY, -- UUID
+ agent_name TEXT NOT NULL,
+ filename TEXT NOT NULL, -- Display name
+ stored_filename TEXT NOT NULL, -- UUID on disk
+ size_bytes INTEGER NOT NULL,
+ mime_type TEXT,
+ download_token TEXT UNIQUE NOT NULL,
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL,
+ revoked_at TEXT,
+ one_time INTEGER DEFAULT 0, -- Deferred column (not used)
+ consumed_at TEXT, -- Deferred column (not used)
+ download_count INTEGER DEFAULT 0,
+ last_downloaded_at TEXT,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
+ ON DELETE CASCADE ON UPDATE CASCADE
+);
+CREATE INDEX idx_agent_files_agent ON agent_shared_files(agent_name);
+CREATE INDEX idx_agent_files_token ON agent_shared_files(download_token);
+CREATE INDEX idx_agent_files_expires ON agent_shared_files(expires_at) WHERE revoked_at IS NULL;
+
+-- Plus one column on the existing agent_ownership table:
+ALTER TABLE agent_ownership ADD COLUMN file_sharing_enabled INTEGER DEFAULT 0;
+```
+
+FK `ON UPDATE/DELETE CASCADE` is declared but not enforced at runtime (platform-wide pattern — SQLite connections don't `PRAGMA foreign_keys=ON`). Explicit cascades are in:
+- `routers/agents.py` delete handler — removes DB rows, on-disk files, and Docker volume
+- `db/agent_settings/metadata.py:rename_agent()` — updates `agent_name` when an agent is renamed
+
+### Storage locations
+
+| Item | Location |
+|------|----------|
+| DB rows | `agent_shared_files` table in `/data/trinity.db` |
+| Agent-side publish dir | `/home/developer/public/` (Docker volume `agent-{name}-public`, mounted only into the agent) |
+| Backend-side extracted bytes | `/data/agent-files/{file_id}` (under the existing `trinity-data` volume — no compose changes) |
+
+---
+
+## Docker Integration
+
+### Volume creation (crud.py + lifecycle.py)
+
+When `agent_ownership.file_sharing_enabled = 1`, the agent start flow:
+1. Creates Docker volume `agent-{name}-public` if missing
+2. Runs alpine `chown 1000:1000 /public` to fix ownership for the `developer` user
+3. Mounts volume at `/home/developer/public` (rw)
+
+Volume is created on first toggle-on + restart and removed on agent delete.
+
+### Backend storage
+
+`/data/agent-files/{file_id}` — flat directory inside the existing `trinity-data` volume. No compose changes needed in dev or prod.
+
+---
+
+## Security Properties
+
+See `docs/drafts/amazing-file-outbound.md` §6 for the full threat model. Key properties:
+
+| # | Threat | Mitigation |
+|---|--------|-----------|
+| S1 | Path traversal — `share_file("../.env")` | `validate_publish_path()` rejects absolute, `..`, backslash; Docker SDK `get_archive` extracts into isolated buffer; backend never mounts agent workspace |
+| S2 | Credential leak via backend filesystem reach | Backend only reads the single file the agent names; never `bind`-mounts `/home/developer/` |
+| S3 | Predictable tokens | 192-bit `secrets.token_urlsafe(32)`; constant-time compare via `secrets.compare_digest` |
+| S6 | XSS via agent-uploaded HTML | `Content-Disposition: attachment` + `X-Content-Type-Options: nosniff` — never inline |
+| S7 | Filename header injection (CRLF) | Sanitizer allows `[A-Za-z0-9._\- ]` only; RFC 6266 UTF-8 percent-encoding for non-ASCII |
+| S8 | MIME spoofing | python-magic detects actual MIME; blocklist rejects PE/ELF/Mach-O/shebang before storage |
+| S9 | Storage DoS | 50 MB per-file + 500 MB per-agent quota (setting-configurable) |
+| S10 | Token enumeration | 192-bit entropy + IP rate limit + audit log |
+| S11 | Cross-tenant download | File addressed by `file_id` only; agent_name resolved from DB row |
+| S14 | Access-policy bypass | Download endpoint runs the same `_agent_requires_email` gate as public chat; session_token validated via `validate_agent_session` (cross-link lookup) |
+| S15 | Agent impersonation via MCP | Backend enforces `current_user.agent_name == path agent_name` for agent-scoped keys (same-agent defense) |
+
+### Deferred / documented limitations
+
+- **One-time download links** deferred. Schema columns retained (`one_time`, `consumed_at`) for future re-enablement.
+- **Token in stored transcripts**: the URL (including `?sig=`) ends up in persisted chat_messages/schedule_executions for agents that include it in their response text. DB read-access allows URL reuse until expiration. Tracked for V1.1.
+- **Shared rate-limit bucket**: currently uses `check_public_link_rate_limit` which shares a bucket with public chat (Phase 1 C5 will split this).
+- **FK not runtime-enforced**: platform-wide pattern; manual cascade in agent delete + rename.
+
+---
+
+## Side Effects
+
+- Audit event `EXECUTION/file_share_download` per GET (logs IP, UA, file_id, size, MIME, target agent)
+- Download counter + `last_downloaded_at` bumped per download (best-effort; failures don't block the download)
+- Agent delete cascades: unlinks on-disk files, removes Docker volume, deletes DB rows
+
+---
+
+## Error Handling
+
+| Condition | HTTP status | Error body |
+|-----------|-------------|------------|
+| Missing `sig` | 401 | `sig required` |
+| Invalid `sig` | 401 | `invalid download_token` |
+| Unknown `file_id` | 404 | `not found` |
+| Revoked | 410 | `revoked` |
+| Expired | 410 | `expired` |
+| Missing session_token when required | 401 | `session_token required` |
+| Invalid session_token | 401 | `invalid or expired session_token` |
+| Storage file missing on disk | 500 | `storage error` (also logged as orphan row) |
+| Rate limit exceeded | 429 | `Too many requests. Please try again later.` |
+
+---
+
+## Testing
+
+### Prerequisites
+- Backend + frontend + mcp-server + agent container all running
+- Admin user logged in, test agent created
+
+### Unit tests
+```bash
+pytest tests/unit/test_agent_shared_files_migration.py \
+ tests/unit/test_file_sharing_mixin.py \
+ tests/unit/test_public_folder_mount_match.py -v
+```
+33 tests covering schema/migration, DB mixin, mount-match helper.
+
+### End-to-end (manual / shell script)
+
+See `docs/drafts/amazing-file-outbound.md` §7 (Steps 1–6) for the full 37-assertion live regression script.
+
+### Happy path sanity check
+```bash
+AGENT=filetest-$(date +%s)
+# 1) create + enable + restart agent
+# 2) agent drops a file in /home/developer/public/
+# 3) POST /api/agents/{name}/shared-files → get URL
+# 4) curl URL → byte-identical download
+# 5) DELETE agent → DB rows, on-disk files, Docker volume all gone
+```
+
+### Status: Live-verified via real Slack round-trip 2026-04-24
+
+---
+
+## Related Flows
+
+- **Upstream**: [public-agent-links.md](public-agent-links.md) — URL + policy-gate pattern we clone
+- **Upstream**: [agent-shared-folders.md](agent-shared-folders.md) — Docker volume pattern we clone
+- **Upstream**: [unified-channel-access-control.md](unified-channel-access-control.md) — `require_email` / `session_token` gate reused here
+- **Adjacent**: [agent-sharing.md](agent-sharing.md) — email allow-list that gates the download endpoint when `require_email=true`
+- **Related**: [audit-trail.md](audit-trail.md) — `file_share_download` events recorded here
+
+## Design References
+
+- Design doc: `docs/drafts/amazing-file-outbound.md`
+- Production readiness plan: `docs/drafts/amazing-file-outbound-production-readiness.md`
+- Phase 1 execution checklist: `docs/drafts/amazing-file-outbound-phase1-execution.md`
diff --git a/docs/memory/feature-flows/github-repo-initialization.md b/docs/memory/feature-flows/github-repo-initialization.md
index af5373480..214722ebb 100644
--- a/docs/memory/feature-flows/github-repo-initialization.md
+++ b/docs/memory/feature-flows/github-repo-initialization.md
@@ -460,12 +460,19 @@ async def initialize_git_in_container(
git_dir = "/home/developer/workspace" if workspace_has_content else "/home/developer"
# Step 2: Append any missing _GITIGNORE_PATTERNS entries to .gitignore
- # (issue #458). Runs for BOTH /home/developer and the legacy
+ # (issues #458 and #462). Runs for BOTH /home/developer and the legacy
# /home/developer/workspace path. The merge is idempotent — each
# pattern is gated by an exact-line `grep -qxF` check — so any
# workspace-supplied `.gitignore` (e.g. written by `/trinity:onboard`)
- # is preserved. Patterns: .bash_logout, .bashrc, .profile,
- # .bash_history, .cache/, .local/, .npm/, .ssh/, .env, .env.*, .mcp.json.
+ # is preserved.
+ #
+ # _GITIGNORE_PATTERNS is the canonical exclusion list (single source
+ # of truth) — see src/backend/services/git_service.py and the mirrored
+ # block in docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md. The list covers
+ # shell init, credentials, instance-specific dirs/files, Claude Code
+ # runtime data, temp files, and local overrides. The same merge runs
+ # again on every Push via _migrate_workspace_gitignore (#462) so
+ # existing agents pick up new entries without re-init.
execute_command_in_container(
command=_build_gitignore_merge_command(git_dir),
timeout=5,
@@ -1098,8 +1105,14 @@ sqlite3 ~/trinity-data/trinity.db "SELECT key, substr(value, 1, 10) || '...' FRO
# - Two branches: main, trinity/test-agent/xxxxxxxx
# - Commits on both branches
# - Agent files: CLAUDE.md, .claude/, .trinity/
-# - .gitignore excludes: .bashrc, .cache/, .ssh/, .env, .env.*, .mcp.json
-# - .env and .mcp.json are NOT in the initial commit (#458)
+# - .gitignore covers the full _GITIGNORE_PATTERNS list — see the canonical
+# constant in src/backend/services/git_service.py (also mirrored in
+# docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md). The list spans shell init,
+# credentials, instance-specific dirs/files, Claude Code runtime data,
+# temp files, and local overrides.
+# - .env, .mcp.json, and Claude Code runtime files (.claude/sessions,
+# .claude/shell-snapshots, .cache, .claude.json, etc.) are NOT in the
+# initial commit (#458 + #462)
```
**Verify in Database**:
@@ -1180,15 +1193,18 @@ sqlite3 ~/trinity-data/trinity.db "SELECT * FROM agent_git_config WHERE agent_na
**Expected**:
- System uses `/home/developer` directly (no workspace subdirectory)
- Backend logs: `Using home directory: /home/developer`
-- `.gitignore` contains system file exclusions plus `.env`, `.env.*`, `.mcp.json` (#458)
-- Agent files (CLAUDE.md, .claude/, .trinity/) pushed to GitHub
+- `.gitignore` contains the full `_GITIGNORE_PATTERNS` canonical list — credentials, instance dirs/files, Claude Code runtime data, temp files (#458 + #462)
+- Agent files (CLAUDE.md, .claude/commands/, .claude/skills/, .claude/agents/, .trinity/) pushed to GitHub
- `.env` and `.mcp.json` written by `inject_credentials` are excluded
+- Claude Code runtime (`.claude/sessions/`, `.claude/shell-snapshots/`, `.claude.json`, etc.) is excluded
- Pre-existing workspace `.gitignore` rules (e.g. from `/trinity:onboard`) are preserved
+- The same merge runs on every subsequent Push via `_migrate_workspace_gitignore` (#462), so existing agents pick up new patterns automatically
**LEGACY Case** (agents created before 2026-02):
- If `/home/developer/workspace/` exists with content, that directory is used
- Backend logs: `Using workspace directory: /home/developer/workspace`
- `.gitignore` merge also runs here (#458 — previously skipped)
+- Detection logic shared with `_detect_git_dir`, used by both init and the post-init Push migration (#462)
**Verify**:
- Check GitHub repo contains agent files but not system files
diff --git a/docs/memory/feature-flows/github-sync.md b/docs/memory/feature-flows/github-sync.md
index 2817754a1..510528e09 100644
--- a/docs/memory/feature-flows/github-sync.md
+++ b/docs/memory/feature-flows/github-sync.md
@@ -236,6 +236,8 @@ sequenceDiagram
User->>UI: Click "Sync" button
UI->>Backend: POST /api/agents/{name}/git/sync
+ Backend->>Container: docker exec — append missing _GITIGNORE_PATTERNS (#462)
+ Backend->>Container: docker exec — git rm --cached newly-ignored tracked files
Backend->>Container: POST /api/git/sync
Container->>Container: git add -A
Container->>Container: git commit -m "Trinity sync: {timestamp}"
@@ -244,6 +246,32 @@ sequenceDiagram
Backend->>UI: Show notification (or branch_ownership_collision modal)
```
+### Per-Push Gitignore Migration (#462)
+
+Before forwarding to the agent's `/api/git/sync`, the platform calls
+`_migrate_workspace_gitignore(agent_name)` in `services/git_service.py`.
+This runs two `docker exec` steps inside the agent container:
+
+1. **Append missing patterns**: `_build_gitignore_merge_command` idempotently
+ appends any `_GITIGNORE_PATTERNS` entries not already present in
+ `/home/developer/.gitignore`. Pre-existing user rules are preserved
+ (each pattern is gated by an exact-line `grep -qxF` check).
+2. **Untrack newly-ignored files**: `_build_rm_cached_ignored_command`
+ runs `git ls-files -ci --exclude-standard | xargs -0 git rm --cached`
+ so files that were force-tracked before a pattern was added (e.g.
+ `.cache/foo` from an old agent) are removed from the index. Working-tree
+ files are left on disk; only the index changes.
+
+The migration is **best effort** — failures are caught, logged at WARNING,
+and Push proceeds against whatever `.gitignore` already exists. Rationale:
+a transient docker exec glitch must not break an operator's Push.
+
+This means existing agents pick up new exclusion patterns automatically on
+their next Push, with no re-init or container rebuild required. The same
+shared `_detect_git_dir` helper used by `initialize_git_in_container`
+guarantees the migration targets the same path as init (`/home/developer`,
+or `/home/developer/workspace` for legacy pre-2026-02 agents).
+
---
## Recovery: Reset-to-Main-Preserve-State (S3, #384)
diff --git a/docs/memory/feature-flows/parallel-capacity.md b/docs/memory/feature-flows/parallel-capacity.md
index 5f6d6aa8e..e0374fd46 100644
--- a/docs/memory/feature-flows/parallel-capacity.md
+++ b/docs/memory/feature-flows/parallel-capacity.md
@@ -1,9 +1,11 @@
# Feature Flow: Parallel Execution Capacity
+> **⚠️ INTERNAL AS OF 2026-04-26 (#428):** `SlotService` is no longer a public capacity API. It is now a private internal of [`CapacityManager`](capacity-management.md), which is the single facade callers should use. The Redis ZSET (`agent:slots:{name}`), per-agent dynamic TTL, and atomic ZADD-with-count semantics described below are unchanged — they are the implementation backing `CapacityManager.acquire/release/reclaim_stale`. New callers should reach for [`capacity-management.md`](capacity-management.md) instead of importing `SlotService` directly.
+>
> **Requirement**: CAPACITY-001 - Per-Agent Parallel Execution Capacity
> **Status**: Implemented (Phase 1: Backend, Phase 2: Frontend UI)
> **Created**: 2026-02-28
-> **Updated**: 2026-03-04
+> **Updated**: 2026-04-26 (#428: SlotService internalized behind CapacityManager)
> **Priority**: P1
## Revision History
@@ -61,8 +63,8 @@ The frontend displays slot usage as a vertical capacity meter bar on the Agents
│ │ 1. Create execution record in database (chat.py:602-613) │ │
│ │ 2. Router acquires slot directly (chat.py:644-651) │ │
│ │ 3. If full → 429 response (chat.py:653-663) │ │
-│ │ 4. Spawn _run_async_task_with_persistence() with release_slot=True │ │
-│ │ 5. Background task releases slot in finally (chat.py:554-557) │ │
+│ │ 4. Spawn _run_async_task_with_persistence() (router pre-acquired)│ │
+│ │ 5. TaskExecutionService releases slot in finally (slot_already_held=True)│
│ │ │ │
│ │ PUBLIC path (public.py:315-322 → task_execution_service.py): │ │
│ │ 1. Delegate to TaskExecutionService.execute_task() │ │
@@ -137,7 +139,7 @@ POST /api/agents/{name}/task (async)
│ 1. db.get_max_parallel_tasks(name) │
│ 2. slot_service.acquire_slot(...) │
│ 3. If not acquired → 429 Too Many Requests │
- │ 4. Spawn background task with release_slot=True │
+ │ 4. Spawn `_run_async_task_with_persistence()` │
└─────────────────────────────────────────────────┘
```
diff --git a/docs/memory/feature-flows/parallel-headless-execution.md b/docs/memory/feature-flows/parallel-headless-execution.md
index 553f2b740..48fd96c3e 100644
--- a/docs/memory/feature-flows/parallel-headless-execution.md
+++ b/docs/memory/feature-flows/parallel-headless-execution.md
@@ -3,13 +3,14 @@
> **Requirement**: 12.1 - Parallel Headless Execution
> **Status**: Implemented
> **Created**: 2025-12-22
-> **Updated**: 2026-04-26 (Issue #520 empty-result detection)
+> **Updated**: 2026-04-27 (#531: `drain_reader_threads` reordered to preserve kernel pipe buffer before close)
> **Verified**: 2026-02-05
## Revision History
| Date | Changes |
|------|---------|
+| 2026-04-27 | **Issue #531 - drain_reader_threads closes stdout pipe before reader can drain backlog**: Root cause of the HTTP 502 "Execution completed without a result message" on long agentic tasks (>5 min, dozens of tool calls). In `drain_reader_threads` (`subprocess_pgroup.py`), the old sequence was: kill grandchildren → `safe_close_pipes()` → join reader (2s). Closing the read end of the pipe before the reader finished discarded the kernel pipe buffer — including the final `{"type":"result"}` JSON line written by claude just before exit. This caused `metadata.cost_usd` and `metadata.duration_ms` to stay `None`, which #521's `_classify_empty_result` correctly identified as a 502. Fix: reorder to kill grandchildren → join reader with `post_kill_grace=30s` (natural drain: grandchildren dead → kernel delivers EOF → reader sees buffered tail → exits cleanly) → `safe_close_pipes()` only as a last resort for genuine wedges. Also: `_classify_empty_result(metadata, raw_message_count, raw_messages)` now accepts `raw_messages` list and derives `num_turns` from it when `metadata.num_turns is None` (which is the case when the result line is lost), so the 502 detail shows an honest turn count. `metadata.tool_count` is accumulated per-message during parsing (not from the result line) and remains reliable. New parameter `post_kill_grace=30` added to `drain_reader_threads`; all call sites use the default. Regression test: `test_buffered_data_preserved_after_grandchild_kill` in `tests/unit/test_subprocess_pgroup.py`. Related: #520 (symptom), #521 (502 classification), #523 (FD_CLOEXEC complement, future). |
| 2026-04-26 | **Issue #520 - Empty-result misreported as success**: When the claude subprocess exits cleanly (`return_code == 0`) but the final `{"type":"result"}` JSON line is dropped before the reader thread captures it (typical cause: a child subprocess inherited stdout, kept the pipe open past claude exit, the reader thread leaked, the pgroup unwind closed the pipe and the result line went with it), `metadata.cost_usd` and `metadata.duration_ms` stay `None`. The success path used to return HTTP 200 anyway — agent-server logged "completed successfully" while backend silently reaped the execution as an orphan minutes later, masking the real failure with a misleading "completed on agent but recovered by watchdog" message. Sibling of #516 (signal-kill path). Added `_classify_empty_result(metadata, raw_message_count)` (`claude_code.py:935-980`) consulted *after* the `return_code != 0` block (#516 + auth heuristics) and *before* response building; when both `cost_usd` and `duration_ms` are `None`, raises HTTP 502 with diagnostic context (tool count, turn count, raw_messages, cause hint). Backend's auth classifier only triggers on 503, so 502 flows through as plain FAILED with the helpful detail preserved — no backend changes. The two-field check is conservative: single-field nullability could be a Claude format quirk; both-None is a strong signal that the terminal `result` message never arrived. 9 unit tests in `tests/unit/test_empty_result_classification.py`. |
| 2026-04-26 | **Issue #516 - Signal kills misclassified as auth failure**: External signal terminations of the claude subprocess (timeout SIGKILL, OOM-kill, parent SIGTERM, operator cancel) used to fall into the auth-fallback heuristics at `claude_code.py:1262-1278` and surface as a misleading "Subscription token may be expired" 503 — same shape as #361 but a different exit path. Added `_classify_signal_exit(return_code, metadata)` helper (`claude_code.py:882-931`) consulted *before* the auth heuristics; it matches Python-native signal exits (`return_code < 0`) and shell-encoded forms (`130, 137, 143` for SIGINT/SIGKILL/SIGTERM) and raises HTTP 504 with a clear "killed by SIGKILL/SIGTERM/SIGINT — likely timeout, OOM, or operator cancel" message. Tightened the zero-token heuristic with `return_code > 0` so signal exits cannot reach it. Backend's `task_execution_service.py:542` only flags AUTH on 503, so 504 falls through to the generic FAILED path — no backend changes needed. The bug became routinely reproducible after #61 (PR #326) added backend-driven `terminate_execution_on_agent()` which makes signal-killed claude subprocesses the common case for any timeout. 21 unit tests in `tests/unit/test_signal_exit_classification.py`. |
| 2026-04-26 | **Issue #498 - Sync long-poll on backlog**: Sync `/task` calls (`async_mode=false`) at capacity used to fail terminally with HTTP 429. They now spill to the same persistent backlog (BACKLOG-001) the async path uses and long-poll on the open HTTP connection until the queued execution reaches a terminal status. Total connection hold capped at `2 × effective_timeout` (queue wait + execution). Router (`chat.py:1007-1144`) pre-acquires the slot mirroring the async path, then on at-capacity calls `backlog.enqueue()` followed by `wait_for_sync_terminal()`. New module `services/sync_waiter.py` owns the in-process registry, `signal_sync_waiter()`, and `wait_for_sync_terminal()` (event + 5s DB-poll fallback). The drain reuses `_run_async_task_with_persistence` unchanged — it now calls `signal_sync_waiter` from its `finally` block to wake any sync waiter. See [persistent-task-backlog.md](persistent-task-backlog.md) for the full backlog flow. |
@@ -561,9 +562,14 @@ Note: Async mode on the `/api/agents/{name}/task` endpoint does **not** use `Tas
class ParallelTaskRequest(BaseModel):
# ... existing fields ...
async_mode: Optional[bool] = False # If true, return immediately with execution_id
+ files: Optional[List[WebFileUpload]] = None # File attachments (#364)
```
-**Background Task Handler** (`src/backend/routers/chat.py:438-650`):
+**File upload** (#364): `execute_parallel_task` decodes files synchronously (before async/sync fork)
+via `upload_service.process_file_uploads()`. Images become vision blocks in `execute_task(images=)`;
+text files are written to `/home/developer/uploads/{user_id}/` before the execution ID is returned.
+
+**Background Task Handler** (`src/backend/routers/chat.py:608-800`):
```python
async def _run_async_task_with_persistence(
agent_name: str,
@@ -576,6 +582,7 @@ async def _run_async_task_with_persistence(
subscription_id: Optional[str] = None,
is_self_task: bool = False,
self_task_activity_id: Optional[str] = None,
+ images: Optional[list] = None, # (#364) pre-decoded vision blocks
):
"""
Async /task background wrapper (issue #95).
diff --git a/docs/memory/feature-flows/persistent-task-backlog.md b/docs/memory/feature-flows/persistent-task-backlog.md
index 8c8b84aa0..ca9b83cb9 100644
--- a/docs/memory/feature-flows/persistent-task-backlog.md
+++ b/docs/memory/feature-flows/persistent-task-backlog.md
@@ -1,11 +1,14 @@
# Feature Flow: Persistent Task Backlog
+> **⚠️ INTERNAL AS OF 2026-04-26 (#428):** `BacklogService` is no longer a public API. It is the persistent overflow store inside [`CapacityManager`](capacity-management.md), reached via `acquire(..., overflow_policy="queue_persistent", overflow_payload=PersistentTaskPayload(...))`. The SQL columns (`schedule_executions.queued_at`, `backlog_metadata`, status `'queued'`), drain-on-release behaviour, 24h expiry, and partial index are unchanged. New callers should reach for [`capacity-management.md`](capacity-management.md) instead of importing `BacklogService` directly.
+>
> **Requirement**: BACKLOG-001 — Persistent task backlog for over-capacity requests
> **Status**: Implemented
> **Created**: 2026-04-13
-> **GitHub Issue**: [#260](https://github.com/abilityai/trinity/issues/260), extended by [#498](https://github.com/abilityai/trinity/issues/498) (sync long-poll)
+> **Updated**: 2026-04-26 (#428: BacklogService internalized behind CapacityManager)
+> **GitHub Issue**: [#260](https://github.com/abilityai/trinity/issues/260), extended by [#498](https://github.com/abilityai/trinity/issues/498) (sync long-poll), internalized by [#428](https://github.com/abilityai/trinity/issues/428)
> **Priority**: P1
-> **Related**: [parallel-capacity.md](parallel-capacity.md), [task-execution-service.md](task-execution-service.md), [parallel-headless-execution.md](parallel-headless-execution.md)
+> **Related**: [capacity-management.md](capacity-management.md), [parallel-capacity.md](parallel-capacity.md), [task-execution-service.md](task-execution-service.md), [parallel-headless-execution.md](parallel-headless-execution.md)
## Overview
diff --git a/docs/memory/feature-flows/public-agent-links.md b/docs/memory/feature-flows/public-agent-links.md
index 59177d967..bae2e40e8 100644
--- a/docs/memory/feature-flows/public-agent-links.md
+++ b/docs/memory/feature-flows/public-agent-links.md
@@ -1,6 +1,6 @@
# Feature Flow: Public Agent Links (12.2)
-**Last Updated**: 2026-04-13
+**Last Updated**: 2026-04-29
## Overview
@@ -37,13 +37,15 @@ Public Agent Links allow agent owners to generate shareable URLs that enable una
| Backend API |
+-------------------------------------------------------------------+
| routers/public_links.py routers/public.py |
-| (Authenticated) (Unauthenticated) |
+| (Authenticated) (Unauthenticated + JWT optional) |
| |
| - CRUD endpoints - Link validation |
| - Owner verification - Email verification |
| - Usage stats - Public chat (async mode) |
| - SSE stream proxy (THINK-001) |
| - Execution status polling |
+| - Session list (JWT, #587) |
+| - Session detail (JWT, #587) |
+-------------------------------------------------------------------+
|
+------------------------+------------------------+
@@ -272,6 +274,8 @@ import { getStatusFromStreamEvent, MIN_LABEL_DISPLAY_MS, HEARTBEAT_TIMEOUT_MS }
| `GET /api/public/intro/{token}` | `public.py:374` | `get_agent_intro()` |
| `GET /api/public/executions/{token}/{execution_id}/stream` | `public.py:676` | `public_stream_execution()` (THINK-001 SSE proxy) |
| `GET /api/public/executions/{token}/{execution_id}/status` | `public.py:735` | `public_execution_status()` (THINK-001 polling) |
+| `GET /api/public/sessions/{token}` | `public.py` | `list_public_sessions()` — JWT required; returns caller's last 20 sessions for this agent link with `preview` field (#587) |
+| `GET /api/public/sessions/{token}/{session_id}` | `public.py` | `get_public_session()` — JWT required; returns session detail with messages; validates session belongs to caller and correct agent (#587) |
### Database Operations
@@ -997,11 +1001,12 @@ Determine session identifier:
db.get_or_create_public_chat_session(link_id, identifier, type)
|
v
-db.add_public_chat_message(session_id, "user", message)
- |
- v
db.build_public_chat_context(session_id, message, max_turns=10)
-> "Previous conversation:\nUser: ...\nAssistant: ...\n\nCurrent message:\nUser: ..."
+ NOTE: context is built BEFORE the user message is stored (#539 fix).
+ |
+ v
+db.add_public_chat_message(session_id, "user", message)
|
v
TaskExecutionService.execute_task(triggered_by="public")
@@ -1144,18 +1149,18 @@ class PublicChatMessage(BaseModel):
- `build_public_chat_context()`
- `delete_public_link_sessions()`
-**Chat Endpoint** (`routers/public.py:215-362`):
-1. Validate link token (line 231)
-2. Determine session identifier (lines 235-263)
+**Chat Endpoint** (`routers/public.py`):
+1. Validate link token
+2. Determine session identifier
- Email links: validate session_token, extract email
- Anonymous links: use provided session_id or generate new
-3. Rate limit check (lines 265-271)
-4. Check agent availability (lines 273-279)
-5. Get or create session (lines 284-288)
-6. Store user message (lines 290-295)
-7. Record usage (lines 297-302)
-8. Build context-enriched prompt (lines 304-309)
-9. **Execute via `TaskExecutionService.execute_task(triggered_by="public")`** (lines 311-322)
+3. Rate limit check
+4. Check agent availability
+5. Get or create session
+6. **Build context-enriched prompt** (#539: must happen BEFORE storing user message to avoid duplication)
+7. Store user message
+8. Record usage
+9. **Execute via `TaskExecutionService.execute_task(triggered_by="public")`**
- Creates `schedule_executions` record
- Acquires capacity slot (returns 429 if at capacity)
- Tracks activity start (Dashboard timeline)
@@ -1771,10 +1776,186 @@ Summarization is triggered every 5th message per `(agent_name, user_email)` pair
---
+## Chat History for Logged-In Users (#587)
+
+**Status**: Implemented (2026-04-29)
+
+Logged-in Trinity users visiting a public chat link can browse and replay their own past chat sessions. Anonymous users see no change to the existing UI.
+
+### Design Principles
+
+- The public link token remains the agent-access credential; the JWT identifies which user's sessions to return.
+- Past sessions are opened in **read-only mode** — the chat input is hidden to prevent false continuity with the live agent context.
+- Access does not require `agent_sharing` membership; token + JWT is sufficient.
+- Only sessions created while logged in are visible ("Logged-in chats only" — anonymous sessions are excluded).
+
+### Data Flow
+
+```
+Logged-in user opens /chat/{token}
+ |
+ v
+PublicChat.vue checks authStore.isAuthenticated
+ -> true: renders ChatHistoryDropdown in header
+ |
+ v
+User clicks history button
+ |
+ v
+ChatHistoryDropdown fetches
+ GET /api/public/sessions/{token}
+ Authorization: Bearer
+ |
+ v
+Backend: validate token -> resolve agent_name
+ SELECT last 20 chat_sessions WHERE
+ agent_name = ? AND user_id = ?
+ ORDER BY last_message_at DESC
+ Includes preview (last message snippet, 120 chars)
+ |
+ v
+Dropdown renders session list
+ - Formatted date (Today / Yesterday / 3d ago / Apr 5)
+ - Message count
+ - Preview snippet
+ |
+ v
+User clicks a session
+ |
+ v
+ChatHistoryDropdown fetches
+ GET /api/public/sessions/{token}/{session_id}
+ Authorization: Bearer
+ |
+ v
+Backend: validate token -> resolve agent_name
+ Verify session.agent_name == agent_name
+ Verify session.user_id == current_user.id
+ Return session with messages
+ |
+ v
+PublicChat.vue: handleHistorySessionSelected({ messages, session })
+ viewingHistorySession = session
+ messages loaded in read-only mode
+ ChatInput hidden
+ Amber banner shown: "Viewing past session — Return to current chat"
+ |
+ v
+User clicks "Return to current chat"
+ |
+ v
+exitHistoryView():
+ viewingHistorySession = null
+ reload current live session or fetchIntro()
+ ChatInput restored
+```
+
+### Backend Implementation
+
+**Two new endpoints in `src/backend/routers/public.py`:**
+
+`GET /api/public/sessions/{token}` — list sessions:
+1. Validate public link token (same `is_link_valid()` check used throughout the module).
+2. Resolve `agent_name` from the link row.
+3. Require JWT (`current_user` dependency — not optional).
+4. Query `chat_sessions` for rows matching `agent_name` and `user_id`, ordered by `last_message_at DESC`, limit 20.
+5. For each session, compute `preview` by reading the most recent `chat_messages` row (`role = 'assistant'` preferred) and truncating to 120 chars.
+6. Return list of session dicts (id, started_at, last_message_at, message_count, preview).
+
+`GET /api/public/sessions/{token}/{session_id}` — session detail:
+1. Validate public link token.
+2. Resolve `agent_name`.
+3. Require JWT.
+4. Fetch session by `session_id`; return 404 if not found.
+5. Assert `session.agent_name == agent_name` and `session.user_id == current_user.id`; return 403 otherwise.
+6. Fetch all messages for the session ordered by `timestamp ASC`.
+7. Return session metadata + messages array.
+
+**No new database tables or migrations.** Both endpoints reuse the existing `chat_sessions` and `chat_messages` tables (documented in the main Database Schema section of `architecture.md`).
+
+### Frontend Layer
+
+#### New Component: `ChatHistoryDropdown.vue`
+
+**File**: `src/frontend/src/components/chat/ChatHistoryDropdown.vue`
+
+| Prop / Emit | Type | Description |
+|-------------|------|-------------|
+| `token` prop | String | Public link token (used in API calls) |
+| `session-selected` emit | `{ messages, session }` | Fired when user selects a session |
+
+**Key behaviors:**
+- On open: fetches `GET /api/public/sessions/{token}` using `authStore.authHeader`.
+- Renders a dropdown list of sessions, each showing formatted date, message count, and preview snippet.
+- Click-outside handler closes the dropdown.
+- Date formatting: "Today", "Yesterday", "3d ago", or locale short date (e.g., "Apr 5").
+
+**Imports** (added to `src/frontend/src/components/chat/index.js`):
+```javascript
+export { default as ChatHistoryDropdown } from './ChatHistoryDropdown.vue'
+```
+
+#### Changes to `PublicChat.vue`
+
+**New imports:**
+```javascript
+import { useAuthStore } from '../stores/auth'
+import { ChatHistoryDropdown } from '../components/chat'
+```
+
+**New state:**
+```javascript
+const authStore = useAuthStore()
+const viewingHistorySession = ref(null) // non-null = read-only history mode
+```
+
+**New methods:**
+
+| Method | Description |
+|--------|-------------|
+| `handleHistorySessionSelected({ messages, session })` | Sets `viewingHistorySession`, replaces displayed messages with the past session's messages |
+| `exitHistoryView()` | Clears `viewingHistorySession`; reloads current live session via `loadHistory()` or falls back to `fetchIntro()` |
+
+**Template changes:**
+- Header: ` `
+- Amber read-only banner: shown when `viewingHistorySession` is set; includes "Return to current chat" button that calls `exitHistoryView()`.
+- `` wrapped in `v-if="!viewingHistorySession"` — hidden during history replay.
+
+### Access Control Summary
+
+| Condition | Sessions endpoint behavior |
+|-----------|---------------------------|
+| No JWT | 401 Unauthorized |
+| Valid JWT, valid token | Returns caller's own sessions only |
+| Valid JWT, wrong user's session_id | 403 Forbidden |
+| Valid JWT, session belongs to different agent | 403 Forbidden |
+
+### Error Handling
+
+| Error Case | HTTP Status | Notes |
+|------------|-------------|-------|
+| Invalid / disabled / expired public link token | 404 | Same as all other `/api/public/*` endpoints |
+| No JWT provided | 401 | `Depends(get_current_user)` rejects unauthenticated requests |
+| Session not found | 404 | Unknown `session_id` |
+| Session belongs to wrong user or agent | 403 | Ownership mismatch |
+
+### Files
+
+| File | Description |
+|------|-------------|
+| `src/backend/routers/public.py` | Two new endpoint handlers: `list_public_sessions()`, `get_public_session()` |
+| `src/frontend/src/components/chat/ChatHistoryDropdown.vue` | New dropdown component (new file) |
+| `src/frontend/src/components/chat/index.js` | Exports `ChatHistoryDropdown` |
+| `src/frontend/src/views/PublicChat.vue` | Imports `useAuthStore`, `ChatHistoryDropdown`; new `viewingHistorySession` ref; `handleHistorySessionSelected()`, `exitHistoryView()` methods; conditional header, banner, input |
+
+---
+
## Revision History
| Date | Changes |
|------|---------|
+| 2026-04-29 | **#587 Chat History for Logged-In Users**: Two new JWT-authenticated endpoints (`GET /api/public/sessions/{token}`, `GET /api/public/sessions/{token}/{session_id}`) in `public.py`. New `ChatHistoryDropdown.vue` component. `PublicChat.vue` gains `viewingHistorySession` ref, `handleHistorySessionSelected()`, `exitHistoryView()`, amber read-only banner, and hidden `ChatInput` while in history mode. No new DB tables — reuses `chat_sessions`/`chat_messages`. |
+| 2026-04-27 | **fix #539 Context duplication**: `build_public_chat_context()` was called AFTER `add_public_chat_message(role="user")`, causing the current user message to appear twice in every agent prompt (once in "Previous conversation:", once in "Current message:"). Fixed by swapping the call order — context built first from prior history, user message stored after. Added 6 unit tests in `tests/unit/test_public_chat_context.py`. Updated PUB-005 data flow and backend implementation step ordering to reflect correct call order. |
| 2026-02-19 | **CHAT-001 Shared Components Refactor**: PublicChat.vue now uses shared components from `components/chat/` (ChatMessages, ChatInput, ChatBubble, ChatLoadingIndicator). Shared with new ChatPanel.vue authenticated chat. Updated method line numbers, added Shared Chat Components section. File now 611 lines. |
| 2026-02-18 | **Tab consolidation**: Public Links tab removed from AgentDetail.vue. PublicLinksPanel now embedded within SharingPanel.vue (lines 82-83, 92), accessible via "Sharing" tab. Updated Entry Points, Components table, Frontend Files table, and Related Flows sections. |
| 2025-12-22 | Initial documentation |
diff --git a/docs/memory/feature-flows/scheduler-pre-check.md b/docs/memory/feature-flows/scheduler-pre-check.md
new file mode 100644
index 000000000..f941e20e2
--- /dev/null
+++ b/docs/memory/feature-flows/scheduler-pre-check.md
@@ -0,0 +1,140 @@
+# Feature: Conditional Schedule Pre-Check (SCHED-COND-001 / Issue #454)
+
+## Overview
+Optional template-supplied hook that lets a scheduled cron tick be skipped deterministically without waking Claude. Before firing a cron-triggered chat, the scheduler calls a **backend** internal endpoint, which `docker exec`s the executable `~/.trinity/pre-check` file inside the target agent container. Non-empty stdout becomes the chat prompt; empty stdout + exit 0 records a skipped execution. Eliminates per-tick token burn on poll-driven agents (PR reviewers, inbox monitors, alert routers).
+
+The hook is **language-agnostic** — Trinity execs the file directly, so the interpreter is chosen by the file's shebang line. Templates ship Python, bash, node, Go binaries — Trinity does not care.
+
+## User Story
+As the author of a poll-driven agent template, I want a cheap deterministic gate to run before Claude wakes so empty polls (scan→no work) don't burn tokens. As a Trinity operator, I don't want to configure the gate per schedule — the agent template owns it, I just schedule the cadence.
+
+## Entry Points
+- **Template contract**: ship `~/.trinity/pre-check` as a `+x` executable file with a shebang (`#!/usr/bin/env python3`, `#!/bin/bash`, etc.). Prints chat prompt to stdout when work is found; exits 0 with empty stdout to skip; exits non-zero on error (fail-open).
+- **Backend endpoint** (internal, `X-Internal-Secret` auth): `POST /api/internal/agents/{name}/pre-check` — runs the script and returns stdout + exit code. Called only by `trinity-scheduler`.
+- **No operator-facing API change**: schedule CRUD endpoints and the Schedules UI are unchanged. Operators create normal cron schedules; agents own the gate.
+
+## Frontend Layer
+No UI change. Skipped executions appear in the existing schedule executions list alongside `success`/`failed` rows — the frontend already renders the `status` field as a badge.
+
+## Backend Layer
+**Router**: `src/backend/routers/internal.py` — `POST /api/internal/agents/{name}/pre-check`
+
+Uses `services/docker_service.execute_command_in_container` (the same primitive as `services/git_service.py` persistent-state allowlist, `services/ssh_service.py` key provisioning, `services/agent_service/terminal.py`, `routers/system_agent.py`, `adapters/message_router.py` Slack ingest, etc.).
+
+Two exec steps:
+1. `test -f /home/developer/.trinity/pre-check` (5s timeout). If the file doesn't exist, return `{"hook_present": False}` immediately — scheduler treats as "no decision, fire as usual." Note `-f`, not `-x`: a file present but missing the executable bit is treated as "hook present" so the operator gets a 126 in the logs rather than a silent backward-compat fall-through.
+2. `/home/developer/.trinity/pre-check` (60s timeout). Trinity execs the path directly — no `python3` prefix. Interpreter resolution is the file's shebang. Returns the output (capped at 32 KB) and exit code.
+
+Returns:
+```json
+{"hook_present": true, "exit_code": 0, "stdout": "Review PR #1\n", "stderr": ""}
+```
+
+## Scheduler Layer
+**Service**: `src/scheduler/service.py` — `_run_pre_check(agent_name)`
+
+Calls the backend's internal endpoint (not the agent directly — topology stays "scheduler → backend → agent"). Translates the backend response into a scheduler decision:
+
+| Backend response | Scheduler decision |
+|---|---|
+| `hook_present: false` | `None` → fire as usual (backward compat for templates without a hook) |
+| `exit_code != 0` | `None` → fail-open + log stderr (broken hook must not suppress work) |
+| `exit_code == 0`, empty stdout | `{"fire": False, "reason": "pre-check returned empty stdout"}` → record skipped execution |
+| `exit_code == 0`, non-empty stdout | `{"fire": True, "message": stdout.strip()}` → fire with stdout as override |
+| HTTP error / malformed JSON / timeout | `None` → fail-open + log |
+
+Intercept point in `_execute_schedule_with_lock`: called only when `triggered_by == "schedule"` (cron). Manual triggers bypass entirely.
+
+```python
+effective_message = schedule.message
+if triggered_by == "schedule":
+ decision = await self._run_pre_check(schedule.agent_name)
+ if decision is not None:
+ if not decision.get("fire", True):
+ self.db.create_skipped_execution(...)
+ self._publish_event({"type": "schedule_execution_skipped", ...})
+ return
+ override = decision.get("message")
+ if override:
+ effective_message = override
+# ... fires with effective_message
+```
+
+## Data Layer
+**Zero schema change.** Reuses existing:
+- `ExecutionStatus.SKIPPED` (already defined for Issue #46 — APScheduler max-instances drops).
+- `SchedulerDatabase.create_skipped_execution(...)`.
+
+Skip rows carry `status='skipped'`, `error="pre-check: "`, `duration_ms=0`, `started_at == completed_at`.
+
+## WebSocket Layer
+New event type: `schedule_execution_skipped`.
+
+```json
+{
+ "type": "schedule_execution_skipped",
+ "agent": "pr-reviewer",
+ "schedule_id": "LidXcOwtDsDuFTFGvkUqCw",
+ "execution_id": "I2DWodfpYpuJbs1TTZ42Ig",
+ "schedule_name": "PR review poll",
+ "reason": "pre-check: pre-check returned empty stdout"
+}
+```
+
+## Side Effects
+- Skipped execution row written to `schedule_executions`
+- `schedule.last_run_at` and `next_run_at` updated (so missed-schedule detection still works)
+- WebSocket event broadcast
+- No `/api/internal/execute-task` call, no backend task creation, no Claude invocation, no backlog slot acquisition
+
+## Error Handling
+| Condition | Scheduler behavior |
+|---|---|
+| Agent container doesn't exist | Backend returns 404 → fire as usual (schedule likely stale, let execute-task path handle the 404 surfacing) |
+| `~/.trinity/pre-check` absent | `hook_present: false` → fire as usual (backward compat) |
+| File present but not `+x` (exit 126) | Fail-open — log "hook for X exited 126", fire with `schedule.message`. Operator's signal to `chmod +x` the hook. |
+| Shebang missing or interpreter not found (exit 127) | Fail-open — log shows "command not found"; operator fixes shebang. |
+| Hook exits non-zero for any other reason | Fail-open — log stderr, fire with `schedule.message` |
+| Exec timeout (>60s) | Backend returns non-zero exit → fail-open |
+| Backend unreachable (connection error) | Fail-open — fire as usual, log warning |
+| Backend 5xx / malformed JSON | Fail-open |
+| Exit 0, empty stdout | Record skipped execution, no chat dispatch |
+| Exit 0, non-empty stdout | Fire with stdout as chat message (overrides `schedule.message`) |
+| Manual trigger (`triggered_by != 'schedule'`) | Skip pre-check entirely — explicit operator intent always fires |
+
+## Security
+- Pre-check runs inside the agent's container as `developer`, same sandbox as chat-mode tool calls. No new privilege over what chat-mode tool calls can already do.
+- **Template review expectation**: `.trinity/pre-check` is exec'd directly via the kernel — full process privileges of `developer`, in whatever language the shebang names. It can spawn subprocesses, open sockets, read files. This is intentional (operators already trust the template's `CLAUDE.md`, skills, and tool invocations), but `.trinity/pre-check` should be reviewed with the same scrutiny as any other executable the template ships.
+- Backend endpoint is gated by the existing `X-Internal-Secret` header (C-003). Only `trinity-scheduler` and other internal services can invoke it.
+- Stdout cap at 32 KB on the backend side — oversized output is truncated, still valid as a chat prompt (or truncated to "looks non-empty" which is fine for the fire-with-override path).
+- Fail-open policy means a malicious/broken pre-check cannot suppress scheduled invocations (worst case: wastes tokens — today's baseline).
+
+## Testing
+**Scheduler-side** (`tests/scheduler_tests/test_pre_check.py`, 13 tests):
+- `_run_pre_check` translation — `hook_present: False` → None; non-zero exit → None; empty stdout → skip; non-empty stdout → fire with message; 404 / 5xx / connection error / malformed JSON all → None (fail-open).
+- `_execute_schedule_with_lock` branch — skip records execution with correct reason; fire-true with message uses override; fail-open routes through backend; fire-true without message uses `schedule.message`; manual trigger bypasses pre-check.
+
+Full scheduler suite: 162/162 passing (was 161 before; +1 for the new `malformed JSON` case).
+
+No test file on the agent-server side — there's no agent-server router anymore.
+
+**Live end-to-end** (verified 2026-04-22 in local Trinity on the HTTP-endpoint version before pivoting to docker-exec):
+- Empty scan → skip row in DB, `$0` cost, zero backend chat activity.
+- Open PR → next tick fires with override message, Claude runs `/review`, posts comment.
+- Subsequent tick with existing bot comment → `fire:false` again (stateless dedup via GitHub).
+
+## Related Flows
+- `feature-flows/agent-event-subscriptions.md` — EVT-001, the event-driven analogue (other trigger source, same "non-cron agent invocation" theme).
+- `feature-flows/scheduler-service.md` — base scheduler behavior this extends.
+- `docs/planning/PR_REVIEWER_AGENT.md` — the motivating use case that drove this feature.
+
+## Architectural notes
+- Preserves **Invariant #11 (Docker as source of truth)**: the pre-check primitive is `docker exec`, matching `git_service.py`'s persistent-state allowlist, `ssh_service.py`, the agent terminal, etc.
+- Preserves the **"scheduler → backend → agent"** topology: scheduler never opens a direct HTTP edge to agent-server containers.
+- Reuses **`execute_command_in_container`**, an established async helper in `services/docker_service.py`.
+- No new schema, no new long-lived process, no new primitive — just a new internal endpoint and a scheduler branch.
+
+## Migration / Rollout
+- Zero migration required (no schema change).
+- Existing schedules and agent templates behave identically after deploy (script absent → fall back to today's fire semantics).
+- Templates opt in by shipping `.trinity/pre-check` (executable, `+x`, with a shebang). No Trinity-side flag. File extension is intentionally absent — the file is whatever the shebang says it is.
diff --git a/docs/memory/feature-flows/slack-channel-routing.md b/docs/memory/feature-flows/slack-channel-routing.md
index fffc9296e..f395fd73c 100644
--- a/docs/memory/feature-flows/slack-channel-routing.md
+++ b/docs/memory/feature-flows/slack-channel-routing.md
@@ -38,7 +38,7 @@ As a **platform admin**, I want Slack messages to go through the same execution
### Agent Detail — Sharing Tab (Per-Agent)
- `SlackChannelPanel.vue` — Three states:
- - **Bound**: Shows `#channel-name`, workspace name, DM default badge, "Unbind" button
+ - **Bound**: Shows `#channel-name`, workspace name, DM-default badge **or** "Make default" button (with hover tooltip explaining DM routing), and "Unbind" button. The Unbind button is **disabled** when this agent is the DM default *and* the workspace has other bound agents — promoting another agent first via the "Make default" button on its panel is required (#584).
- **Unbound**: "Create Slack Channel" button → creates channel in Slack + binds to agent
- **Access denied**: Informational message for non-owner shared users
- `SharingPanel.vue` — Renders `SlackChannelPanel` between Team Sharing and Public Links sections
@@ -55,9 +55,10 @@ As a **platform admin**, I want Slack messages to go through the same execution
- `POST /api/settings/slack/install` → `{oauth_url}` (browser redirect)
### API Calls (Per-Agent Channel)
-- `GET /api/agents/{name}/slack/channel` → `{bound, channel_name, channel_id, workspace_name}`
+- `GET /api/agents/{name}/slack/channel` → `{bound, channel_name, channel_id, workspace_name, is_dm_default, workspace_agent_count}`
- `POST /api/agents/{name}/slack/channel` → `{status, channel_name, channel_id, workspace_name}`
-- `DELETE /api/agents/{name}/slack/channel` → `{unbound, workspace_name}`
+- `DELETE /api/agents/{name}/slack/channel` → `{unbound, workspace_name}` — **409** if the agent is the workspace's DM default and other agents are still bound (#584)
+- `PUT /api/agents/{name}/slack/channel/dm-default` → `{status, team_id, workspace_name, previous, new_default}` — owner-only; single-tx clear-then-set on `is_dm_default`; audit-logged via `AGENT_LIFECYCLE/slack_dm_default_changed` (#584)
## Backend Layer
@@ -110,6 +111,8 @@ Priority in `SlackAdapter.get_agent_name()`:
| GET | `/api/agents/{name}/public-links/{id}/slack` | `routers/slack.py` | Connection status |
| DELETE | `/api/agents/{name}/public-links/{id}/slack` | `routers/slack.py` | Disconnect |
| PUT | `/api/agents/{name}/public-links/{id}/slack` | `routers/slack.py` | Update settings (enable/disable) |
+| PUT | `/api/agents/{name}/slack/channel/dm-default` | `routers/slack.py` | Make this agent the DM-default for its workspace (#584) |
+| DELETE | `/api/agents/{name}/slack/channel` | `routers/slack.py` | Unbind — refuses with 409 if agent is the DM default and others are bound (#584) |
### Business Logic
diff --git a/docs/memory/feature-flows/slack-file-sharing.md b/docs/memory/feature-flows/slack-file-sharing.md
index 6bb5490bc..8ece91511 100644
--- a/docs/memory/feature-flows/slack-file-sharing.md
+++ b/docs/memory/feature-flows/slack-file-sharing.md
@@ -27,14 +27,16 @@ ChannelMessageRouter._handle_message_inner()
Step 3b: File upload rate limit (5/min per user)
↓
Step 7b: _handle_file_uploads(adapter, message, agent_name, container, session_id)
+ → downloads each file via adapter.download_file() → assembles raw_files list
+ → delegates to services/upload_service.process_file_uploads() (#364)
↓
-For each file:
+process_file_uploads() for each file:
├── Unsupported format? → skip with message
- ├── _sanitize_filename(): NFKC + basename + safe-chars + 200-char
+ ├── sanitize_filename(): NFKC + basename + safe-chars + 200-char
│ truncation + collision dedup (-1, -2, …) (#487)
- ├── adapter.download_file() → bytes (channel-agnostic)
- ├── Image? → base64 → "[File uploaded by {uploader}]: name (size) — image
- │ attached inline\n"
+ ├── size check against limit
+ ├── magic-byte MIME validation (python-magic, graceful fallback)
+ ├── Image? → base64 vision block → image_data list (via stream-json)
└── Text? → tar archive → container_put_archive to uploads/{session_id}/
→ "[File uploaded by {uploader}]: name (size) saved to {path}"
↓
@@ -68,23 +70,29 @@ No frontend changes. File handling is entirely backend (Slack → adapter → ro
### Message Router (`adapters/message_router.py`)
- Step 3b: File upload rate limit (`_FILE_UPLOAD_RATE_LIMIT_MAX=5`, 60s window)
-- Step 7b: `_handle_file_uploads(verified_email=...)` — download, route by type,
- copy/embed; returns `(descriptions, upload_dir, all_writes_failed)`
-- Filename sanitization (`_sanitize_filename`, #487): NFKC unicode normalize
- → `os.path.basename` → safe-chars regex → `file_{id}` fallback (empty,
- dot-only, or hidden dotfiles like `.env`) → 200-char truncation preserving
- extension → collision dedup with `-1`, `-2`, … suffix
-- Session ID sanitization: `re.sub(r"[^a-zA-Z0-9_-]", "_", session_id)` for shell safety
+- Step 7b: `_handle_file_uploads(verified_email=...)` — downloads each file via
+ `adapter.download_file()`, then delegates to `upload_service.process_file_uploads()` (#364)
+ The inline validation/write logic was extracted to the shared service so web chat
+ (ChatPanel, PublicChat) reuses the same path
+- `_sanitize_filename()` and `_format_file_size()` in this file delegate to
+ `services/upload_service.sanitize_filename()` / `format_file_size()`
- Chat injection format: `[File uploaded by {uploader}]: {name} ({size}) saved
to {path}` — `uploader` is the verified email (Issue #311) or
`adapter.get_source_identifier(message)`
-- Image budget: 5MB/image, 10MB total, excess skipped
-- Unsupported MIME rejection: PDF, ZIP, tar, gzip, rar, video/*, audio/*
-- Allowed tools: `Read` added only for non-image files
- All-writes-failed handling: when every write attempt fails, reply via
channel with explicit error and skip agent execution (#487 AC6)
- Cleanup on all exit paths (success, task failure, exception, all-failed abort)
+### Upload Service (`services/upload_service.py`) — shared since #364
+- `process_file_uploads()` — core logic: sanitize, size-check, MIME-validate, image/file dispatch
+- `sanitize_filename()`: NFKC normalize → `os.path.basename` → safe-chars → hidden-dotfile
+ rejection → 200-char truncation → collision dedup with `-1`, `-2`, … suffix (#487)
+- Channel limits: `CHANNEL_MAX_FILES=10`, `CHANNEL_MAX_FILE_SIZE=10MB`
+- Image budget: `CHANNEL_MAX_IMAGE_SIZE=5MB`, `CHANNEL_MAX_TOTAL_IMAGE_SIZE=10MB`
+- Unsupported MIME rejection: PDF, ZIP, tar, gzip, rar, video/*, audio/*
+- Magic-byte MIME validation via python-magic (graceful fallback if unavailable)
+- Session ID sanitized with `re.sub(r"[^a-zA-Z0-9_-]", "_", session_id)` for shell safety
+
### Slack Service (`services/slack_service.py`)
- `download_file(bot_token, url, max_size)`: GET with Authorization header, follow redirects, 10MB cap
- OAuth scopes: `files:read` added (requires workspace reinstall)
@@ -153,3 +161,4 @@ No frontend changes. File handling is entirely backend (Slack → adapter → ro
- [slack-channel-routing.md](slack-channel-routing.md) — Channel adapter abstraction (SLACK-002)
- [slack-integration.md](slack-integration.md) — Original Slack integration (SLACK-001)
+- [web-chat-file-upload.md](web-chat-file-upload.md) — Web chat file upload (ChatPanel + PublicChat), uses same shared upload_service (#364)
diff --git a/docs/memory/feature-flows/subscription-auto-switch.md b/docs/memory/feature-flows/subscription-auto-switch.md
index b85147253..4f943df55 100644
--- a/docs/memory/feature-flows/subscription-auto-switch.md
+++ b/docs/memory/feature-flows/subscription-auto-switch.md
@@ -1,42 +1,61 @@
# Feature Flow: Subscription Auto-Switch (SUB-003)
> **Requirement**: `docs/requirements/SUB-003-subscription-auto-switch.md`
-> **Issue**: #153
-> **Status**: Implemented (2026-03-21)
+> **Issue**: #153, threshold + scope update #441
+> **Status**: Implemented (2026-03-21), updated 2026-04-25 (#441)
## Overview
-Automatically switches an agent to a different subscription when it encounters 2+ consecutive rate-limit (429) errors from the Claude API. Opt-in via system setting.
+Automatically switches an agent to a different subscription on the first
+subscription failure — either a rate-limit (429) **or** an auth-class
+failure (401/403/credit balance/expired token). Default ON (opt-out via
+system setting `auto_switch_subscriptions`).
## Flow
```
-Agent container detects rate limit → returns 429 to backend
+Agent container detects rate limit OR auth failure → returns 429/503 to backend
↓
-Backend catches 429 in:
- - TaskExecutionService.execute_task() [schedules, MCP, agent-to-agent]
- - chat_with_agent() [interactive chat]
- - background task handler [async tasks]
+Backend catches the failure in:
+ - TaskExecutionService.execute_task() [schedules, MCP, agent-to-agent, async]
+ - chat_with_agent() [interactive chat sync path]
↓
-subscription_auto_switch.handle_rate_limit_error(agent_name)
+Classify:
+ - 429 → handle_subscription_failure(..., failure_kind="rate_limit")
+ - 503 OR is_auth_failure(error_msg) → handle_subscription_failure(..., failure_kind="auth")
↓
Check: setting enabled? → No → return None
↓ Yes
Check: agent has subscription? → No → return None
↓ Yes
-Record rate-limit event, get consecutive count
+Record failure event, get count (informational; no threshold gate)
↓
-Count < 2? → return None (wait for more)
- ↓ ≥ 2
-Find best alternative subscription (fewest agents, not rate-limited)
+Find best alternative subscription (fewest agents, not rate-limited in last 2h)
↓
No alternative? → return None (log warning)
↓ Found
Switch: DB update + container restart + log activity + send notification
↓
-Return switch result to caller → 429 response includes auto_switch info
+Return switch result → caller surfaces 429/503 with auto_switch info + retry hint
```
+## Trigger Surface
+
+| Layer | Signal | Failure kind |
+|-------|--------|--------------|
+| HTTP 429 from agent | rate-limit reached | `rate_limit` |
+| HTTP 503 from agent | auth failure (#285 detection) | `auth` |
+| Error message matches `AUTH_INDICATORS` | credit balance / expired token / unauthorized / etc. | `auth` |
+
+`AUTH_INDICATORS` (canonical list in
+`src/backend/services/subscription_auto_switch.py::is_auth_failure`):
+`credit balance`, `unauthorized`, `authentication`, `credentials`,
+`forbidden`, `401`, `403`, `oauth`, `token expired`, `not authenticated`.
+
+The scheduler service (`src/scheduler/service.py`) maintains a duplicate
+copy of this list because it runs in a separate container and cannot
+import from `backend.services`. Keep the two in sync when editing either.
+
## Files
| Layer | File | Purpose |
@@ -71,7 +90,7 @@ Return switch result to caller → 429 response includes auto_switch info
| Key | Default | Description |
|-----|---------|-------------|
-| `auto_switch_subscriptions` | `"false"` | Enable/disable auto-switch |
+| `auto_switch_subscriptions` | `"true"` (#441) | Enable/disable auto-switch |
## API Endpoints
@@ -114,8 +133,8 @@ anyway, so the omission was silent.
## Edge Cases
-- **All subscriptions exhausted**: No switch, error surfaces as normal 429. `_perform_auto_switch` does **not** clear rate-limit events for the old subscription — those events are the signal that keeps `is_subscription_rate_limited()` truthful, so the just-drained sub is not offered as a candidate on the next cycle (issue #444).
+- **All subscriptions exhausted**: No switch, error surfaces as normal 429/503. `_perform_auto_switch` does **not** clear rate-limit events for the old subscription — those events are the signal that keeps `is_subscription_rate_limited()` truthful, so the just-drained sub is not offered as a candidate on the next cycle (issue #444).
- **API key agents**: Auto-switch only applies to subscription-based agents
-- **Flip-flopping**: 2-consecutive-error requirement prevents immediate re-switch
+- **Flip-flopping** (#441 update): the 2h skip-list (`is_subscription_rate_limited` ∧ `select_best_alternative_subscription`) is now the only thrash guard. Pre-#441 the threshold also required 2 consecutive 429s before switching, but that gated user-visible failures unnecessarily — the skip-list alone is sufficient because a just-drained sub stays flagged for 2h post-switch.
- **Concurrent switches**: SQLite serialization prevents races
- **Cleanup**: Records older than 24h are pruned hourly by `CleanupService` (phase 6, #476); the 2h "is rate-limited" window drives candidate filtering independently of cleanup
diff --git a/docs/memory/feature-flows/task-execution-service.md b/docs/memory/feature-flows/task-execution-service.md
index bf02f0764..8e1558b45 100644
--- a/docs/memory/feature-flows/task-execution-service.md
+++ b/docs/memory/feature-flows/task-execution-service.md
@@ -1,5 +1,7 @@
# Feature: Task Execution Service (EXEC-024)
+> **Updated 2026-04-26 (#428):** Slot acquisition/release now goes through [`CapacityManager`](capacity-management.md) (`acquire(overflow_policy="reject")` + `release()`) rather than calling `SlotService` directly. The `slot_already_held` parameter still applies — routers pre-acquire via `CapacityManager` and pass `slot_already_held=True` so the service's `finally` block remains the single release point.
+
## Overview
Service that encapsulates the task-execution lifecycle (execution record, slot management, activity tracking, agent HTTP call with retry, credential sanitization, response persistence). Used by most — but not all — execution paths.
@@ -58,7 +60,7 @@ Callers inspect `result.status` to decide HTTP response. Status values come from
Moved from `routers/chat.py`. Module-level async function. Used by:
- `TaskExecutionService.execute_task()` internally (line 249)
-- `routers/chat.py` for `/chat` endpoint (line 248) and `_execute_task_background` (line 477)
+- `routers/chat.py` for `/chat` endpoint and `_run_async_task_with_persistence` (the async-mode wrapper introduced by #95; previously named `_execute_task_background`)
```python
async def agent_post_with_retry(
@@ -138,6 +140,7 @@ async def execute_task(
parent_activity_id: Optional[str] = None, # Issue #95: CHAT_START parent linkage
extra_activity_details: Optional[dict] = None, # Issue #95: merged into CHAT_START details
slot_already_held: bool = False, # Issue #95: async path pre-acquires slot upfront
+ images: Optional[list] = None, # #562: vision content blocks for channel images
) -> TaskExecutionResult:
```
@@ -163,7 +166,7 @@ The endpoint handles:
2. Determine `triggered_by` from headers (lines 686-691)
3. Create execution record early (lines 694-705) -- passed to service as `execution_id`
4. Collaboration tracking for agent-to-agent (lines 710-732) -- stays in router
-5. **Async mode branch** (lines 735-808) -- spawns `_execute_task_background()`, does NOT use service
+5. **Async mode branch** -- pre-acquires capacity slot, then spawns `_run_async_task_with_persistence()` which delegates to `task_execution_service.execute_task(slot_already_held=True)` (post-#95)
6. **Sync mode branch** (lines 810-827) -- delegates to `task_execution_service.execute_task()`
7. Collaboration activity completion (lines 830-839)
8. Error translation to HTTP exceptions (lines 842-857)
diff --git a/docs/memory/feature-flows/telegram-integration.md b/docs/memory/feature-flows/telegram-integration.md
index b32d62f02..d4bce46e2 100644
--- a/docs/memory/feature-flows/telegram-integration.md
+++ b/docs/memory/feature-flows/telegram-integration.md
@@ -808,7 +808,7 @@ the Slack pattern).
**Chat injection format** (every channel, not just Telegram):
- Successful file: `[File uploaded by {uploader}]: {filename} ({size}) saved to {dest_path}`
-- Successful image: `[File uploaded by {uploader}]: {filename} ({size}) — image attached inline` followed by ``
+- Successful image: `[File uploaded by {uploader}]: {filename} ({size}) — image provided for visual analysis` (image delivered as vision content block — see #562)
- Workspace write failure: `[File upload failed]: {filename} — {reason}`
`{uploader}` is the verified email when `adapter.resolve_verified_email`
@@ -837,8 +837,44 @@ human-readable provenance for any file in its workspace.
`dest_path`, `storage` (`container_file` or `inline_base64`), and
`uploader` so the audit trail captures who uploaded what to which agent.
+### Phase 3: Vision Delivery via stream-json (#562)
+
+**Problem**: Images embedded as `data:` URIs in the text prompt were opaque strings — the Claude Code CLI never forwarded them to the Claude API as real image content blocks, so agents could not see the images.
+
+**Fix**: `_handle_file_uploads()` now returns image MIME/base64 dicts as a 4th element of its tuple (`image_data: list`). For image MIME types, the file is still written to the container workspace (read tool fallback), but it is also collected for vision delivery. The description changes from `"image attached inline"` to `"image provided for visual analysis"` (no `data:` URI appended).
+
+`message_router.py` passes `images=image_data or None` into `task_execution_service.execute_task()`. The service adds it to the `/api/task` HTTP payload. The agent-side `chat.py` router passes it to `runtime.execute_headless()`. In `claude_code.py`, when `images` is truthy, two changes take effect:
+
+1. `--input-format stream-json` is appended to the `claude` CLI command.
+2. `stdin` payload is a JSON message instead of plain text:
+ ```json
+ {"type": "user", "message": {"role": "user", "content": [
+ {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": ""}},
+ {"type": "text", "text": ""}
+ ]}}
+ ```
+
+The `GeminiRuntime` and `AgentRuntime` ABC gained an `images` parameter (ignored by Gemini today) to prevent `TypeError` on any image task.
+
+**Stdin write ordering**: stdout/stderr reader threads are started before `process.stdin.write()` to prevent pipe deadlock when the image payload is large. The write itself runs inside the executor thread (not the async event loop) to avoid blocking.
+
+**Key files**:
+- `src/backend/adapters/message_router.py` — 4-tuple return, `images or None` passthrough
+- `src/backend/services/task_execution_service.py` — `images` param, forwarded in payload
+- `docker/base-image/agent_server/models.py` — `ParallelTaskRequest.images` field
+- `docker/base-image/agent_server/routers/chat.py` — passes `images` to `execute_headless()`
+- `docker/base-image/agent_server/services/claude_code.py` — stream-json stdin + flag
+- `docker/base-image/agent_server/services/runtime_adapter.py` — ABC signature updated
+- `docker/base-image/agent_server/services/gemini_runtime.py` — GeminiRuntime signature updated
+
### Tests
+17 unit tests in `tests/unit/test_channel_image_vision.py` (#562):
+- `TestParallelTaskRequestImages` (4 tests): model field defaults and acceptance
+- `TestStreamJsonPayload` (6 tests): payload construction for 0/1/N images
+- `TestCmdContainsInputFormat` (4 tests): `--input-format stream-json` added iff images present
+- `TestHandleFileUploadsImageReturn` (3 tests): 4-tuple return, image vs non-image MIME
+
27 unit tests in `tests/unit/test_file_upload.py`:
- `TestTelegramFileExtraction` (4 tests): Photo/document extraction
- `TestTelegramFileDownload` (3 tests): Bot API two-step download
@@ -884,3 +920,4 @@ human-readable provenance for any file in its workspace.
| 2026-04-16 | #354 Phase 1: Telegram file upload support. `_extract_files()` and `download_file()` in adapter. Post-download size/MIME validation in router. python-magic dependency. 11 unit tests. |
| 2026-04-18 | #318: Voice transcription via Gemini. `process_voice()` in telegram_media.py, voice processing hook in message_router.py. Limits: 5 min duration, 10MB size. 22 unit tests. |
| 2026-04-25 | #487 Phase 2: workspace delivery hardened. New `_sanitize_filename` helper (NFKC + basename + safe-chars + 200-char truncation + collision dedup). Chat injection format `[File uploaded by {uploader}]: {name} ({size}) saved to {path}`. All-writes-failed now replies via channel and aborts execution. Audit entries include `uploader`. 16 new tests (27 total in `test_file_upload.py`). |
+| 2026-04-28 | #562: Vision delivery fixed. Replaced broken base64 data URI text embedding with proper `--input-format stream-json` vision content blocks delivered via Claude CLI stdin. `_handle_file_uploads` returns 4-tuple with `image_data`. `GeminiRuntime` and `AgentRuntime` ABC updated to accept `images` param. 17 new tests in `test_channel_image_vision.py`. |
diff --git a/docs/memory/feature-flows/voice-chat.md b/docs/memory/feature-flows/voice-chat.md
index dfa42fe19..08fdc14be 100644
--- a/docs/memory/feature-flows/voice-chat.md
+++ b/docs/memory/feature-flows/voice-chat.md
@@ -1,7 +1,7 @@
# Voice Chat — Gemini 2.5 Flash Native Audio
-**Status**: 🚧 Phase 1 MVP Implemented
-**Date**: 2026-03-23
+**Status**: ✅ Phase 1 + Tool Calling Complete
+**Date**: 2026-04-29
**Priority**: P1
---
@@ -14,73 +14,68 @@ Users need a fast, natural way to speak with agents via the browser. Text chat c
## Core Concept
-Use **Gemini 2.5 Flash Native Audio** (`gemini-live-2.5-flash-native-audio`) as a real-time voice proxy for the agent. Claude Code remains the agent's brain (generates the system prompt and handles complex tasks), but the voice conversation runs on Gemini's speech-to-speech model for speed (~280ms TTFT).
+Use **Gemini 2.5 Flash Native Audio** (`gemini-live-2.5-flash-native-audio`) as a real-time voice proxy for the agent. Claude Code remains the agent's brain (handles complex tasks via tool calls), but the voice conversation runs on Gemini's speech-to-speech model for speed (~280ms TTFT). During voice sessions, Gemini can invoke a `run_task` function declaration to delegate work to the underlying Claude agent.
### Why Not Claude for Voice?
Anthropic has no speech-to-speech or realtime audio API. Any Claude voice pipeline would require STT → Claude text API (~500-800ms TTFT) → TTS, totaling 800ms-1.3s. Gemini's native audio model handles audio in/out natively with ~280ms latency and built-in turn-taking, barge-in, and emotion.
-### Why Gemini 2.5 Flash Native Audio?
-
-- Native speech-to-speech (no STT/TTS pipeline)
-- ~280ms time-to-first-token
-- 30 HD voices, 24 languages
-- Supports custom system prompts
-- Affective dialog (responds to user's tone)
-- Function calling mid-conversation
-- GA on Vertex AI
-- WebSocket-based Live API
-
---
## Architecture
```
-┌─────────────────────────────────────────────────────────┐
-│ Browser (Agent Detail) │
-│ │
-│ ┌──────────────┐ ┌───────────────────────────────┐ │
-│ │ Chat Panel │ │ Voice Overlay / Panel │ │
-│ │ (existing) │ │ │ │
-│ │ │ │ [Waveform / Speaking Status] │ │
-│ │ ... msgs │ │ [Mute] [End Call] │ │
-│ │ │ │ │ │
-│ └──────────────┘ └──────────┬────────────────────┘ │
-│ │ │
-│ WebSocket (audio frames) │
-└─────────────────────────┬───────────────────────────────┘
- │
- ▼
- ┌───────────────────────┐
- │ Trinity Backend │
- │ │
- │ /api/agents/{name}/ │
- │ voice/start │
- │ voice/stop │
- │ │
- │ WebSocket proxy: │
- │ Browser ↔ Gemini │
- │ Live API │
- │ │
- │ On session end: │
- │ - Extract transcript │
- │ - Save to ChatMessage│
- │ - Update ChatSession │
- └───────────────────────┘
- │
- ▼
- ┌───────────────────────┐
- │ Gemini Live API │
- │ (Vertex AI) │
- │ │
- │ Model: │
- │ gemini-live-2.5- │
- │ flash-native-audio │
- │ │
- │ System prompt: │
- │ agent personality + │
- │ conversation summary │
- └───────────────────────┘
+┌────────────────────────────────────────────────────────────┐
+│ Browser (Agent Detail) │
+│ │
+│ ┌──────────────┐ ┌──────────────────────────────────┐ │
+│ │ Chat Panel │ │ VoiceOverlay.vue (canvas orb) │ │
+│ │ (existing) │ │ │ │
+│ │ │ │ Canvas orb — value noise + curl │ │
+│ │ ... msgs │ │ noise particles (220, 3 layers) │ │
+│ │ │ │ State hues: idle/connecting=0° │ │
+│ │ │ │ listening=+90° (green) │ │
+│ │ │ │ speaking=+210° (indigo) │ │
+│ │ │ │ tool_calling=amber badge overlay │ │
+│ │ │ │ [Mute] [End Call] │ │
+│ └──────────────┘ └──────────────┬───────────────────┘ │
+│ │ │
+│ useVoiceSession.js composable │
+│ (amplitude polling @ 30ms, │
+│ tool_call / tool_result WS handlers) │
+│ │ WebSocket │
+└─────────────────────────────────────┼───────────────────────┘
+ │
+ ▼
+ ┌───────────────────────┐
+ │ Trinity Backend │
+ │ │
+ │ routers/voice.py │
+ │ POST voice/start │
+ │ WS /ws/voice/{id} │
+ │ │
+ │ services/gemini_voice │
+ │ VoiceSession │
+ │ _execute_and_respond()│
+ │ (30s timeout) │
+ │ │
+ │ on_tool_call → │
+ │ WS frame + audit log│
+ │ on_tool_result → │
+ │ WS frame │
+ └────────────┬───────────┘
+ │
+ ┌────────────┴───────────┐
+ │ │
+ ▼ ▼
+ ┌───────────────────┐ ┌────────────────────┐
+ │ Gemini Live API │ │ Agent Container │
+ │ (Vertex AI) │ │ (Claude Code) │
+ │ │ │ │
+ │ tools=[ │ │ agent_client │
+ │ run_task fn │ │ .task(prompt) │
+ │ ] │ │ │
+ └───────────────────┘ └────────────────────┘
```
---
@@ -91,44 +86,28 @@ Anthropic has no speech-to-speech or realtime audio API. Any Claude voice pipeli
1. User is on Agent Detail page, Chat tab (authenticated)
2. User clicks **"Talk"** button (microphone icon) next to the chat input
-3. Backend receives `POST /api/agents/{name}/voice/start`
+3. `POST /api/agents/{name}/voice/start` with optional `voice_name`
4. Backend prepares the voice session:
- a. Loads the agent's **voice system prompt** (pre-created, stored as agent config)
- b. Fetches recent chat history for this session
- c. Summarizes the conversation so far into a concise context block
- d. Combines: `voice_system_prompt + "\n\n## Conversation so far:\n" + summary`
-5. Backend opens a WebSocket connection to Gemini Live API with the combined prompt
-6. Backend establishes a WebSocket bridge: browser ↔ backend ↔ Gemini
-7. Browser captures microphone audio via `getUserMedia()` and streams to backend
-8. Gemini responds with audio frames streamed back to the browser
-9. Voice overlay appears with speaking indicators
+ a. 3-level system prompt fallback: DB field → container file `voice-agent-system-prompt.md` → auto-generate from template info → generic
+ b. Fetches recent chat history for context injection
+5. Backend opens connection to Gemini Live API with `tools=[_RUN_TASK_TOOL]`
+6. WebSocket bridge established: browser ↔ backend ↔ Gemini
+7. Canvas orb overlay appears; state transitions drive hue rotation
### During the Voice Session
-- User speaks naturally; Gemini responds in real-time
-- Gemini uses the agent's personality from the system prompt
-- Gemini has context of the prior text conversation via the summary
-- User can interrupt (barge-in) at any time
-- Backend accumulates the transcript (Gemini provides text alongside audio)
+- User speaks; Gemini responds in real-time (~280ms TTFT)
+- When Gemini calls `run_task`, backend dispatches `asyncio.create_task(_execute_and_respond())` (30s timeout)
+- `tool_call` WS frame sent → orb shows amber badge; `tool_result` frame sent → orb returns to listening state
+- All tool calls written to platform audit log
+- Backend accumulates transcript from Gemini `serverContent` messages
### Ending a Voice Session
-1. User clicks **"End"** button, or closes the overlay
-2. Backend sends session close to Gemini Live API
-3. Backend extracts the full transcript from the session
-4. Transcript is saved as ChatMessage entries in the existing chat session:
- - Each user utterance → `ChatMessage(role="user", content=text)`
- - Each agent response → `ChatMessage(role="assistant", content=text)`
- - Messages tagged with `source="voice"` for UI differentiation
-5. Chat panel refreshes and shows the voice conversation inline with text messages
-6. User can continue the conversation via text or start another voice session
-
-### Picking Up Context
-
-When starting a new voice session mid-conversation:
-- The summary includes ALL prior messages (both text and previous voice transcripts)
-- This means the voice agent knows what was discussed in text AND in previous voice sessions
-- Seamless continuity between modalities
+1. User clicks **"End"** button or closes overlay
+2. Backend closes Gemini session, cancels any pending `_pending_tool_tasks`
+3. Transcript saved as `ChatMessage` rows in existing `chat_messages` table
+4. Chat panel refreshes showing voice conversation inline with text messages
---
@@ -136,81 +115,108 @@ When starting a new voice session mid-conversation:
### VOICE-001: Voice Session Initialization
-**Status**: 🚧 Phase 1 Implemented
+**Status**: ✅ Implemented
| Requirement | Detail |
|-------------|--------|
-| Backend endpoint | `POST /api/agents/{name}/voice/start` → returns WebSocket URL |
-| System prompt source | Agent config field: `voice_system_prompt` (pre-created by agent owner) |
-| Context injection | Summarize last N messages of current chat session (truncate to fit Gemini context) |
-| Gemini connection | Open WebSocket to `generativelanguage.googleapis.com` Live API |
-| Authentication | Vertex AI service account or Gemini API key (platform-level config) |
-| Audio format | PCM 16-bit, 16kHz mono (Gemini Live API default) |
+| Backend endpoint | `POST /api/agents/{name}/voice/start` → returns `voice_session_id` + WebSocket URL |
+| System prompt source | 3-level fallback: DB → `voice-agent-system-prompt.md` container file → auto-generate → generic |
+| Context injection | Summarize last N messages of current chat session |
+| Gemini connection | `google-genai` SDK, `generativelanguage.googleapis.com` Live API |
+| Authentication | `GEMINI_API_KEY` platform setting |
+| Audio format | PCM 16-bit, 16kHz mono |
+| Voice name | Passed via `voice_name` field in `VoiceStartRequest` |
### VOICE-002: Audio Streaming Bridge
-**Status**: ⏳ Not Started
+**Status**: ✅ Implemented
| Requirement | Detail |
|-------------|--------|
| Browser → Backend | WebSocket carrying raw audio frames from `getUserMedia()` |
-| Backend → Gemini | Forward audio frames to Gemini Live API WebSocket |
+| Backend → Gemini | Forward audio frames to Gemini Live API |
| Gemini → Backend | Receive audio response frames + transcript text |
-| Backend → Browser | Forward audio frames for playback via `AudioContext` |
-| Latency target | < 100ms added by the proxy layer |
-| Concurrency | One active voice session per user per agent |
+| Backend → Browser | Forward audio frames for playback |
+| Playback engine | AudioWorklet-first (blob URL inlined) with ScriptProcessor fallback |
+| Amplitude | `createAudioPlayer()` exposes `getAmplitude()` (0–1 float) via `AnalyserNode` |
### VOICE-003: Transcript Persistence
-**Status**: ⏳ Not Started
+**Status**: ✅ Implemented
| Requirement | Detail |
|-------------|--------|
-| Transcript extraction | Capture text from Gemini's `serverContent` messages (text alongside audio) |
-| Storage | Save as `ChatMessage` rows in existing `chat_messages` table |
-| Message metadata | `source` field = `"voice"` to distinguish from text messages |
-| Session linkage | Messages belong to the user's current `ChatSession` |
-| Timing | Save on session end (batch) AND incrementally during session |
-| Cost tracking | Track Gemini API cost per voice session |
+| Transcript extraction | Captured from Gemini `serverContent` messages during session |
+| Storage | Saved as `ChatMessage` rows in `chat_messages` table |
+| Session linkage | Belongs to user's current `ChatSession` |
+| Timing | Saved on session end |
### VOICE-004: Frontend Voice UI
-**Status**: ⏳ Not Started
+**Status**: ✅ Implemented
| Requirement | Detail |
|-------------|--------|
| Trigger | Microphone icon button next to chat input textarea |
-| Voice overlay | Appears over/beside chat panel when voice is active |
-| Visual feedback | Waveform or pulsing indicator showing who is speaking |
+| Voice overlay | `VoiceOverlay.vue` — full canvas orb, pure JS (no CDN) |
+| Particle system | Value noise + curl noise, 220 smoke particles in 3 layers, 9 pre-rendered sprite canvases |
+| State hues | idle/connecting: 0°, listening: +90° (green), speaking: +210° (indigo), tool_calling: amber badge |
| Controls | Mute mic toggle, End call button |
-| Browser API | `navigator.mediaDevices.getUserMedia({ audio: true })` |
-| Audio playback | `AudioContext` + `AudioWorklet` for low-latency playback |
-| Permission | Request mic permission on first click, show clear prompt |
-| Mobile | Must work on mobile browsers (Safari, Chrome) |
+| Amplitude polling | `setInterval(30ms)` via `amplitude` ref in composable |
+| Audio capture | `navigator.mediaDevices.getUserMedia({ audio: true })` |
+| Audio playback | AudioWorklet with ScriptProcessor fallback |
### VOICE-005: Voice System Prompt
-**Status**: ⏳ Not Started
+**Status**: ✅ Implemented
| Requirement | Detail |
|-------------|--------|
-| Storage | New field on agent: `voice_system_prompt` (text, nullable) |
-| Creation | Agent owner writes it manually OR generates from agent's main CLAUDE.md |
-| Content | Concise personality description, conversation style, voice-specific instructions |
-| Voice-specific rules | E.g., "Keep responses under 2 sentences", "Use casual language", "Don't use markdown" |
-| Fallback | If no voice prompt set, derive a basic one from agent name + description |
+| Lookup order | 1) DB field, 2) `voice-agent-system-prompt.md` in container, 3) auto-generate from template info, 4) generic fallback |
+| Implementation | `_get_voice_system_prompt()` in `routers/voice.py` |
### VOICE-006: Conversation Summary for Context
-**Status**: ⏳ Not Started
+**Status**: ✅ Implemented (truncation approach)
| Requirement | Detail |
|-------------|--------|
| Trigger | On voice session start |
-| Input | All ChatMessage rows for current session (text + prior voice transcripts) |
-| Method | Call Claude (fast model, e.g., Haiku) to summarize OR simple truncation of last N messages |
-| Output | Concise summary (< 2000 tokens) injected into Gemini system prompt |
-| Fallback | If no prior messages, just use the voice system prompt alone |
+| Method | Truncation of last N messages injected into system prompt |
+| Fallback | Voice system prompt alone if no prior messages |
+
+### VOICE-007: Tool Calling (run_task)
+
+**Status**: ✅ Implemented (Phase 3 — shipped early)
+
+| Requirement | Detail |
+|-------------|--------|
+| Function declaration | `_RUN_TASK_TOOL` (`FunctionDeclaration` for `run_task`) registered in `LiveConnectConfig` |
+| Execution | `_execute_and_respond()` coroutine, `asyncio.create_task` per call, 30s `wait_for` timeout |
+| Agent call | `agent_client.task(prompt)` (lazy import), truncated to `_TOOL_PROMPT_MAX=2000` chars |
+| Error handling | `AgentNotReachableError` → "not currently running"; `AgentRequestError` → "Task error: ..." |
+| Empty prompt | Falls back to "No prompt" |
+| Session tracking | `_pending_tool_tasks` dict on `VoiceSession`; all cancelled on `end_session()` |
+| WS events | `{type: "tool_call", tool_name: "run_task"}` and `{type: "tool_result", ...}` frames |
+| Audit | Platform audit log written on each tool call via `on_tool_call` callback |
+
+---
+
+## WebSocket Message Types
+
+**Client → Server:**
+```json
+{ "type": "audio", "data": "" }
+```
+
+**Server → Client:**
+```json
+{ "type": "audio", "data": "" }
+{ "type": "transcript", "role": "user|assistant", "text": "..." }
+{ "type": "status", "state": "listening|speaking|processing" }
+{ "type": "tool_call", "tool_name": "run_task" }
+{ "type": "tool_result", "tool_name": "run_task", "result": "..." }
+```
---
@@ -221,7 +227,8 @@ When starting a new voice session mid-conversation:
**Request:**
```json
{
- "session_id": "optional - existing chat session to continue"
+ "session_id": "optional - existing chat session to continue",
+ "voice_name": "optional - Gemini voice name"
}
```
@@ -234,46 +241,8 @@ When starting a new voice session mid-conversation:
}
```
-### WebSocket /ws/voice/{voice_session_id}
-
-**Client → Server frames:**
-```json
-{
- "type": "audio",
- "data": ""
-}
-```
-
-**Server → Client frames:**
-```json
-{
- "type": "audio",
- "data": ""
-}
-```
-```json
-{
- "type": "transcript",
- "role": "user|assistant",
- "text": "what the user/agent said"
-}
-```
-```json
-{
- "type": "status",
- "state": "listening|speaking|processing"
-}
-```
-
### POST /api/agents/{name}/voice/stop
-**Request:**
-```json
-{
- "voice_session_id": "vs_abc123"
-}
-```
-
**Response:**
```json
{
@@ -288,76 +257,64 @@ When starting a new voice session mid-conversation:
## Configuration
-### Platform-Level (Settings)
+### Platform-Level
| Setting | Description |
|---------|-------------|
-| `GEMINI_API_KEY` | API key for Gemini Live API (or Vertex AI service account) |
-| `VOICE_ENABLED` | Global toggle to enable/disable voice feature |
+| `GEMINI_API_KEY` | API key for Gemini Live API |
+| `VOICE_ENABLED` | Global toggle |
| `VOICE_MODEL` | Model ID (default: `gemini-2.5-flash-native-audio-preview-12-2025`) |
-| `VOICE_MAX_DURATION` | Max voice session duration in seconds (default: 300) |
+| `VOICE_MAX_DURATION` | Max session duration in seconds (default: 300) |
### Per-Agent
| Setting | Description |
|---------|-------------|
-| `voice_system_prompt` | Agent-specific voice personality prompt |
-| `voice_enabled` | Per-agent toggle (default: true if platform voice is enabled) |
-| `voice_name` | Gemini voice selection (from 30 HD voices) |
+| `voice_system_prompt` | Agent-specific voice personality prompt (DB field) |
+| `voice_enabled` | Per-agent toggle |
+| `voice_name` | Gemini voice selection (passed at session start) |
+
+---
+
+## Key Implementation Files
+
+| Layer | File | Purpose |
+|-------|------|---------|
+| **Backend** | `src/backend/routers/voice.py` | Voice endpoints + WebSocket handler, `_get_voice_system_prompt()`, `on_tool_call`/`on_tool_result` callbacks |
+| **Backend** | `src/backend/services/gemini_voice.py` | `VoiceSession`, `_RUN_TASK_TOOL` declaration, `_execute_tool()`, `_execute_and_respond()`, `_pending_tool_tasks` |
+| **Frontend** | `src/frontend/src/components/chat/VoiceOverlay.vue` | Canvas orb (value noise + curl noise particles, state hues, amber tool badge) |
+| **Frontend** | `src/frontend/src/composables/useVoiceSession.js` | Session state (`toolName`, `amplitude`, `isToolCalling`), WS message handlers, amplitude polling |
+| **Frontend** | `src/frontend/src/utils/audio.js` | AudioWorklet-first capture/playback, `getAmplitude()` via `AnalyserNode` |
+| **Tests** | `tests/unit/test_voice_tools.py` | 12 unit tests: `_execute_tool`, `_execute_and_respond`, tool declaration, session cancellation |
---
## Scope & Phasing
-### Phase 1: MVP (This Implementation)
+### Phase 1: MVP ✅ Complete
- Authenticated chat only (not public links)
- Single agent at a time
-- Basic voice overlay UI (no waveform, just status indicators)
-- Transcript saved on session end (not incremental)
-- Manual voice system prompt (agent owner writes it)
+- Voice overlay with canvas orb visualization
+- Transcript saved on session end
+- 3-level voice system prompt fallback
- Gemini API key in platform settings
+- `voice_name` selection at session start
-### Phase 2: Polish
-
-- Real-time waveform visualization
-- Incremental transcript display in chat during voice session
-- Auto-generate voice prompt from agent's CLAUDE.md
-- Voice quality/latency metrics
-- Public link voice support
-
-### Phase 3: Advanced
-
-- Function calling (Gemini calls Trinity MCP tools mid-voice-session)
-- Multi-language voice with auto-detection
-- Voice cloning / custom voice per agent
-- Voice-to-voice agent-to-agent communication
+### Phase 2: Polish (Partial)
----
-
-## Dependencies
+- ✅ Real-time amplitude visualization (canvas orb driven by `getAmplitude()`)
+- ⏳ Incremental transcript display in chat during session
+- ⏳ Voice quality/latency metrics
+- ⏳ Public link voice support
-| Dependency | Purpose | Status |
-|------------|---------|--------|
-| Gemini API key | Access to Live API | Need to configure |
-| `google-genai` Python SDK | Server-side Gemini Live API client | Need to install |
-| Browser `getUserMedia` | Microphone access | Available in all modern browsers |
-| Browser `AudioContext` | Audio playback | Available in all modern browsers |
-| Existing ChatPanel.vue | Integration point for voice button | ✅ Exists |
-| Existing ChatMessage model | Storage for transcripts | ✅ Exists |
-| Existing ChatSession model | Session tracking | ✅ Exists |
-
----
+### Phase 3: Tool Calling ✅ Complete (shipped with Phase 1)
-## Key Implementation Files (Planned)
-
-| Layer | File | Purpose |
-|-------|------|---------|
-| **Backend** | `src/backend/routers/voice.py` | Voice API endpoints + WebSocket handler |
-| **Backend** | `src/backend/services/gemini_voice.py` | Gemini Live API client wrapper |
-| **Frontend** | `src/frontend/src/components/chat/VoiceOverlay.vue` | Voice session UI overlay |
-| **Frontend** | `src/frontend/src/composables/useVoiceSession.js` | Voice session state management |
-| **Frontend** | `src/frontend/src/utils/audio.js` | Audio capture and playback utilities |
+- ✅ `run_task` function calling (Gemini delegates to Claude agent mid-session)
+- ✅ Amber badge UI state during tool execution
+- ✅ 30s timeout with error recovery
+- ⏳ Multi-language voice with auto-detection
+- ⏳ Voice cloning / custom voice per agent
---
@@ -367,4 +324,3 @@ When starting a new voice session mid-conversation:
- [Persistent Chat Tracking](./persistent-chat-tracking.md) — Message storage
- [Gemini Runtime](./gemini-runtime.md) — Existing Gemini integration
- [Gemini Live API Docs](https://ai.google.dev/gemini-api/docs/live-api) — Official API reference
-- [Gemini Live API Capabilities](https://ai.google.dev/gemini-api/docs/live-api/capabilities) — Audio features
diff --git a/docs/memory/feature-flows/web-chat-file-upload.md b/docs/memory/feature-flows/web-chat-file-upload.md
new file mode 100644
index 000000000..a194ace97
--- /dev/null
+++ b/docs/memory/feature-flows/web-chat-file-upload.md
@@ -0,0 +1,306 @@
+# Feature: Web Chat File Upload
+
+## Overview
+Adds drag-and-drop / file-picker to authenticated chat (ChatPanel.vue) and public chat (PublicChat.vue), encoding files as base64 in the JSON request body and reusing the same validation/write infrastructure as the Telegram/Slack/WhatsApp channel adapters.
+
+## User Story
+As a user chatting with an agent, I want to attach files to my message so that the agent can visually analyze images or read text/CSV/JSON files I provide.
+
+## Entry Points
+- **UI (auth)**: `src/frontend/src/components/chat/ChatInput.vue` — paperclip button / drag-and-drop on wrapper div
+- **UI (public)**: `src/frontend/src/views/PublicChat.vue:745` — same ChatInput component embedded in public chat
+- **API (auth)**: `POST /api/agents/{name}/task`
+- **API (public)**: `POST /api/public/chat/{token}`
+
+## Frontend Layer
+
+### Components
+
+**`src/frontend/src/components/chat/ChatInput.vue`**
+
+Key state (lines 272-274):
+- `pendingFiles` ref — `[{name, mimetype, size, data_base64}]`
+- `fileInputRef` ref — hidden ` `
+- `dragOver` ref — boolean for drag-over highlight
+
+Client-side limits (lines 226-227):
+```javascript
+const MAX_FILES = 3
+const MAX_FILE_BYTES = 5 * 1024 * 1024 // 5 MB
+```
+
+File encoding (lines 404-425):
+```javascript
+function encodeFile(file) {
+ return new Promise((resolve) => {
+ const reader = new FileReader()
+ reader.onload = (e) => resolve(e.target.result) // data: URI
+ reader.readAsDataURL(file)
+ })
+}
+
+async function addFiles(fileList) {
+ // Reads each file via FileReader.readAsDataURL → data: URI string
+ // Parses MIME from data: URI prefix (not file.type — unreliable cross-browser)
+ // Checks size <= MAX_FILE_BYTES; alerts and skips oversized files
+ pendingFiles.value.push({ name, mimetype, size, data_base64 })
+}
+```
+
+Drop / picker handlers (lines 428-442):
+- `onFileInputChange` — triggered by hidden ` `; resets input value to allow re-selection
+- `onDrop` — handles `@drop.prevent` on wrapper div; delegates to `addFiles`
+- `removeFile(idx)` — removes chip from preview list
+
+Submit (lines 445-457):
+```javascript
+function handleSubmit() {
+ emit('submit', localMessage.value.trim(), [...pendingFiles.value])
+ pendingFiles.value = []
+}
+```
+
+**`src/frontend/src/components/ChatPanel.vue:606`**
+```javascript
+const sendMessage = async (userMessage, files = []) => {
+ const payload = {
+ message: contextPrompt,
+ async_mode: true,
+ files: files.length > 0 ? files : undefined,
+ // ...other fields
+ }
+ // POST /api/agents/{name}/task
+}
+```
+
+**`src/frontend/src/views/PublicChat.vue:745`**
+```javascript
+const sendMessage = async (userMessage, files = []) => {
+ const payload = {
+ message: userMessage,
+ async_mode: true,
+ files: files.length > 0 ? files : undefined,
+ }
+ // POST /api/public/chat/{token}
+}
+```
+
+### nginx
+`src/frontend/nginx.conf:8`:
+```nginx
+client_max_body_size 25m;
+```
+Required because 3 files × 5 MB each, base64-encoded in JSON, is approximately 21 MB. Without this directive nginx silently rejects the request with 413 before it reaches the backend.
+
+## Backend Layer
+
+### Models
+
+**`src/backend/db_models.py:430`** — `WebFileUpload`:
+```python
+class WebFileUpload(BaseModel):
+ name: str
+ mimetype: str
+ size: int
+ data_base64: str # raw base64 or data: URI from FileReader.readAsDataURL()
+```
+
+**`src/backend/db_models.py:444`** — `PublicChatRequest.files`:
+```python
+files: Optional[List[WebFileUpload]] = None # (#364)
+```
+
+**`src/backend/models.py:99`** — `ParallelTaskRequest.files`:
+```python
+files: Optional[List[WebFileUpload]] = None # (#364)
+```
+
+### Endpoints
+
+**Authenticated chat** — `src/backend/routers/chat.py:858-894` (`execute_parallel_task`):
+- File processing block runs synchronously before the async/sync fork
+- `session_id` is `str(current_user.id)` (stable per user, scopes the upload directory)
+- `uploader` is `current_user.email or current_user.username`
+- On `all_writes_failed=True` → `502` with `"File upload failed: could not write to agent workspace."`
+- Appends file descriptions to `request.message` as a newline-joined block
+
+**Public chat** — `src/backend/routers/public.py:493-527` (`public_chat`):
+- Same pattern; `session_id` is `session_identifier` (the anonymous token or email)
+- `uploader` is `verified_email or f"anonymous ({client_ip})"`
+- File descriptions appended to `context_prompt` (not `chat_request.message`) so the stored user message doesn't contain the injection block
+- Images passed as `images=_pub_image_data` to both `_execute_public_chat_background` and `execute_task`
+
+### Shared Upload Service
+
+**`src/backend/services/upload_service.py`** — extracted from `adapters/message_router._handle_file_uploads()`.
+
+**`decode_web_file(f: dict) -> Optional[bytes]`** (line 345):
+- Strips `data:mime;base64,` prefix from FileReader output
+- Falls back to raw base64 if no prefix present
+- Returns `None` on decode failure
+
+**`sanitize_filename(name, file_id, used_names) -> str`** (line 62):
+- NFKC normalize → `os.path.basename` → strip unsafe chars via `[^\w.\-()]` regex
+- Rejects hidden filenames starting with `.` (e.g., `.env`, `.gitignore`)
+- Truncates to 200 chars (preserves extension up to 16 chars)
+- Collision dedup: appends `-1`, `-2`, … suffix
+
+**`process_file_uploads(raw_files, agent_name, container, session_id, uploader, source, ...) -> (descriptions, upload_dir, all_writes_failed, image_data)`** (line 119):
+
+```
+for each file (up to max_files):
+ 1. sanitize_filename() — NFKC, path traversal, dedup
+ 2. Reject unsupported MIME categories (PDF, ZIP, TAR, video/, audio/)
+ 3. Actual size check against declared limit (TOCTOU defense)
+ 4. Magic-byte MIME validation via python-magic (graceful fallback):
+ - Image MIME mislabel (JPEG vs PNG) → accept with detected MIME
+ - text/plain vs text/csv → accept
+ - Other mismatch → reject with "file type mismatch"
+ 5a. If image → base64-encode → append to image_data list (vision blocks)
+ 5b. If non-image → container mkdir -p + put_archive to /home/developer/uploads/{session_id}/
+ 6. Emit platform_audit_service.log(event_type=EXECUTION, event_action="file_upload")
+```
+
+Returns:
+- `descriptions` — list of context strings injected into the agent prompt
+- `upload_dir` — container path for cleanup, or `None` if no non-image writes occurred
+- `all_writes_failed` — `True` when at least one write was attempted but all failed
+- `image_data` — `[{"media_type": str, "data": base64_str}]` for `execute_task(images=...)`
+
+**Constants** (lines 35-43):
+| Constant | Value |
+|----------|-------|
+| `WEB_MAX_FILES` | 3 |
+| `WEB_MAX_FILE_SIZE` | 5 MB |
+| `WEB_MAX_IMAGE_SIZE` | 5 MB |
+| `WEB_MAX_TOTAL_IMAGE_SIZE` | 10 MB |
+| `CHANNEL_MAX_FILES` | 10 |
+| `CHANNEL_MAX_FILE_SIZE` | 10 MB |
+
+### Business Logic
+
+1. Frontend encodes file via `FileReader.readAsDataURL` → `data: URI`
+2. `decode_web_file()` strips prefix → raw bytes
+3. `process_file_uploads()` validates, MIME-checks, and routes:
+ - Images → base64 collected in `image_data` list
+ - Non-images → written to `/home/developer/uploads/{session_id}/{filename}` in the running container via Docker `put_archive`
+4. File descriptions appended to the message text
+5. `task_execution_service.execute_task(images=image_data)` invokes Claude Code with `--input-format stream-json`, delivering images as vision content blocks (not embedded text)
+6. Text files are readable by the agent at `/home/developer/uploads/{session_id}/`
+
+### Docker Operations
+- `container_exec_run(container, f"mkdir -p {upload_dir}", user="developer")` — create upload directory
+- `container_put_archive(container, upload_dir, tar_bytes)` — write file into container via tar stream with uid/gid 1000, mode 0o644
+
+## Agent Layer
+
+Images are passed through to Claude Code via `--input-format stream-json` as content blocks. There is no agent-server endpoint involved; the backend writes files directly into the agent container (Docker SDK) and passes image bytes through the existing `execute_task` path.
+
+## Side Effects
+
+- **Audit log**: `platform_audit_service.log(event_type=EXECUTION, event_action="file_upload")` fires once per successfully processed file; records filename, size, MIME, storage type (`stream_json_vision` for images, `container_file` for non-images), uploader, sender_id, channel_id, agent_name
+- **No WebSocket broadcast** specific to file uploads; the enclosing chat execution broadcasts normally via activity tracking
+
+## Error Handling
+
+| Error Case | HTTP Status | Detail |
+|------------|-------------|--------|
+| All non-image writes fail | 502 | `"File upload failed: could not write to agent workspace."` |
+| Single file rejected (size) | — | Description appended to prompt: `"{name} — rejected (exceeds X limit)"` |
+| Single file rejected (MIME mismatch) | — | Description: `"{name} — rejected (file type mismatch)"` |
+| Unsupported format (PDF, ZIP, video) | — | Description: `"{name} — unsupported format ({mime}). Text, CSV, JSON, and image files are supported."` |
+| File count exceeds max | — | Description: `"({n} more file(s) skipped — max {max} per message)"` |
+| Total image size exceeded | — | Description: `"{name} — skipped (total image size limit reached)"` |
+| Client-side oversized file | — | `alert()` shown in browser, file not added to `pendingFiles` |
+| nginx body too large | 413 | nginx rejects before backend (prevented by `client_max_body_size 25m`) |
+| base64 decode failure | — | `decode_web_file()` returns `None`; file processed as download-failed |
+
+## Complete Flow Diagram
+
+```
+User selects/drops files in ChatInput.vue
+ → FileReader.readAsDataURL → data: URI string
+ → addFiles() checks size <= 5 MB, parses MIME from URI prefix
+ → pendingFiles [{name, mimetype, size, data_base64}]
+ → handleSubmit() emits ('submit', message, files[])
+ ↓
+ChatPanel.vue sendMessage(msg, files) PublicChat.vue sendMessage(msg, files)
+ payload = { message, files, async_mode, … } payload = { message, files, async_mode, … }
+ POST /api/agents/{name}/task POST /api/public/chat/{token}
+ ↓
+nginx (client_max_body_size 25m)
+ ↓
+routers/chat.py:execute_parallel_task routers/public.py:public_chat
+ line 858-894 line 493-527
+ decode_web_file() — strips data: URI prefix
+ process_file_uploads() [services/upload_service.py]
+ ↓
+upload_service.process_file_uploads():
+ for each file (max 3):
+ sanitize_filename() — unicode NFKC, path traversal, dedup
+ reject unsupported MIME (PDF, ZIP, video, audio)
+ size check vs WEB_MAX_FILE_SIZE / WEB_MAX_IMAGE_SIZE
+ magic-byte MIME validation (python-magic; fallback graceful)
+ if image:
+ base64-encode → append to image_data list
+ audit log: storage=stream_json_vision
+ if non-image:
+ docker exec mkdir -p /home/developer/uploads/{session_id}/
+ docker put_archive → file written to container
+ audit log: storage=container_file
+ returns (descriptions, upload_dir, all_writes_failed, image_data)
+ ↓
+router appends descriptions to message text
+router calls task_execution_service.execute_task(images=image_data)
+ ↓
+execute_task → claude code --input-format stream-json
+ images delivered as vision content blocks (not text data URIs)
+ text files readable at /home/developer/uploads/{session_id}/
+```
+
+## Testing
+
+### Prerequisites
+- Backend running with Docker socket access
+- At least one running agent container
+- python-magic installed (or tests confirm graceful fallback path)
+
+### Unit Tests
+`tests/unit/test_web_file_upload.py` — 17 unit tests covering:
+- `sanitize_filename`: path traversal, unicode normalization, hidden-file rejection, truncation, collision dedup
+- `decode_web_file`: data: URI prefix stripping, raw base64, empty input
+- `process_file_uploads`: size limits, unsupported MIME gating, image dispatch, non-image container write, `all_writes_failed` detection
+
+### Integration Test Steps
+
+1. **Drag an image onto the chat input**
+ Expected: preview chip appears above textarea with filename; chip is removable
+ Verify: `pendingFiles` has one entry with `mimetype` parsed from data: URI
+
+2. **Submit message with image attachment**
+ Expected: POST payload includes `files` array; image appears in agent context as a vision block
+ Verify: Claude Code receives image via `--input-format stream-json`; agent can describe image content
+
+3. **Attach a text file (`.csv`)**
+ Expected: file written to `/home/developer/uploads/{session_id}/file.csv` in container
+ Verify: `docker exec {container} ls /home/developer/uploads/` shows the file
+
+4. **Exceed 5 MB limit client-side**
+ Expected: `alert()` shown; file not added to pending list
+
+5. **Attempt to attach a PDF**
+ Expected: file sent to backend; rejected by `process_file_uploads` with unsupported format description injected into prompt
+
+6. **Exceed 3-file limit**
+ Expected: only first 3 files processed; overflow count injected into prompt
+
+7. **Public chat file upload**
+ Expected: same behavior via `POST /api/public/chat/{token}`; `uploader` shows email or anonymous IP
+
+## Related Flows
+- [authenticated-chat-tab.md](feature-flows/authenticated-chat-tab.md) — ChatPanel context and session management
+- [public-agent-links.md](feature-flows/public-agent-links.md) — public chat route and session handling
+- [slack-file-sharing.md](feature-flows/slack-file-sharing.md) — original channel adapter file upload (shared infrastructure)
+- [telegram-integration.md](feature-flows/telegram-integration.md) — Telegram file upload Phase 1/2 (shared `process_file_uploads`)
+- [parallel-headless-execution.md](feature-flows/parallel-headless-execution.md) — `execute_task` and `--input-format stream-json`
+- [audit-trail.md](feature-flows/audit-trail.md) — platform_audit_service used for file upload events
diff --git a/docs/memory/requirements.md b/docs/memory/requirements.md
index 7f3c868a5..056546998 100644
--- a/docs/memory/requirements.md
+++ b/docs/memory/requirements.md
@@ -284,6 +284,11 @@ Trinity is autonomous agent orchestration and infrastructure — sovereign infra
- **Key Features**: Docker socket capture, VRL transforms, platform.json/agents.json output
- **Flow**: `docs/memory/feature-flows/vector-logging.md`
+### 8.8 Frontend E2E Test Infrastructure
+- **Status**: ✅ Implemented (2026-04-29)
+- **Description**: Playwright-based smoke test harness for the Trinity frontend, gated on the `ui` PR label in CI (#556)
+- **Key Features**: Chromium-only smoke suite (dashboard, agents, operating room, templates), storage-state auth pattern (login once, reuse session), label-gated CI workflow (~5 min, opt-in), on-failure artifact upload (screenshots, videos, Trinity logs)
+
---
## 9. Agent Collaboration
@@ -373,6 +378,23 @@ Trinity is autonomous agent orchestration and infrastructure — sovereign infra
- Configurable `POLL_INTERVAL` env var (default 10s)
- **Root Cause**: TCP connection drops after 15-30 min on long-running scheduled tasks, causing false `failed` status even though agent work completed successfully
+### 10.6.1 Conditional Schedule Pre-Check (SCHED-COND-001)
+- **Status**: ✅ Implemented (2026-04-22)
+- **Requirement ID**: SCHED-COND-001
+- **GitHub Issue**: #454
+- **Description**: Optional template-supplied hook that lets a scheduled cron tick be skipped deterministically — the scheduler calls a new internal backend endpoint which `docker exec`s the executable `~/.trinity/pre-check` file inside the target agent container; non-empty stdout becomes the chat prompt, empty stdout + exit 0 records a skipped execution. The hook is language-agnostic (interpreter chosen by shebang). Eliminates Claude token cost on empty polls for poll-driven agents (PR reviewers, inbox monitors, alert routers, RSS watchers).
+- **Key Features**:
+ - Contract: agent templates drop an executable `~/.trinity/pre-check` file with a shebang (`#!/usr/bin/env python3`, `#!/bin/bash`, …). Trinity execs it directly — no `python3` prefix, no language assumption. Stdout is the chat prompt; empty stdout + exit 0 = skip; non-zero exit = fail-open.
+ - Backend endpoint: `POST /api/internal/agents/{name}/pre-check` (X-Internal-Secret gated) runs the script via `execute_command_in_container` — the same primitive used by `git_service.py` (persistent-state allowlist, #384 S3), `ssh_service.py`, `agent_service/terminal.py`, `adapters/message_router.py`, `routers/system_agent.py`, `routers/voice.py`.
+ - Fail-open: script absent, non-zero exit, timeout, backend 5xx / malformed response → scheduler fires as usual. A broken pre-check never silently suppresses scheduled work.
+ - Message override: non-empty stdout replaces `schedule.message` for that one invocation — lets the agent inject real work items (e.g. the PR list) into the chat prompt.
+ - Skip record: empty stdout writes a row to `schedule_executions` with `status='skipped'`, reason, and zero cost — visible in the Trinity UI alongside successful runs.
+ - Manual triggers bypass pre-check entirely (explicit operator intent always fires).
+ - Zero DB schema change (reuses existing `ExecutionStatus.SKIPPED` + `create_skipped_execution`).
+ - **No new HTTP edge**: scheduler calls backend, backend `docker exec`s into agent. Topology stays "scheduler → backend → agent" (Invariant #11).
+- **Test plan**: 13 unit tests covering backend-response translation (hook absent / non-zero exit / empty stdout / fire-with-message / 404 / 5xx / connection error / malformed JSON) + scheduler branch behaviors (skip, override, fail-open, manual-bypass). Full 162-test scheduler suite passes.
+- **Root Cause**: No platform primitive for "deterministic gate before LLM invocation." Previously required per-template daemons backgrounded inside agent containers — invisible to Trinity UI, reimplemented per template, no skip metrics.
+
### 10.7 Per-Agent Execution Timeout (TIMEOUT-001)
- **Status**: ✅ Implemented (2026-03-12)
- **Requirement ID**: TIMEOUT-001
@@ -388,10 +410,10 @@ Trinity is autonomous agent orchestration and infrastructure — sovereign infra
- **Flow**: `docs/memory/feature-flows/parallel-capacity.md` (updated), `docs/memory/feature-flows/task-execution-service.md` (updated)
### 10.8 Persistent Task Backlog (BACKLOG-001)
-- **Status**: ✅ Implemented (2026-04-13)
+- **Status**: ✅ Implemented (2026-04-13); internalized behind `CapacityManager` (#428, 2026-04-26)
- **Requirement ID**: BACKLOG-001
-- **GitHub Issue**: #260
-- **Description**: Async `/task` requests that arrive at full parallel capacity now spill into a durable SQLite-backed FIFO backlog instead of returning HTTP 429. Queued items drain automatically when slots free via a `SlotService` release callback; 60s maintenance task expires stale rows and drains orphans after restart.
+- **GitHub Issue**: #260, internalized by #428
+- **Description**: Async `/task` requests that arrive at full parallel capacity spill into a durable SQLite-backed FIFO backlog instead of returning HTTP 429. Reached via the unified `CapacityManager.acquire(..., overflow_policy="queue_persistent", overflow_payload=...)` facade; queued items drain automatically when slots free via the manager's release-callback wiring; 60s `CapacityManager.run_maintenance()` tick expires stale rows and drains orphans after restart.
- **Key Features**:
- New `QUEUED` value on `TaskExecutionStatus`; reuses `schedule_executions` with `queued_at` + `backlog_metadata` columns
- Partial index `idx_executions_queued` for cheap O(log n) FIFO claim via atomic `UPDATE ... RETURNING`
@@ -658,6 +680,29 @@ Trinity is autonomous agent orchestration and infrastructure — sovereign infra
- Resume banner in Chat tab showing execution context
- **Spec**: `docs/requirements/CONTINUE_EXECUTION_AS_CHAT.md`
+### 13.10 Outbound File Sharing (FILES-001)
+- **Status**: ✅ Implemented (2026-04-24)
+- **Requirement ID**: FILES-001
+- **GitHub Issue**: #295
+- **Priority**: P1
+- **Description**: Agents publish files to a public download URL with token-based auth, 7-day default expiration, and inheritance of the agent's channel-access policy. The URL is a universal delivery mechanism that works across web, Slack, Telegram, WhatsApp, and email — replacing fragile per-channel workarounds.
+- **Key Features**:
+ - Per-agent opt-in toggle + Docker volume `agent-{name}-public` mounted at `/home/developer/public/`
+ - `share_file` MCP tool (agent-scoped) — publishes a file and returns a download URL
+ - Internal endpoint `POST /api/internal/agent-files/share` (agent-server path, `X-Internal-Secret` auth)
+ - MCP-path endpoint `POST /api/agents/{name}/shared-files` (owner/admin or agent-scoped key)
+ - Public download endpoint `GET /api/files/{file_id}?sig={token}` — 192-bit signed token, constant-time compare, streaming, `Content-Disposition: attachment`, `X-Content-Type-Options: nosniff`, audit logged as `file_share_download`
+ - List / revoke endpoints for the owner (`GET` / `DELETE /api/agents/{name}/shared-files[/{id}]`)
+ - UI panel in Agent Detail → Sharing tab (toggle, quota, table, copy URL, revoke)
+ - File validation: relative path only, no `..` escapes, 50 MB per file, 500 MB per-agent quota, magic-byte MIME detection with executable blocklist (PE/ELF/Mach-O/shebang)
+ - Agent delete cascades: DB rows + on-disk files + Docker volume all removed
+ - Agent rename cascades: `rename_agent()` in `db/agent_settings/metadata.py` updates our table
+- **Database**: `agent_shared_files` table + `agent_ownership.file_sharing_enabled` column (FK `ON DELETE CASCADE ON UPDATE CASCADE`, though enforcement is via the manual-cascade pattern used platform-wide)
+- **Security (audited)**: path traversal rejection, filesystem isolation (backend never mounts agent workspace; `docker get_archive` only pulls the agent-named file), agent-scope defense (agent-scoped MCP keys can't share files for a different agent), no `download_token` param name (renamed to `sig` to avoid credential-sanitizer redaction)
+- **Deferred (tracked for future)**: one-time download links (schema columns retained), platform-wide storage cap, streaming tar extraction, UUID-prefix directory sharding, dedicated rate-limit bucket
+- **Design doc**: `docs/drafts/amazing-file-outbound.md`
+- **Flow**: `docs/memory/feature-flows/file-sharing-outbound.md`
+
---
## 14. Multi-Runtime Support
@@ -1820,40 +1865,46 @@ Standalone mobile-friendly admin page for managing agents on the go. Designed as
## 29. Voice Chat (VOICE-001)
### 29.1 Voice Session Initialization (VOICE-001)
-- **Status**: 🚧 In Progress (Phase 1 MVP)
+- **Status**: ✅ Implemented
- **Description**: Real-time voice conversations with agents via Gemini 2.5 Flash Native Audio
- **Key Features**: `POST /api/agents/{name}/voice/start` loads voice prompt + summarizes prior chat, opens Gemini Live API WebSocket connection
- **Architecture**: Browser (mic) → WebSocket → Backend (proxy) → Gemini Live API → Backend → WebSocket → Browser (speaker)
### 29.2 Audio Streaming Bridge (VOICE-002)
-- **Status**: 🚧 In Progress
+- **Status**: ✅ Implemented
- **Description**: WebSocket proxy: browser ↔ backend ↔ Gemini, <100ms added latency
- **Key Features**: PCM 16kHz mono input, 24kHz mono output, base64 frame encoding
### 29.3 Transcript Persistence (VOICE-003)
-- **Status**: 🚧 In Progress
+- **Status**: ✅ Implemented
- **Description**: Voice transcripts saved as ChatMessage rows with `source="voice"`, inline in existing chat sessions
- **Key Features**: Automatic transcript extraction from Gemini, `source` column on chat_messages table
### 29.4 Frontend Voice UI (VOICE-004)
-- **Status**: 🚧 In Progress
+- **Status**: ✅ Implemented
- **Description**: Mic button next to chat input, voice overlay with status/mute/end controls
- **Key Features**: VoiceOverlay component, pulsing status indicators, live transcript display, mute toggle
### 29.5 Voice System Prompt (VOICE-005)
-- **Status**: 🚧 In Progress
+- **Status**: ✅ Implemented
- **Description**: Per-agent `voice_system_prompt` field for voice personality
- **Key Features**: Stored on agent_ownership table, fallback to auto-generated prompt from agent name
### 29.6 Context Summary (VOICE-006)
-- **Status**: 🚧 In Progress
+- **Status**: ✅ Implemented
- **Description**: On voice start, summarize prior messages and inject into Gemini system prompt
- **Key Features**: Last 20 messages truncated to ~750 tokens, injected as conversation context
+### 29.7 Tool Calls + Canvas Orb (VOICE-007)
+- **Status**: ✅ Implemented (#581)
+- **Description**: Gemini voice sessions can invoke Trinity's `run_task` tool to dispatch agent tasks mid-conversation; frontend canvas orb replaces the static overlay
+- **Key Features**: Single `run_task` tool declaration sent to Gemini Live API; non-blocking `asyncio.create_task` dispatch with 30s timeout; prompt injection mitigation (`_TOOL_PROMPT_MAX = 2000` chars); `_pending_tool_tasks` dict with cancellation on session end; canvas orb in `VoiceOverlay.vue` with `isToolCalling` state (no CDN dependencies); platform audit log on every tool call
+- **Architecture**: Gemini → `tool_call` WS message → `_execute_and_respond()` → `POST /api/agents/{name}/chat` → Gemini `tool_response`
+
### Phase Roadmap
-1. **Phase 1 (MVP)**: Authenticated chat only, basic overlay, transcript on session end, manual voice prompt
-2. **Phase 2 (Polish)**: Real-time waveform, incremental transcript, auto-generate voice prompt from CLAUDE.md
-3. **Phase 3 (Advanced)**: Function calling, multi-language auto-detection, custom voice per agent
+1. **Phase 1 (MVP)**: Authenticated chat only, basic overlay, transcript on session end, manual voice prompt ✅
+2. **Phase 2 (Polish)**: Real-time waveform, incremental transcript, auto-generate voice prompt from CLAUDE.md ✅
+3. **Phase 3 (Advanced)**: Tool calling (run_task), canvas orb ✅ — multi-language auto-detection, custom voice per agent (deferred)
---
diff --git a/docs/planning/CANARY_HARNESS_PHASE_1.md b/docs/planning/CANARY_HARNESS_PHASE_1.md
new file mode 100644
index 000000000..176bb2ab8
--- /dev/null
+++ b/docs/planning/CANARY_HARNESS_PHASE_1.md
@@ -0,0 +1,216 @@
+# Canary Invariant Harness — Phase 1 Design
+
+**Date:** 2026-04-27
+**Status:** Proposal — implements Phase 1 of [#411](https://github.com/Abilityai/trinity/issues/411).
+**Reference:** [`docs/testing/orchestration-invariant-catalog.md`](../testing/orchestration-invariant-catalog.md)
+
+---
+
+## Scope
+
+Catalog's Phase 1 subset is 12 invariants. AC of #411 requires three running continuously on staging at deploy time (S-01, E-02, L-03); the remaining 9 ship as follow-up PRs against the same infrastructure.
+
+This doc specifies the full Phase 1 design (all 12) so each follow-up is purely an additive code change.
+
+## Invariant coverage
+
+Each row gives the check semantics and what the snapshot collector must capture for it.
+
+| ID | Check (one-line) | Snapshot inputs |
+|---|---|---|
+| **S-01** | Per agent A: `Redis ZRANGE agent:slots:A` (minus drain sentinels < 5s old) == `SQL exec_ids WHERE agent_name=A AND status='running'` | Redis ZSET, SQL running rows |
+| **S-02** | Per agent A: `ZCARD agent:slots:A ≤ agent_ownership.max_parallel_tasks` | Redis ZCARD, SQL max_parallel |
+| **S-03** | Per slot member: `TTL agent:slot:A:{eid} ≥ timeout_seconds + 300` | Redis TTL, per-execution timeout (schedule override or agent default) |
+| **E-01** | `SQL count WHERE status='running' AND started_at < now() - (timeout + 300s)` == 0 | SQL running rows + timeouts |
+| **E-02** | No `update_execution_status` log line shows `terminal_state → non_terminal_state` since last snapshot | Vector log diff (`update_execution_status` lines) |
+| **E-05** | `SQL count WHERE status='running' AND started_at < now()-60s AND claude_session_id IS NULL` == 0 | SQL running rows |
+| **E-06** | For every SQL row `status='running' AND started_at < now()-60s`, exec_id appears in agent's `GET /api/executions/running` | SQL running rows + agent registry |
+| **B-01** | Per agent A: `backlog.get_queued_count(A) == SQL count WHERE status='queued' AND agent_name=A` | Backlog state, SQL queued rows |
+| **B-02** | If queued count > 0, then `ZCARD agent:slots:A == max_parallel_tasks` (or drain pending ≤60s) | SQL queued, Redis ZCARD, SQL max_parallel |
+| **L-03** | Orphan scan: no row in {agent_sharing, agent_schedules, schedule_executions (non-terminal), agent_permissions, agent_event_subscriptions, mcp_api_keys (scope='agent'), slack_channel_agents, agent_shared_folder_config, chat_sessions (active)} references an `agent_name` not in agent_ownership; no Redis `agent:slots:{name}` for missing agent | SQL multi-table joins, Redis KEYS |
+| **G-01** | After backend restart: no `status='running'` SQL row without matching agent registry entry, after startup-sweep + 5min grace | SQL running rows + agent registry (post-restart only) |
+| **R-01** | Per running agent container: `docker exec ps -eo stat,comm` shows zero zombie `claude` processes | Container exec output |
+
+## Snapshot format
+
+Single typed dataclass returned by the collector. One snapshot per check cycle.
+
+```python
+@dataclass
+class Snapshot:
+ timestamp: datetime
+ agents: list[AgentSnapshot]
+ transitions_since_last: list[StatusTransition] # for E-02
+ orphan_refs: dict[str, list[OrphanRef]] # for L-03; keyed by table
+
+@dataclass
+class AgentSnapshot:
+ name: str
+ container_running: bool
+ max_parallel: int
+ execution_timeout_seconds: int
+ # Redis
+ slot_ids: set[str]
+ slot_ttls: dict[str, int] # eid -> TTL seconds
+ # SQL
+ running_exec_ids: set[str]
+ queued_exec_ids: set[str]
+ overdue_running_ids: set[str] # started_at < now() - (timeout+300)
+ no_session_running_ids: set[str] # claude_session_id IS NULL > 60s
+ # Backlog service
+ backlog_queued_count: int
+ # Agent registry
+ registry_running_ids: set[str] | None # None if agent unreachable this cycle
+ # Container exec
+ zombie_claude_count: int | None # None if exec failed
+
+@dataclass
+class StatusTransition:
+ execution_id: str
+ from_status: str
+ to_status: str
+ timestamp: datetime
+ log_source: str # vector log id
+
+@dataclass
+class OrphanRef:
+ table: str
+ column: str
+ referenced_agent_name: str
+ row_id: str
+```
+
+The collector lives at `src/canary/snapshot.py`. Public API:
+
+```python
+async def collect_snapshot(
+ *,
+ since: datetime | None = None, # for transitions_since_last
+ include_zombies: bool = True,
+) -> Snapshot
+```
+
+Reusable from unit tests by passing fixtures. Phase 2's scenario runner uses the same primitive.
+
+### Snapshot sources
+
+| Source | Access | Used by |
+|---|---|---|
+| SQLite | Backend API (per catalog rec for Phase 1 — enforces real code path) | S-01, S-02, S-03, E-01, E-05, E-06, B-01, B-02, L-03, G-01 |
+| Redis | `ZRANGE`/`ZCARD`/`TTL`/`KEYS` via MCP tool | S-01, S-02, S-03, B-02, L-03 |
+| Agent registries | Parallel `GET /api/executions/running` via `asyncio.gather` | E-06, G-01 |
+| Container exec | `docker exec {name} ps -eo stat,comm` | R-01 |
+| Vector logs | Read JSON log file diff since last snapshot timestamp | E-02 |
+
+Agent-unreachable cases set the relevant fields to `None` rather than raising; affected invariants skip the cycle for that agent (mirrors the cleanup service's "skip-and-retry-next-cycle" pattern from PR #403).
+
+## Invariant library
+
+Twelve pure functions: `check(snapshot) → list[ViolationReport]`. Each ~30-50 LOC.
+
+```
+src/canary/invariants/
+ s01_slot_row_bijection.py
+ s02_no_overbooking.py
+ s03_slot_ttl.py
+ e01_terminal_closure.py
+ e02_no_phantom_reversal.py
+ e05_dispatched_has_session.py
+ e06_no_completed_unreported.py
+ b01_queue_status_coherence.py
+ b02_queued_implies_full.py
+ l03_delete_cascades.py
+ g01_no_restart_leak.py
+ r01_no_zombies.py
+```
+
+`ViolationReport` matches the `canary_violations` schema below.
+
+## Fleet
+
+Pre-seeded agents the canary observes. All run Claude Code (no architectural bypass exists); minimize cost via trivial prompts.
+
+Fleet is designed against the invariant set: each agent exists for specific invariants.
+
+| Agent | Settings | Schedule | Exercises |
+|---|---|---|---|
+| `canary-burst` | `max_parallel=1`, default 15min timeout | every 30s, short prompt (`"reply ok"`) | S-01, S-02, B-01, B-02, E-02, E-05, R-01 |
+| `canary-long` | `max_parallel=2`, custom 45min timeout | every 5min, multi-tool prompt (~10–60s) | S-03, E-01, E-06, R-01 |
+| `canary-rotate-{ts}` | default | hourly create + delete by canary skill | L-03 |
+
+**Why these and not others:**
+- `canary-burst` cadence (30s) < expected task duration so backlog overflows — exercises B-01/B-02 organically.
+- `canary-long` is the only agent with non-default timeout — without it, S-03 has nothing to check.
+- `canary-rotate-{ts}` is the only active-mutation agent in Phase 1 (everything else is observation). Without it L-03 sits vacuously green.
+- G-01 (restart leak) needs a backend restart, not a fleet member.
+
+For the AC's 3-invariant initial deploy: only `canary-burst` + `canary-rotate-{ts}` are required (S-01 and E-02 fire on `canary-burst` traffic, L-03 on rotation). `canary-long` is added when S-03/E-01/E-06 invariants are wired up.
+
+Naming follows catalog §Open Questions: `canary-*` prefix, dedicated synthetic operator user.
+
+## Database migration
+
+```sql
+CREATE TABLE canary_violations (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ invariant_id TEXT NOT NULL, -- 'S-01', 'E-02', 'L-03', ...
+ tier TEXT NOT NULL, -- 'A' or 'B'
+ severity TEXT NOT NULL, -- 'critical', 'major', 'minor'
+ snapshot_time TEXT NOT NULL,
+ observed_state TEXT NOT NULL, -- JSON payload, invariant-specific
+ signal_query TEXT, -- the check that fired (debugging aid)
+ created_at TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+CREATE INDEX idx_canary_violations_invariant ON canary_violations(invariant_id, snapshot_time DESC);
+CREATE INDEX idx_canary_violations_severity ON canary_violations(severity, snapshot_time DESC);
+```
+
+Versioned migration in `src/backend/db/migrations.py`. Read endpoint: `GET /api/canary/violations` (admin-only, supports filters by invariant_id / severity / time range).
+
+## Canary agent template
+
+New template at `config/agent-templates/canary-invariant/`:
+
+- `template.yaml` — deletion-protected, owned by synthetic operator user
+- `CLAUDE.md` — operating instructions
+- `dashboard.yaml` — green/red widget per invariant + 24h violation trend per invariant
+- Scheduled skill `/check-invariants` every 5 min: `collect_snapshot()` → run all enabled invariants → write violations + push alerts
+
+Deletion-protected mirrors the `trinity-system` pattern.
+
+## Alert channel
+
+Three layers:
+
+1. **Persistent** — every violation written to `canary_violations`. Source of truth for trend queries and forensic replay.
+2. **Dashboard** — green/red per invariant + 24h sparklines via the agent's `dashboard.yaml`.
+3. **Push** — alert *only* on state transitions (green→red) and severity thresholds. Never on every check; otherwise operators learn to ignore them.
+
+**Push channel: TBD.** Slack / Telegram / email. Needs to be decided before staging deploy. Existing Slack/Telegram channel adapters can be reused (no new transport needed).
+
+## Rollout
+
+1. Migration + read endpoint
+2. Snapshot collector (full schema)
+3. S-01, E-02, L-03 invariants + tests + canary agent template + `canary-burst` and `canary-rotate` fleet
+4. Push alert wiring (channel decided)
+5. Deploy to staging; observe for 30 days → close AC
+6. (Follow-up PRs) S-02, S-03, E-01, E-05, E-06, B-01, B-02, G-01, R-01 invariants + add `canary-long` to fleet
+
+Each step is a separate PR.
+
+## Acceptance criteria mapping
+
+- ✅ Catalog reviewed → S-03 and E-05 added to Phase 1 subset (same PR)
+- ✅ Phase 1 design doc reviewed → this doc covers all 12 invariants, snapshot format, alert channel
+- 🔲 `canary_violations` table + snapshot-collector → steps 1-2
+- 🔲 First 3 invariants running on staging → steps 3-5
+- 🔲 One real violation caught and alerted (or 30 days clean) → step 5
+
+## Open questions
+
+1. **Staging deploy access** — does it exist; how to provision canary + fleet
+2. **Push alert channel** — Slack / Telegram / email
+3. **Snapshot retention** — keep raw snapshots for forensic replay or just violations
+4. **Container exec for R-01** — `docker exec` from the canary agent requires Docker socket access; alternative is exposing a `GET /api/zombies` endpoint on the agent server. Open for review.
diff --git a/docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md b/docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md
index acf4ce34e..6137aeca0 100644
--- a/docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md
+++ b/docs/planning/ORCHESTRATION_RELIABILITY_2026-04.md
@@ -3,7 +3,7 @@
**Date:** 2026-04-13 (revised 2026-04-20, 2026-04-26)
**Status:** Proposed sequencing for execution-time orchestration, event subscriptions, and multi-agent reliability.
-**Progress:** Sprint A — **7/7 complete**. Sprint B — **1/1 complete**. Sprint C — **3/5 complete**: #260 (PR #316), #271 (PR #332), #264 (PR #334). **#294 closed** (pause rationale vindicated — see Sprint C row). **#291 now unblocked** (was pending #430; #430 now shipped). Sprint D — **2/4 complete: #306 + #430 shipped.** **Next: #428 (CAPACITY-CONSOLIDATE) after 2-week soak on #306.**
+**Progress:** Sprint A — **7/7 complete**. Sprint B — **1/1 complete**. Sprint C — **5/5 complete**: #260 (PR #316), #271 (PR #332), #264 (PR #334), #291 (PR #484). **#294 closed** without implementation (pause rationale vindicated — see Sprint C row). Sprint D — **3/4 complete: #306 + #430 + #428 shipped.** Sprint D′ — **1/3 complete: #524 shipped (minimal scope — CAS guard + state machine doc; full projector deferred).** **Next: #429 (CLEANUP-COLLAPSE) after #428 soak, plus #525/#526 in parallel.**
**2026-04-20 revision:** After reviewing the accumulated orchestration surface (three queue abstractions, nine cleanup paths, twelve status-column writers, seven dispatch sites), the next priority shifted from finishing Sprint C to **push-based completion (#306) + consolidation** — see *Tier 2.5 — Simplification* below. The cleanup pyramid is load-bearing, so simplification is **additive-first**: new paths ship alongside old ones and the watchdog is retired only after push has soaked.
@@ -26,10 +26,13 @@ Shipping #260 on top of today's foundation would produce a *persistent* backlog
```
Sprint A (unblock): #95 ✅, #285 ✅, #226 ✅, #286 ✅, #61 ✅, #132 ✅, #56 ✅ ← COMPLETE
Sprint B (trace): #305 ✅ ← COMPLETE
-Sprint C (orchestrate): #260 ✅ → #271 ✅ → #264 ✅ → [#294 PAUSED] → [#291 PAUSED]
+Sprint C (orchestrate): #260 ✅ → #271 ✅ → #264 ✅ → #294 🚫 → #291 ✅
Sprint D (simplify): #306 ✅ → #428 (CAPACITY-CONSOLIDATE) → #429 (CLEANUP-COLLAPSE)
#430 ✅ (parallel — process engine deleted; archive on branch archive/process-engine)
(and #408 dissolves once #428 lands — verified 2026-04-24: long-running call still present, #306 alone insufficient)
+Sprint D′ (harden): #524 (state machine) → unblocks #429's terminal-writer guarantee
+ #525 (idempotency keys) — parallel
+ #526 (dispatch circuit breaker) — parallel; better with #307
Sprint E (telemetry): #307
Sprint F (scale): #24, #18
```
@@ -51,6 +54,62 @@ Sprint F (scale): #24, #18
---
+## Target architecture: actor model (destination)
+
+**Added 2026-04-26.** The tier-by-tier work above is the *path*. This section names the *destination* so each step has a coherent direction. Not a rewrite proposal — most components already exist in the right shape; they're wired as fallbacks rather than as the primary path.
+
+**The destination is the actor model.** Each agent is an actor with:
+
+1. **A mailbox** — durable per-agent inbox. Other agents and external triggers drop messages here. The mailbox *is* the queue; there is no separate dispatch path.
+2. **A journal** — append-only record in the agent's git repo. Replayable. Source of truth for "what happened to this agent," not a central platform table.
+3. **A processor** — pulls from mailbox, executes, appends to journal, emits reply messages. Configurable parallelism regulates throughput.
+
+The platform's job collapses to four things: deliver messages, supervise actors, project journals into observability (UI, audit), and translate human-facing sync (chat, webhook) into async messages via edge adapters.
+
+### Why this is reachable, not a rewrite
+
+| Actor model concept | Trinity component today | Gap to close |
+|---|---|---|
+| Mailbox | `BacklogService` (SQLite FIFO) | Today fires only on overflow — make it the only path in |
+| Processor + parallelism | `SlotService` (Redis ZSET) | Already correct in shape; becomes inbox consumer rate |
+| Journal | Agent git repo + `~/.trinity/operator-queue.json` | Generalize: every workflow checkpoint commits to repo |
+| Message transport | `EventBus` / Redis Streams (#306) | Add typed message envelope |
+| Single-writer state | `ExecutionStateProjector` (#524) | Projector = journal-to-DB mapping |
+| At-least-once delivery | Idempotency keys (#525) | Required by the protocol |
+| Per-actor health gate | Dispatch circuit breaker (#526) | Stops feeding mailboxes of dead actors |
+| Auto-scaling signal | Inbox depth | Falls out for free once mailbox is the only path |
+
+Rewiring, not rebuilding. Nothing on this list gets thrown away.
+
+### Transition roadmap (gated, evidence-driven)
+
+**Phase 1 — Ship Sprint D + D′ as planned.** Every issue in those sprints (#428, #429, #524, #525, #526) bends the architecture toward the actor model. **Do not pause to redesign — this *is* the path.**
+
+**Phase 2 — Prove the model on one boundary.** Pick the smallest, lowest-risk surface: **MCP `chat_with_agent` (agent → agent only)**. Route it through `BacklogService` as a typed message instead of as a sync HTTP call. One sprint. Same agent code on both sides. Human-facing `/chat` untouched.
+
+**Phase 3 — Decision gate.** Read the result honestly:
+- If message-passing is clearly simpler (fewer slots/timeouts/cleanup paths) and latency is acceptable → fold next boundary (`/task`, then schedules, then webhooks).
+- If hidden complexity surfaces (backpressure, discovery, debugging) → stay on the planned path. Sprint D + D′ still leaves the system in a much better place than today.
+
+**Phase 4 — Human chat stays sync via edge adapter.** WebSocket holds open, drops a message in the agent's inbox, waits for the reply, forwards it. Looks sync to the human; underneath it's async. Sync semantics live in a thin shim, not in the core.
+
+### Non-goals (explicitly)
+
+- Flag-day rewrite. The current platform stays load-bearing throughout.
+- Replacing components that already work in actor-shape (`SlotService`, `EventBus`, `BacklogService`).
+- Touching human-facing chat UX. Edge-adapter latency is acceptable; redesigning browser interactions is not on the table.
+
+### Pre-experiment artifact
+
+Before scheduling Phase 2, write one page with two sections:
+
+1. **Message envelope** — fields every inter-agent message carries (e.g., `id`, `from`, `to`, `correlation_id`, `causation_id`, `kind`, `payload`, `idempotency_key`, `deadline`).
+2. **Journal format** — what the agent appends to its repo per processed message (`journal.ndjson`? structured `state.yaml` per workflow?).
+
+If both fit cleanly on a postcard, the model is sound and Phase 2 proceeds. If either sprawls, the model isn't ready and the planned path remains the best one available.
+
+---
+
## Tier 0 — Fix state-corruption bugs (Sprint A)
**Goal:** Make execution state authoritative and correct. No new features.
@@ -104,7 +163,7 @@ Sprint F (scale): #24, #18
| ~~#271~~ ✅ | ~~Retry mechanism for scheduled executions~~ | **Shipped** in PR #332. Configurable `max_retries` (0-5, default 1) and `retry_delay_seconds` (30-600, default 60). Rate-limited (429) failures use 2x delay. Retries persist to DB and survive scheduler restart via `_recover_pending_retries()`. New status: `pending_retry`. |
| ~~#294~~ 🚫 | ~~Business task validation (VALIDATE-001)~~ **— CLOSED** | Closed without implementation. Pause rationale held up: a second full Claude session per task is a 2x cost feature that's better subsumed by cheaper in-process primitives (output schemas, post-hoc validators). No new execution machinery shipped. |
| ~~#264~~ ✅ | ~~Self-execute during chat (SELF-EXEC-001)~~ | **Shipped** in PR #334. Detects source==target, sets `X-Self-Task` header, optionally injects result back into chat session via `inject_result` parameter. Uses backlog for overflow when at capacity. |
-| #291 | Agent webhook triggers (WEBHOOK-001) **— UNBLOCKED (2026-04-24, #430 shipped)** | External → agent dispatch. Process engine deleted (#430), so "reuse process-engine triggers" option is gone — implement as a new trigger surface that funnels through `TaskExecutionService`. |
+| ~~#291~~ ✅ | ~~Agent webhook triggers (WEBHOOK-001)~~ | **Shipped** in PR #484 (2026-04-25; follow-up fix PR #493). New `routers/webhooks.py` exposes `POST /api/webhooks/{webhook_token}` (no JWT, rate-limited 10 calls/60s, returns 202). Per-schedule `webhook_token` (43-char `secrets.token_urlsafe(32)`) lives on `agent_schedules` with partial unique index for O(1) lookup; rotate via `POST .../webhook` (instantly invalidates old URL), revoke via DELETE. Optional `{"context": "..."}` body (≤4000 chars) wrapped in framing header before append to schedule message. All triggers audit-logged with `triggered_by="webhook"`. Funnels through the unified executor — no process-engine reuse. **Deviation from issue spec:** opaque token, not HMAC-signed URL — simpler revocation/rotation story; idempotency deferred to #525. |
### Architectural shift
@@ -151,7 +210,7 @@ Retry and validation are **not new infrastructure** — they're just new trigger
| # | Title | Why it's here |
|---|-------|---------------|
| ~~#306~~ ✅ | ~~Redis Streams event bus (RELIABILITY-003)~~ — keystone | **Shipped.** `services/event_bus.py` (`EventBus` publisher + `StreamDispatcher` consumer), `XADD`/`XREAD BLOCK`, reconnect replay via `last-event-id` query param (regex-gated, `REPLAY_GAP_LIMIT=5000` → `resync_required`), 3-failure client eviction, MAXLEN-trimmed stream. Frontend tracks `_eid` and handles `resync_required`. `ConnectionManager.broadcast()` now funnels through `XADD`. 2-week soak window before Tier 2.5 consolidation — track push success rate + orphan count. |
-| **NEW** | **#428 (CAPACITY-CONSOLIDATE)** | Merge `ExecutionQueue` + `SlotService` + `BacklogService` into one `CapacityManager` with `(max_concurrent, overflow_policy)` config. `/chat` = `(1, queue_in_memory)`. `/task` = `(N, queue_persistent)`. Depends on #306 so the drain/TTL logic has the event consumer to lean on. |
+| ~~#428~~ ✅ | ~~CAPACITY-CONSOLIDATE~~ | **Shipped (2026-04-26).** New `services/capacity_manager.py` is the single public facade for capacity (`acquire`/`release`/`status`/`reclaim_stale`/`force_release`). Composes `SlotService` (Redis ZSET counter) and `BacklogService` (SQL persistent overflow) as private internals; owns the in-memory overflow store (Redis LIST, depth 3, lifted from the deleted `ExecutionQueue`). `/chat` uses `(max_parallel_tasks, queue_in_memory)`; `/task` uses `(max_parallel_tasks, queue_persistent)`. Single path — no feature flag — per user direction; `dev`-soak + clean revert is the rollback mechanism. `ExecutionQueue` deleted (~360 LOC). 7 caller sites collapsed to one API. 21 new unit tests + 35 watchdog + 33 backlog tests pass. Wire format unchanged (Redis keys, SQL columns) so in-flight executions are unaffected. |
| **NEW** | **#429 (CLEANUP-COLLAPSE)** | Once agent is authoritative for "is this running?" (via push), retire Phase 1/1b/1c/3 reconciliation. Slot TTL disappears — capacity is recomputed from DB, not TTL'd. Target: 9 paths → 1 periodic `DB ⟷ agent./api/running` sync. **Do not ship until #306 has been in prod ≥2 weeks with zero observed orphans.** |
| ~~#430~~ ✅ | ~~Process Engine decision (PROCESS-ENGINE-DECISION)~~ | **Shipped (2026-04-24). Option B — delete.** `services/process_engine/` removed (~8 000 LOC). All PE routers (`processes`, `process_templates`, `executions`, `audit`, `alerts`, `approvals`, `triggers`) deleted. Frontend views/stores/components removed. Dead imports purged from `main.py`. Cost-alerts tab removed from OperatingRoom (was already 404ing). Archive preserved on branch `archive/process-engine`. Every future orchestration invariant now applies universally — no more "except process engine" footnotes. |
@@ -182,6 +241,41 @@ Worst case: new paths break and we fall back to existing paths. Old code gets de
---
+## Tier 2.6 — Reliability hardening (Sprint D′) — **NEW (2026-04-26)**
+
+**Goal:** Close the three architectural gaps that survive even after Sprint D ships. Surfaced from a structural critique (2026-04-26): the unified funnel, push transport, and consolidated capacity manager fix the *plumbing*, but leave three contract-level holes — split state authority, no producer idempotency, and no producer-side health gating. These don't dissolve out of #428/#429; they need their own work.
+
+### Sequencing within Sprint D′
+
+```
+#524 (state machine contract) ─► prerequisite to #429 deletion of cleanup phases
+ ─► enables single-writer guarantee by construction
+#525 (idempotency keys) parallel — independent of state machine
+ ─► most acute for webhooks (#291) and scheduler dispatch
+#526 (dispatch circuit breaker) consumes #307 heartbeat (Tier 3) when available
+ ─► can ship with failure-rate detection alone first
+```
+
+| # | Title | Why it's here |
+|---|-------|---------------|
+| ~~#524~~ ✅ | ~~Agent-authoritative execution state machine (RELIABILITY-005)~~ | **Shipped (minimal scope, 2026-04-27).** Full projector architecture deferred — too risky without proper restart-recovery and transport design (agents have no Redis access). Shipped instead: (1) CAS guard in `update_execution_status` — SUCCESS always wins, non-success terminal writes blocked if row already terminal; (2) TOCTOU fix in `mark_stale_executions_failed` / `mark_no_session_executions_failed` — inner UPDATE now carries `AND status = 'running'`; (3) `_recover_execution` routed through already-guarded `mark_execution_failed_by_watchdog`; (4) state machine documented in `TaskExecutionStatus` docstring + `PENDING_RETRY` added to enum. Full projector (`ExecutionStateProjector`, agent event emission, `projected_status` shadow column) remains as future work before #429 can retire cleanup phases. |
+| **#525** | **Idempotency keys at trigger boundaries (RELIABILITY-006)** | `Idempotency-Key` header on every producer (chat, task, internal/scheduler, webhooks #291, MCP, event-sub, self-exec). Backend stores `(key → execution_id)` for 24h and short-circuits duplicates. Webhook trigger auto-derives a key from `(token, body_hash)` for naive senders. Scheduler uses `(schedule_id, fire_time)`. **The unified funnel makes duplicates more uniform — this is the missing dedup layer.** |
+| **#526** | **Per-agent dispatch circuit breaker (RELIABILITY-007)** | Producer-side breaker in front of `SlotService` / `BacklogService`. Opens on rolling failure rate; fast-fails 503 instead of enqueuing into a doomed backlog. Drains existing backlog on trip with `circuit_open` reason. Distinct from #304 (closed — agent-to-agent) and #307 (heartbeat *signal*); this is the **consumer** of that signal. |
+
+### Architectural shift
+
+**Before Sprint D′:** Status column has ~12 writers patched by Phase 3 re-verify. Webhook re-deliveries and scheduler-to-backend network blips silently produce duplicate executions. Agent with expired auth keeps draining slots and queuing 50 doomed tasks until somebody notices.
+
+**After Sprint D′:** Agent emits canonical state events; backend has one projector that is the only `status` writer. Every trigger boundary accepts `Idempotency-Key`; duplicates short-circuit. Unhealthy agent trips a breaker within seconds and starts fast-failing 503; backlog is never poisoned.
+
+### Verification gates before exiting Tier 2.6
+
+- Invariant test asserts no execution has a FAILED→SUCCESS or double-terminal transition in `audit_log` over a 30-day window.
+- Idempotent replay rate observable in logs/metrics; webhook duplicate-storm test (10 retries within 1 s) produces exactly one execution.
+- Forced failure injection (revoke an agent's API key) → circuit opens within 60 s and existing backlog drains with `circuit_open` failures, not 24-hour timeouts.
+
+---
+
## Tier 3 — Remaining push telemetry (Sprint E)
**Goal:** Finish the polling-to-push migration that #306 started.
@@ -224,6 +318,28 @@ Worst case: new paths break and we fall back to existing paths. Old code gets de
---
+## Future considerations (not yet ranked, surfaced 2026-04-26)
+
+These are real architectural gaps surfaced by the same critique that produced Tier 2.6. They're recorded here so they don't rot — promote to issues when the symptoms warrant or when Sprint D′ exits and we have head-room. Roughly ordered by leverage.
+
+1. **Persist agent-side `ProcessRegistry` across restarts.** Today it's pure in-memory. A container crash loses PIDs, log buffer, and `/last-error` source — exactly when an operator most needs them. Write-through to a small SQLite file in the container, or have the agent re-emit "I'm running these IDs" on startup. ~100-line change, outsized recovery value. (Largely subsumed by #524 if the projector mirrors agent state durably; revisit after #524 lands.)
+
+2. **Move retry into `TaskExecutionService`, classified by `error_code`.** Today only the scheduler retries (#271). Webhook, MCP, chat, fan-out, event-sub do not. The funnel already knows the typed `error_code`; only `NETWORK` and rate-limit `CAPACITY` should auto-retry. Producers should not have retry policies at all. *Pairs with #525 (idempotency) — retries inside the funnel must not duplicate.*
+
+3. **Synchronous ack on terminate, with escalation.** Today: SIGINT, 5 s wait, hope. Replace with: backend SIGINT → agent confirms receipt → agent confirms process-group reaped → backend marks terminated. Bounded budget (~10 s) before escalating to container restart, instead of waiting up to 5 min for cleanup to notice.
+
+4. **Two streams, not one.** Split `trinity:events` into `trinity:ui-events` (lossy, MAXLEN-trimmed, live UI fan-out) and `trinity:audit-events` (durable, never trimmed, persisted to `audit_log`). Today slow WS clients can cause audit data to disappear silently because both share `MAXLEN ~10000`. The transports have different reliability requirements; treating them as one quietly costs observability.
+
+5. **Fairness primitive in `CapacityManager` (#428).** When the three queues collapse, don't ship pure FIFO — add per-user / per-subscription token buckets so one tenant can't saturate a shared agent. Hard to retrofit later; cheap to design in while #428 is live work.
+
+6. **Producer-side backpressure on `EventBus`.** Today `XADD` never blocks producers; under sustained pressure, `MAXLEN ~10000` silently trims and slow clients are evicted. No producer flow control. Consider bounded-blocking publish or a separate audit lane (overlaps with #4 above).
+
+7. **A single "execution lifecycle" contract document.** One page: states, allowed transitions, who's authorized to write each one, what events get emitted. The implicit-across-files version is exactly why the system grew 12 status writers nobody could name. (Becomes maintenance work after #524 lands — write the contract document while building it.)
+
+These are recommendations, not commitments. Re-read this list at each sprint planning checkpoint.
+
+---
+
## Architecture snapshots
### Today
@@ -314,7 +430,7 @@ Scheduler failure ─► _maybe_schedule_retry() ✅ #271
└─ APScheduler DateTrigger → _execute_retry()
New triggers, all funnel into the same executor:
- • Webhook ─► schedule dispatch (HMAC-signed URL) [#291 pending]
+ • Webhook ─► schedule dispatch (token-in-URL, rate-limited) ✅ #291
• Self-execute ─► X-Self-Task, optional inject_result ✅ #264
• Retry ─► new execution with retry_of_execution_id ✅ #271
• Validation ─► auditor session, writes business_status [#294 pending]
@@ -333,10 +449,12 @@ New triggers, all funnel into the same executor:
5. ~~Confirm scope cuts for #260: FIFO-only v1, depth 50 default, 24h expiry.~~ ✅ Shipped with these cuts in PR #316.
6. ~~Rescope #132 against `src/scheduler/service.py`~~ — ✅ Shipped in PR #328.
7. ~~Re-estimate #56~~ — ✅ Shipped in PR #329.
-8. ~~Decide #291 direction~~ — **Paused pending #430 (PROCESS-ENGINE-DECISION).**
+8. ~~Decide #291 direction~~ — ✅ **Shipped (PR #484, 2026-04-25).** Token-in-URL trigger funnels through `TaskExecutionService`. Idempotency layer deferred to #525.
9. ~~**Next:** Pick up #294 (validation).~~ — **Closed without implementation.**
10. ~~**Next (2026-04-20):** Pick up **#306**~~ — ✅ **Shipped.** Soak window started on merge date; track push success rate + orphan count.
11. ~~**Follow-up:** Create and rank the three new issues from Tier 2.5~~ — Issues #428, #429, #430 exist; rank + tier assignment tracked in the Roadmap project board (groomed 2026-04-22).
-12. ~~**Next (current):** Pick up **#428 (CAPACITY-CONSOLIDATE)** once #306 has ≥2 weeks of clean soak (zero orphan recoveries, push success ≥99.9%). In parallel, pick up **#430 (PROCESS-ENGINE-DECISION)** — no soak dependency, unblocks #291.~~ ✅ **#430 shipped (2026-04-24)**: Option B (delete) executed. Process engine archived on `archive/process-engine` branch. **Next:** pick up **#291 (WEBHOOK-001)** — now unblocked — and **#428 (CAPACITY-CONSOLIDATE)** once soak completes.
-13. ~~**Re-evaluate #408**~~ — Re-audited 2026-04-26: long-running HTTP call still present at `services/task_execution_service.py:412-419` (`agent_post_with_retry` awaits full agent response with `timeout = timeout_seconds + 10`, up to ~2h under TIMEOUT-001). #306 changed only WebSocket transport, not backend→agent dispatch — it cannot dissolve #408 on its own. Confirms line 32: dissolution is gated on **#428 (CapacityManager + push-completion)**, not #306. #408 stays open until #428 ships.
+12. ~~**Next (current):** Pick up **#428 (CAPACITY-CONSOLIDATE)** once #306 has ≥2 weeks of clean soak (zero orphan recoveries, push success ≥99.9%). In parallel, pick up **#430 (PROCESS-ENGINE-DECISION)** — no soak dependency, unblocks #291.~~ ✅ **#430 shipped (2026-04-24)**, ✅ **#291 shipped (PR #484, 2026-04-25)**, ✅ **#428 shipped (2026-04-26).** **Soak deviation:** user explicitly accepted shipping #428 after only 5 days of #306 soak rather than the planned 2 weeks; mitigated by additive-style refactor (Redis keys + SQL columns unchanged), `dev`-soak before `main`, and clean revert path.
+13. **Re-evaluate #408** — #306 is live, so the predicted dissolution condition now holds. Verify no long-running HTTP call remains in `TaskExecutionService` and close as dissolved (no direct code change needed).
14. **Then:** #429 (CLEANUP-COLLAPSE) gated on #428 landing + continued clean soak — the riskiest step per §"Additive-first migration."
+15. **New (2026-04-26): Tier 2.6 hardening** — pick up **#524 (state machine contract)** as prerequisite to actually deleting cleanup phases in #429; **#525 (idempotency keys)** in parallel (most acute for webhooks #291); **#526 (dispatch circuit breaker)** parallel, sharpens further when #307 heartbeat ships. See *Tier 2.6 — Reliability hardening* and *Future considerations* sections.
+16. **New (2026-04-26): Phase 2 actor-model experiment (parallel to Sprint D′).** First write the pre-experiment postcard (message envelope + journal format). If it fits cleanly, scope and run the smallest test — convert MCP `chat_with_agent` (agent→agent only) to message-passing through `BacklogService`. See *Target architecture: actor model* for the full transition roadmap and decision gate.
diff --git a/docs/planning/PR_REVIEWER_AGENT.md b/docs/planning/PR_REVIEWER_AGENT.md
new file mode 100644
index 000000000..d82e71987
--- /dev/null
+++ b/docs/planning/PR_REVIEWER_AGENT.md
@@ -0,0 +1,272 @@
+# PR Reviewer Agent — Planning Doc
+
+**Status**: Draft — pending review
+**Date**: 2026-04-22
+**Goal**: Autonomous agent that watches a configurable set of GitHub repos, runs `/review` on every new PR, and posts the resulting review as a PR comment. Safety goal: the agent has **near-zero** direct ability to mutate any repo; all side effects go through a narrowly-scoped deterministic Python CLI.
+
+---
+
+## 1. Trust Boundary
+
+| Layer | Can do | Cannot do |
+|-------|--------|-----------|
+| **Deterministic CLI** (`pr-reviewer`) | Read PR list, fetch diff, post issue comment, update local state DB | Close/merge PRs, push code, create branches, approve/request-changes, edit files, run arbitrary `gh` |
+| **Agent (Claude)** | Call `pr-reviewer` subcommands on an allow-list, read fetched diffs, invoke the `/review` skill, write review markdown to a sandbox path | Talk to GitHub directly, run raw `gh`/`git`, use a PAT, see credential values |
+
+Concretely: the **GitHub PAT never lands in the agent's environment**. It lives in the CLI process's env, invoked as an out-of-process subprocess by the agent, and the CLI enforces the allow-list at its entry point. If the agent is ever jailbroken to try `gh pr merge` or `git push`, it has no token and no tool to do so.
+
+---
+
+## 2. Architecture
+
+```
+┌──────────────────────────────────────────────────────────────────────┐
+│ Trinity Agent Container │
+│ │
+│ ┌─────────────────────────┐ │
+│ │ pr-reviewer daemon │ ◄── sleep(interval) loop, no Claude │
+│ │ - scan every N minutes │ │
+│ │ - if empty: sleep again │ │
+│ │ - if work: POST /api/chat to localhost Claude │
+│ └──────────┬──────────────┘ │
+│ │ (only when PRs found) │
+│ ▼ │
+│ ┌────────────────────────┐ subprocess ┌──────────────────────┐ │
+│ │ Claude Code (agent) │ ───────────► │ pr-reviewer CLI │ │
+│ │ - /review skill │ │ (same binary) │ │
+│ │ - no PAT in env │ ◄───stdout─── │ - GITHUB_TOKEN in env │ │
+│ └────────────────────────┘ └──────────┬───────────┘ │
+│ │ │
+│ ┌─────────────────────┼──────────┐ │
+│ │ ▼ │ │
+│ │ SQLite state GitHub API │ │
+│ │ (reviewed_prs) (fine-grained│ │
+│ │ PAT, RW on │ │
+│ │ PRs only) │ │
+│ └────────────────────────────────┘ │
+└──────────────────────────────────────────────────────────────────────┘
+```
+
+### Trigger model — deterministic, zero-token-on-empty
+
+Trinity's cron fires a chat turn, which costs Claude tokens on every poll even when nothing to do. To avoid that, polling lives **outside** Claude: a sibling daemon in the same container runs the scan. Claude is only woken when there is real work.
+
+**Daemon loop** (in-container, no Claude invocations):
+1. `pr-reviewer daemon --interval 900` starts at container boot, alongside `agent-server.py`.
+2. Every 15 min: `pr-reviewer scan` → list of new PRs since last run.
+3. **If empty** → sleep, no token cost, nothing happens.
+4. **If non-empty** → `POST http://localhost:8000/api/chat` with a single batch message: `"Review the following PRs: abilityai/trinity#371, abilityai/abilities#42"`.
+
+**Claude loop** (only fires when daemon hands work off):
+1. Agent receives the batch prompt via local chat.
+2. For each PR: `pr-reviewer fetch #` → CLI writes `diff.md` + `meta.json` under `~/work///`.
+3. Agent reads those files, invokes `/review`, writes `review.md`.
+4. Agent calls `pr-reviewer post # --file review.md` → CLI posts the comment and marks `state.db`.
+5. Agent reports per-PR status and exits the chat turn.
+
+**Why not Trinity scheduler directly?** Scheduler messages are chat messages — every fire is a Claude invocation. For a 15-min poll across quiet repos, that's ~96 empty turns/day × ~500 tokens = ~48k wasted tokens/day. The daemon sidesteps this by keeping polling deterministic and Python-only.
+
+**Fallback option** if we cannot add a daemon to the base image: use the Trinity scheduler with a minimal prompt (`"run pr-reviewer scan; if empty, reply DONE"`) — still costs tokens per poll but kept small. Not recommended unless image changes are blocked.
+
+---
+
+## 3. The Deterministic CLI — `pr-reviewer`
+
+Single Python module, shipped with the agent. Uses `PyGithub` (or raw `httpx` against GitHub REST). Subcommands are the **only** way side effects happen.
+
+| Subcommand | Purpose | Writes? |
+|------------|---------|---------|
+| `scan` | List PRs needing review across configured repos. Returns JSON `[{repo, number, head_sha, title, author, url}, ...]`. | Reads only |
+| `fetch #` | Download diff + metadata into `work///{diff.md, meta.json}`. | Local filesystem only |
+| `post # --file ` | Post the file contents as a single PR issue comment. Prepends a bot-identity header. Refuses if PR already has a bot comment for current `head_sha`. | GitHub issue comment + `state.db` |
+| `status` | Show recently reviewed PRs (from `state.db`). | Reads only |
+| `config validate` | Lint the YAML config. | Reads only |
+| `daemon --interval ` | Long-running loop: `scan` → if work, wake Claude via localhost `/api/chat`, else sleep. Never calls GitHub writes itself — only triggers the Claude loop. Started at container boot. | Local chat HTTP only |
+
+**Hard-coded restrictions inside the CLI** (not configurable, so the agent cannot loosen them):
+- Only `POST /repos/{owner}/{repo}/issues/{number}/comments` is reachable for writes.
+- No `PATCH`, no `MERGE`, no reviews API (`/pulls/{n}/reviews`), no branch/ref writes.
+- Comment body capped (e.g. 60 KB) — hard rejection over that.
+- Repo allow-list from config; any `repo` arg outside the list → error.
+- Rate limit: max **N comments per hour per repo** (configurable, default 6) — CLI sleeps/fails rather than exceeding.
+- Dry-run mode (`--dry-run` or `DRY_RUN=1`) that logs but never calls GitHub — default on for first deploy.
+
+---
+
+## 4. Config Shape
+
+`~/work/config.yaml` in the agent workspace:
+
+```yaml
+repos:
+ - owner: abilityai
+ repo: trinity
+ filters:
+ draft: false
+ labels_any: [] # empty = all
+ labels_none: [skip-bot-review]
+ authors_none: [dependabot[bot]]
+ - owner: abilityai
+ repo: abilities
+
+policy:
+ review_on: new_pr # new_pr | new_pr_and_push
+ max_comments_per_hour: 6
+ comment_header: "🤖 **Trinity PR Reviewer**"
+ dry_run: true # flip to false after a week of dry-run output
+```
+
+---
+
+## 5. State
+
+Local SQLite at `~/.pr-reviewer/state.db`, owned by the CLI:
+
+```sql
+CREATE TABLE reviewed_prs (
+ repo TEXT NOT NULL,
+ pr_number INTEGER NOT NULL,
+ head_sha TEXT NOT NULL,
+ reviewed_at TEXT NOT NULL,
+ comment_url TEXT,
+ comment_body_sha TEXT, -- for idempotency / detect drift
+ PRIMARY KEY (repo, pr_number, head_sha)
+);
+CREATE TABLE rate_limit (
+ repo TEXT NOT NULL,
+ posted_at TEXT NOT NULL
+);
+```
+
+This means **re-review** behavior is deterministic: a PR with a new head SHA reopens for review iff `policy.review_on == new_pr_and_push`, otherwise stays silent.
+
+---
+
+## 6. PAT Scoping
+
+One fine-grained PAT issued by the bot's GitHub account:
+- **Repositories**: explicit allow-list (same as config)
+- **Permissions**: `pull_requests: read & write`, `contents: read`, `metadata: read` — **nothing else** (no workflows, no actions, no admin)
+- Stored via Trinity's CRED-002 `.env` injection as `GITHUB_TOKEN`
+- `.env` file readable only by the CLI process? In practice the agent container can `cat .env`, so this is belt-and-braces: the CLI's allow-list is the real gate, the narrow PAT is defense-in-depth.
+
+---
+
+## 7. Trinity Platform Integration
+
+| Concern | How |
+|---------|-----|
+| **Deployment** | Custom agent template with `pr-reviewer` CLI pre-installed; create via `POST /api/agents` or Trinity UI. |
+| **Credentials** | `GITHUB_TOKEN` (fine-grained PAT) injected via `POST /api/agents/{name}/credentials/inject`. No other secrets needed. |
+| **Scheduling** | **No Trinity cron schedule.** `pr-reviewer daemon --interval 900` runs in-container next to `agent-server.py`, backgrounded from `~/.trinity/setup.sh` (which Trinity's `startup.sh` already invokes on every boot — no base-image change needed). The daemon does the polling; Claude is only invoked when the daemon POSTs work to localhost `/api/chat`. Zero Claude tokens burned on empty polls. |
+| **Read-only mode** | Turn on `PUT /api/agents/{name}/read-only` — blocks accidental edits to source files; `work/` stays writable because read-only only gates source paths. |
+| **Autonomy** | Enabled; no human in loop for routine reviews. |
+| **Audit trail** | Each CLI invocation logs to stdout → Vector → `agents.json`. Every GitHub write the CLI makes is recorded with PR id + comment URL. Optionally emit Trinity events via MCP `emit_event` for a dashboard. |
+| **Kill switch** | Flip `dry_run: true` in config OR disable the schedule via `PUT /api/agents/{name}/autonomy`. Either stops posts immediately. |
+
+---
+
+## 8. Agent System Prompt (sketch)
+
+```
+You are the PR Reviewer agent. You review GitHub pull requests using the
+/review skill and post the result as a comment.
+
+You have exactly ONE tool for GitHub interaction: the `pr-reviewer` CLI.
+You MUST NOT use `gh`, `git push`, `curl`, or any other network tool
+against GitHub. You do not have a GitHub token and these calls will fail.
+
+Loop per invocation:
+ 1. Run `pr-reviewer scan`. Parse JSON.
+ 2. For each entry: `pr-reviewer fetch #`, then read the
+ diff, run /review, write review.md, then
+ `pr-reviewer post # --file review.md`.
+ 3. If the CLI rejects a post (rate limit, duplicate, size), skip and
+ continue. Do not retry aggressively.
+ 4. Report per-PR status at the end.
+
+Never modify files outside `~/work/`. Never open shells against the
+GitHub API directly.
+```
+
+---
+
+## 9. V1 Scope
+
+In scope:
+- Single top-level PR comment with the review markdown.
+- Polling-based discovery via `scan` (no webhooks yet).
+- SQLite state, dry-run default, rate limit, repo allow-list.
+- One bot identity (one PAT), multi-repo.
+
+Out of scope (V2+):
+- **Per-line code comments** via `/pulls/{n}/reviews` — needs another CLI subcommand with its own allow-list. Adds complexity; defer until V1 is stable.
+- **Webhook-driven** (GitHub App or Cloudflare webhook → Trinity) for sub-minute latency. Polling is fine for V1.
+- **Learning from reactions** (👎 on reviews → tune prompt).
+- **PR approve/request-changes** — explicitly never, to preserve the "comment only" trust boundary.
+
+---
+
+## 10. Security Review Checklist (pre-deploy)
+
+- [ ] PAT is fine-grained, scoped to configured repos only, `pull_requests:write` only
+- [ ] CLI allow-list tested: attempting `pr-reviewer post` with a repo not in config fails closed
+- [ ] CLI size-cap tested: 100 KB body rejected
+- [ ] CLI rate-limit tested: 7th comment in an hour blocked
+- [ ] Duplicate-comment guard tested: same head_sha twice → CLI refuses
+- [ ] Dry-run default verified: fresh deploy posts nothing to GitHub
+- [ ] Agent read-only mode enabled
+- [ ] Audit log confirms all write attempts (successful and refused)
+
+---
+
+## 11. Open Questions
+
+1. ~~Daemon launch mechanism~~ — **resolved**. No base-image change needed. `docker/base-image/startup.sh` already sources `/home/developer/.trinity/setup.sh` on every boot (used for restoring user packages). The template for this agent ships a `.trinity/setup.sh` that backgrounds the daemon:
+
+ ```bash
+ nohup python3 /home/developer/bin/pr-reviewer daemon --interval 900 \
+ > /home/developer/logs/daemon.log 2>&1 &
+ ```
+
+ Same `&`-backgrounding pattern Trinity already uses for `agent-server.py`. Daemon restarts with the container. Crash recovery: a `while true; do ...; done` wrapper or systemd-style restart policy in the daemon itself.
+2. **Which review skill exactly** — the existing `/review` in `.claude/skills/review` (pre-landing PR review) applies to Trinity's own branch. For external repos we won't have a `main` to diff against in the agent container. Likely need a variant that works off the PR diff payload directly. Is this a new skill or a flag on the existing one?
+3. **One PAT across repos or per-repo PATs?** One is simpler; per-repo is tighter blast radius if one repo's scope changes.
+4. **Re-review on new pushes?** Default to single-review-per-PR (cheaper, less noise). Override per-repo in config if desired.
+5. **Review latency target?** Polling every 15 min is cheap. Sub-5-min needs webhooks → adds Cloudflare Tunnel + HTTP endpoint to the agent. Defer?
+6. **Handling draft PRs** — default skip, configurable?
+7. **Bot identity** — dedicated GitHub user (recommended, clean audit trail) or run as a human account?
+
+---
+
+## 12. Delivery Plan
+
+| Step | Artifact | Est. effort |
+|------|----------|-------------|
+| 1 | `pr-reviewer` CLI (Python, PyGithub, subcommands + allow-list + state DB + tests) | 1 day |
+| 2 | Agent template (CLAUDE.md + system prompt + schedule + config) | 0.5 day |
+| 3 | Fine-grained PAT issuance, credential injection | 0.5 day |
+| 4 | Dry-run soak on abilityai/trinity + abilityai/abilities (observe, do not post) | 2–3 days |
+| 5 | Flip `dry_run: false`, monitor first 20 reviews for quality + rate-limit behavior | ongoing |
+| 6 | Follow-up issue for V2 (per-line comments, webhooks) | later |
+
+---
+
+## 13. Failure Modes & Responses
+
+| Failure | Detection | Response |
+|---------|-----------|----------|
+| PAT revoked / expired | CLI 401 from GitHub | Agent reports; ops rotates PAT via credential injection endpoint |
+| Agent posts spam / low-quality reviews | Human reviewer 👎 / issue report | Flip `dry_run: true` via config update + agent chat; disable schedule |
+| Rate limit hit | CLI refuses | Already the intended behavior — safe |
+| CLI bug writes wrong repo | Should be impossible (allow-list) — unit-tested | CLI tests gate the release |
+| Agent tries direct `gh`/`git push` | No token → fails; logged to Vector | Audit log review, tune system prompt |
+
+---
+
+## 14. Related
+
+- Trinity credential injection: `docs/memory/architecture.md` §Credentials (CRED-002)
+- Scheduling: `docs/memory/architecture.md` §Background Services
+- Existing review skill: `.claude/skills/review/` (needs variant for external-repo diffs — see Open Q #1)
+- GitHub fine-grained PATs: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#fine-grained-personal-access-tokens
diff --git a/docs/requirements/SUB-003-subscription-auto-switch.md b/docs/requirements/SUB-003-subscription-auto-switch.md
index 60c7654f3..b23d59939 100644
--- a/docs/requirements/SUB-003-subscription-auto-switch.md
+++ b/docs/requirements/SUB-003-subscription-auto-switch.md
@@ -3,7 +3,7 @@
> **Requirement ID**: SUB-003
> **Extends**: SUB-002 (Subscription Management)
> **Priority**: HIGH
-> **Status**: ⏳ Not Started
+> **Status**: ✅ Implemented (2026-03-21), updated 2026-04-25 (#441 — threshold 2 → 1, broadened to auth failures, default flipped to on)
---
@@ -11,13 +11,15 @@
When an agent hits a Claude subscription usage limit ("out of extra usage"), all scheduled and interactive executions fail with HTTP 429 until the subscription resets (often hours/days away). The user must manually notice the error, go to Settings, and reassign a different subscription. This is disruptive for autonomous agents that run on schedules.
+The same disruption happens on auth-class failures (401/403, expired OAuth token, low credit balance) — those signal the subscription itself is broken and need the same auto-recovery.
+
## Requirements
### Preconditions (ALL must be true for auto-switch to trigger)
-1. **Repeated failure**: The agent has encountered a subscription rate-limit error in **2 or more consecutive executions** (not just a single transient hit)
+1. **Subscription failure observed**: The agent has just received either a rate-limit (429) **or** an auth-class failure (401/403/credit balance/expired token) on its current subscription. (Pre-#441 this required 2 consecutive 429s — that gate is removed; the 2h skip-list on alternative selection is the only thrash guard now.)
2. **Multiple subscriptions available**: There are **≥2 subscription credentials** registered in the system
-3. **Setting enabled**: A system-level setting **"Allow automatic subscription switching"** is checked (opt-in, default OFF)
+3. **Setting enabled**: A system-level setting **"Allow automatic subscription switching"** is checked. Default **ON** (opt-out) per #441 — operators who explicitly disable it keep their choice.
### Behavior
@@ -32,8 +34,8 @@ When all three preconditions are met:
3. **Log the switch**: Create a structured log entry and agent activity event:
```
- [SUB-003] Auto-switched agent "{agent_name}" from subscription "{old_sub}" to "{new_sub}"
- after {n} consecutive rate-limit errors
+ [SUB-003] Auto-switching agent "{agent_name}" from "{old_sub}" to "{new_sub}"
+ after {a rate-limit error | an authentication failure}
```
4. **Notify**: Send a notification (via existing notification system) to the agent owner so they're aware of the automatic switch
@@ -49,9 +51,9 @@ When all three preconditions are met:
### Settings UI
-- Add a checkbox to **Settings → Subscriptions** section: **"Automatically switch subscriptions when usage limits are reached"**
-- Below the checkbox, show helper text: _"When enabled, agents will automatically try a different subscription after 2 consecutive rate-limit errors. Requires at least 2 registered subscriptions."_
-- Store as system setting: `auto_switch_subscriptions` (boolean, default `false`)
+- Checkbox in **Settings → Subscriptions** section: **"Automatically switch subscriptions when usage limits or auth failures are reached"**
+- Helper text: _"When enabled, agents automatically try a different subscription on the first rate-limit (429) or auth failure (401/403/expired token/low credit). Requires at least 2 registered subscriptions."_
+- Store as system setting: `auto_switch_subscriptions` (boolean, default `true` per #441 — opt-out, not opt-in)
### Dashboard Visibility
@@ -107,19 +109,24 @@ Backend logs event, sends notification, retries execution
- **All subscriptions exhausted**: Log warning, do not switch, surface error to user as today
- **Agent has ANTHROPIC_API_KEY (not subscription)**: Auto-switch does not apply — only for subscription-based agents
- **Concurrent switches**: Use DB-level locking to prevent two agents from switching to the same subscription simultaneously
-- **Rapid flip-flopping**: If an agent switches and the new subscription also hits a limit, the 2-consecutive-error requirement prevents immediate re-switch — it needs 2 more failures first, giving time for the original to reset
+- **Rapid flip-flopping** (#441): the 2h skip-list on alternative selection (`is_subscription_rate_limited` + `select_best_alternative_subscription`) is the only thrash guard. When an agent switches A→B and B also fails, A stays flagged for 2h post-switch (by the still-recorded events from before the switch — see #444), so no ping-pong back to A.
---
## Acceptance Criteria
-- [ ] System setting `auto_switch_subscriptions` exists and defaults to OFF
-- [ ] Settings UI shows checkbox with helper text in Subscriptions section
-- [ ] Rate-limit errors are tracked per (agent, subscription) with timestamps
-- [ ] After 2+ consecutive rate-limit errors, agent is auto-switched to a different subscription
-- [ ] Rate-limited subscriptions (last 2 hours) are skipped during selection
-- [ ] Auto-switch is logged as an activity event on the agent
-- [ ] Notification is sent to agent owner on auto-switch
-- [ ] Failed execution is retried once after successful switch
-- [ ] No switch occurs if setting is disabled, only 1 subscription exists, or all alternatives are rate-limited
-- [ ] Subscription badge in agent header updates after auto-switch
+- [x] System setting `auto_switch_subscriptions` exists and defaults to ON (#441 — flipped to opt-out)
+- [x] Settings UI shows checkbox with helper text in Subscriptions section
+- [x] Subscription failure events are tracked per (agent, subscription) with timestamps
+- [x] **A single rate-limit (429) failure** triggers auto-switch to a different subscription (#441 — threshold 2 → 1)
+- [x] **A single auth-class failure** (401/403/credit balance/expired token/etc.) also triggers auto-switch (#441 — broadened scope)
+- [x] Rate-limited subscriptions (last 2 hours) are skipped during selection — no regression on the ping-pong guard (#444)
+- [x] Auto-switch is logged as an activity event on the agent
+- [x] Notification is sent to agent owner on auto-switch (text adapts per failure kind)
+- [x] No switch occurs if setting is disabled, only 1 subscription exists, or all alternatives are recently rate-limited
+- [x] Subscription badge in agent header updates after auto-switch
+
+## History
+
+- 2026-03-21 — initial implementation (issue #153, threshold = 2, 429-only, default OFF)
+- 2026-04-25 — #441: threshold dropped to 1, broadened to auth-class failures, default flipped to ON. `handle_rate_limit_error` kept as a backward-compat shim around the new `handle_subscription_failure(failure_kind=…)`.
diff --git a/docs/security-reports/cso-2026-04-29-diff-587.json b/docs/security-reports/cso-2026-04-29-diff-587.json
new file mode 100644
index 000000000..54364b8d1
--- /dev/null
+++ b/docs/security-reports/cso-2026-04-29-diff-587.json
@@ -0,0 +1,47 @@
+{
+ "version": "1.0",
+ "date": "2026-04-29",
+ "mode": "diff",
+ "scope": "all",
+ "branch": "feature/587-public-chat-history",
+ "base": "dev",
+ "phases_run": ["P0","P1","P2","P3","P4","P5","P6","P7","P8","P9","P10","P11","P12","P13","P14"],
+ "attack_surface": {
+ "new_authenticated_endpoints": 2,
+ "new_frontend_components": 1,
+ "new_dependencies": 0,
+ "ci_changes": 1
+ },
+ "findings": [],
+ "informational": [
+ {
+ "id": "I1",
+ "title": "No upper-bound constraint on limit param",
+ "confidence": 6,
+ "file": "src/backend/routers/public.py",
+ "category": "DoS (excluded)",
+ "recommendation": "Add Query(ge=1, le=100) guard for consistency"
+ },
+ {
+ "id": "I2",
+ "title": "No audit log event for session history reads",
+ "confidence": 5,
+ "file": "src/backend/routers/public.py",
+ "category": "Logging",
+ "recommendation": "Pre-existing omission, follow-up in audit hardening track"
+ }
+ ],
+ "filter_stats": {
+ "candidates_before_filter": 2,
+ "filtered_out": 2,
+ "reported": 0
+ },
+ "totals": {
+ "critical": 0,
+ "high": 0,
+ "medium": 0,
+ "low": 0,
+ "informational": 2
+ },
+ "verdict": "CLEAN"
+}
diff --git a/docs/security-reports/cso-2026-04-30-diff-364.json b/docs/security-reports/cso-2026-04-30-diff-364.json
new file mode 100644
index 000000000..71819f20c
--- /dev/null
+++ b/docs/security-reports/cso-2026-04-30-diff-364.json
@@ -0,0 +1,53 @@
+{
+ "version": "1.0",
+ "date": "2026-04-30",
+ "mode": "diff",
+ "scope": "branch:feature/581-voice-tool-calls (issue #364 web chat file upload)",
+ "phases_run": ["P0", "P1", "P2", "P4", "P5", "P7", "P9", "P12", "P13", "P14"],
+ "attack_surface": {
+ "new_endpoints": 0,
+ "extended_endpoints": 2,
+ "extended_surfaces": [
+ "POST /api/agents/{name}/task (files[] field added)",
+ "POST /api/public/chat/{token} (files[] field added)"
+ ],
+ "new_services": ["services/upload_service.py"],
+ "infrastructure_changes": ["nginx client_max_body_size 25m"]
+ },
+ "findings": [
+ {
+ "id": 1,
+ "severity": "MEDIUM",
+ "confidence": "8/10",
+ "status": "VERIFIED",
+ "phase": "P9-A04",
+ "category": "Insecure Design",
+ "title": "MIME validation silently degrades when python-magic unavailable",
+ "file": "src/backend/services/upload_service.py",
+ "lines": "24-43",
+ "description": "When python-magic (libmagic) is not installed, _MAGIC_AVAILABLE=False and magic-byte validation is skipped entirely. Files pass the unsupported-MIME gate based only on the client-declared mimetype field.",
+ "exploit_scenario": "1. Attacker POSTs to /api/agents/{name}/task with valid JWT. 2. files[0] declares mimetype='text/csv' but data_base64 encodes a binary payload. 3. _MAGIC_AVAILABLE=False path skips content verification. 4. Binary file written to /home/developer/uploads/{user_id}/ in agent container. 5. Agent reads file; unexpected binary content may affect agent behavior.",
+ "impact": "Low-Medium. Agent container is already isolated; no direct code execution path. Unexpected binary content may produce unexpected agent behavior.",
+ "recommendation": "1. Add python-magic to requirements.txt as hard dependency. 2. Log CRITICAL and fail fast on startup if _MAGIC_AVAILABLE=False in production. 3. Alternatively, add secondary null-byte scan (first 512 bytes) as fallback gate."
+ }
+ ],
+ "supply_chain_summary": "No new dependencies added to requirements.txt. python-magic used conditionally (already present in environment per prior audit). Frontend: no new npm packages.",
+ "filter_stats": {
+ "total_candidates": 9,
+ "excluded_fp_or_hard_exclusion": 7,
+ "below_confidence_gate": 1,
+ "reported": 1
+ },
+ "totals": {
+ "CRITICAL": 0,
+ "HIGH": 0,
+ "MEDIUM": 1,
+ "LOW": 0,
+ "INFO": 0
+ },
+ "trend": {
+ "prior_report": null,
+ "direction": "baseline",
+ "note": "First diff scan for this branch"
+ }
+}
diff --git a/docs/security-reports/cso-2026-04-30-diff-364.md b/docs/security-reports/cso-2026-04-30-diff-364.md
new file mode 100644
index 000000000..04767c944
--- /dev/null
+++ b/docs/security-reports/cso-2026-04-30-diff-364.md
@@ -0,0 +1,44 @@
+# Security Audit — Diff Report
+**Date**: 2026-04-30 | **Mode**: `--diff` | **Branch**: `feature/581-voice-tool-calls` | **Issue**: #364 (web chat file upload) | **Gate**: 8/10
+
+## Summary
+
+- **1 finding**: MEDIUM — MIME validation silent fallback
+- **0 CRITICAL / 0 HIGH**
+- 9 candidates examined, 8 excluded (FP or below gate)
+
+## Finding #1 — MIME Validation Silent Degradation
+
+| | |
+|---|---|
+| **Severity** | MEDIUM |
+| **Confidence** | 8/10 |
+| **Status** | VERIFIED |
+| **File** | `src/backend/services/upload_service.py:24-43` |
+
+**Description**: When `python-magic` (requires system `libmagic`) is unavailable, `_MAGIC_AVAILABLE=False` and magic-byte validation is skipped. Files pass the unsupported-MIME gate based solely on client-declared `mimetype`. A warning is logged but execution continues.
+
+**Exploit**: Authenticated user uploads binary file with `mimetype: "text/csv"` declared → passes MIME gate → written to agent container workspace as-is.
+
+**Impact**: Low-Medium. Isolated container; no direct exec path. May produce unexpected agent behavior.
+
+**Recommendation**:
+1. Pin `python-magic` in `requirements.txt` (hard dependency, not optional)
+2. Add startup assertion: log CRITICAL + raise if `_MAGIC_AVAILABLE=False` in production
+3. Fallback: null-byte scan of first 512 bytes as secondary gate
+
+## Excluded Candidates (not reported)
+
+| Candidate | Reason |
+|-----------|--------|
+| Temp dir not cleaned up in web routes | Hard exclusion #1 (resource exhaustion, bounded disk impact) |
+| Concurrent uploads share user directory | Below 8/10 confidence gate — low practical impact |
+| Client-side limits bypassable | Defense-in-depth, server enforces independently |
+| TOCTOU declared vs actual size | Mitigated: `len(data)` used, not client `size` field |
+| File descriptions in prompt | Sanitized components only, not injection vectors |
+| Public chat file rate limiting | Existing IP rate limit covers the surface |
+| nginx 25 MB ceiling | Correctly sized for use case |
+
+## Trend
+
+No prior diff report for this branch. Baseline established.
diff --git a/docs/security/UnderDefense-Web-Pentest-Attestation-Apr-2026.pdf b/docs/security/UnderDefense-Web-Pentest-Attestation-Apr-2026.pdf
new file mode 100644
index 000000000..a6372ccde
Binary files /dev/null and b/docs/security/UnderDefense-Web-Pentest-Attestation-Apr-2026.pdf differ
diff --git a/docs/testing/orchestration-invariant-catalog.md b/docs/testing/orchestration-invariant-catalog.md
index 690060754..dc4b794f0 100644
--- a/docs/testing/orchestration-invariant-catalog.md
+++ b/docs/testing/orchestration-invariant-catalog.md
@@ -359,14 +359,16 @@ Running cleanup twice back-to-back produces an empty second report. Failure here
## Recommended starting subset
-Ten invariants cover ~80% of orchestration risk:
+Twelve invariants cover ~80% of orchestration risk:
| ID | Invariant | Why |
|----|-----------|-----|
| S-01 | Slot–row bijection | Core orchestration consistency |
| S-02 | No overbooking | Capacity guarantee |
+| S-03 | Slot TTL ≥ execution timeout | #226 |
| E-01 | Terminal-state closure | No stuck executions |
| E-02 | No phantom reversal | #378/#403 |
+| E-05 | Dispatched rows have session | #106 |
| E-06 | No completed-but-not-reported | #129 |
| B-01 | Queue-status coherence | Backlog integrity |
| B-02 | No queued without slots-full | Drain liveness |
diff --git a/docs/user-docs/getting-started/setup.md b/docs/user-docs/getting-started/setup.md
index 266ccbcff..e732fe0fe 100644
--- a/docs/user-docs/getting-started/setup.md
+++ b/docs/user-docs/getting-started/setup.md
@@ -4,9 +4,8 @@ Install Trinity, create your admin account, and start managing agents in minutes
## Concepts
-- **Setup Wizard** -- A one-time configuration flow that appears on your first visit to create the admin account.
-- **Admin Account** -- The primary account with full platform access, authenticated by username and password.
-- **Email Login** -- A passwordless authentication method where users receive a one-time code via email.
+- **Admin Account** -- The primary account with full platform access, authenticated by username and password. Created automatically from `ADMIN_PASSWORD` in `.env`.
+- **Email Login** -- A passwordless authentication method where users receive a one-time code via email. Requires an email service to be configured.
## How It Works
@@ -25,23 +24,28 @@ Install Trinity, create your admin account, and start managing agents in minutes
cd trinity
```
-2. Start all services:
+2. Set `ADMIN_PASSWORD` in `.env` before first boot:
```bash
- ./scripts/deploy/start.sh
+ cp .env.example .env
+ # Edit .env and set ADMIN_PASSWORD to a strong password (min 12 chars)
```
- This builds the base agent image and starts the backend, frontend, MCP server, Redis, and Vector.
+ The `admin` account is created automatically from this value on first start. If you leave it blank, a one-time setup token is printed to the backend logs — paste it into the setup wizard that appears on first visit.
+
+3. Start all services:
-3. Open http://localhost in your browser.
+ ```bash
+ ./scripts/deploy/start.sh
+ ```
-### First-Time Setup Wizard
+ On first run, this detects if the base agent image is missing and builds it automatically (takes 5-10 minutes). Then starts the backend, frontend, MCP server, Redis, scheduler, and Vector.
-On your first visit, Trinity displays a setup wizard. Set your admin password -- it is stored with bcrypt hashing. This creates the `admin` account that you will use to log in.
+4. Open http://localhost in your browser.
### Logging In
-**Admin login:** Enter username `admin` and the password you set during setup.
+**Admin login:** Enter username `admin` and the `ADMIN_PASSWORD` you set in `.env`.
**Email login (passwordless):** Enter your email address, receive a 6-digit verification code, and submit it to log in. This requires email service configuration. The admin manages allowed email addresses under Settings > Email Whitelist.
@@ -63,10 +67,10 @@ On your first visit, Trinity displays a setup wizard. Set your admin password --
./scripts/deploy/start.sh
# Rebuild services after code changes
-docker-compose build
+docker compose build --no-cache backend frontend mcp-server
# View backend logs
-docker-compose logs -f backend
+docker compose logs -f backend
```
### Settings Page (Admin Only)
diff --git a/docs/user-docs/guides/deploying-trinity.md b/docs/user-docs/guides/deploying-trinity.md
index 610183a3e..505e52352 100644
--- a/docs/user-docs/guides/deploying-trinity.md
+++ b/docs/user-docs/guides/deploying-trinity.md
@@ -71,28 +71,46 @@ Four variables are security-critical and must be set before first boot:
**Port conflicts:** The frontend binds `:80` by default. If another process already holds `:80`, add `FRONTEND_PORT=8090` (or any free port) to `.env`.
-### Step 3: Start services
+### Step 3: Build the base agent image
+
+```bash
+./scripts/deploy/build-base-image.sh
+```
+
+This builds `trinity-agent-base:latest` — the Docker image every agent container inherits. **This step is required before you can create any agents.** It takes 5-10 minutes on first run.
+
+> `start.sh` will detect a missing base image and build it automatically. You can skip this step if you prefer.
+
+### Step 4: Start services
```bash
./scripts/deploy/start.sh
```
-This builds the base agent image and starts all services (backend, frontend, MCP server, Redis, Vector). If `CREDENTIAL_ENCRYPTION_KEY` was blank, the script generates it and writes it back to `.env`.
+Starts all platform services (backend, frontend, MCP server, Redis, scheduler, Vector). If `CREDENTIAL_ENCRYPTION_KEY` was blank, the script generates it and writes it back to `.env`.
Open `http://localhost` (or `http://localhost:$FRONTEND_PORT` if you remapped) and log in with `admin` + the password you set in `.env`.
-### Step 4: Connect from Claude Code
+### Step 5: Connect from Claude Code
+
+Create an MCP API key first:
+1. Log in to the web UI
+2. Go to **Settings → Platform API Keys**
+3. Create a new key and copy it
+
+Then connect:
```bash
/trinity:connect
# When prompted, enter:
# URL: http://localhost:8080/mcp
-# Username: admin
-# Password: (your ADMIN_PASSWORD from .env)
+# API Key: (your MCP API key from Settings → Platform API Keys)
```
-### Step 5: Deploy your first agent
+Alternatively, for email-verified login: when prompted, enter your email and follow the verification code flow.
+
+### Step 6: Deploy your first agent
```bash
/trinity:onboard
@@ -109,19 +127,104 @@ Open `http://localhost` (or `http://localhost:$FRONTEND_PORT` if you remapped) a
## Managing Services (Self-Hosted)
```bash
-# Stop all services
-./scripts/deploy/stop.sh
+# Stop all services (preserves agent containers)
+docker compose stop
# Start all services
./scripts/deploy/start.sh
# View backend logs
-docker-compose logs -f backend
+docker compose logs -f backend
+
+# Rebuild platform services after code changes
+docker compose build --no-cache backend frontend mcp-server
+```
+
+> **Do not use `docker compose down`** to stop a running instance — it destroys agent containers and the agent network. Use `docker compose stop` instead.
+
+## Upgrading
+
+```bash
+# 1. Back up the database first
+docker run --rm -v trinity_trinity-data:/data -v $(pwd):/backup alpine \
+ cp /data/trinity.db /backup/trinity.db.backup-$(date +%Y%m%d)
+
+# 2. Pull latest changes
+git pull origin main
+
+# 3. Rebuild platform services (NOT the base image — separate step)
+docker compose build --no-cache backend frontend mcp-server
+
+# 4. Restart platform services
+docker compose restart backend frontend mcp-server scheduler
+
+# 5. Verify health
+./scripts/deploy/verify-platform.sh
+```
+
+To roll back: restore the DB backup → `git checkout ` → rebuild → restart.
+
+## Health Verification
-# Rebuild after code changes
-docker-compose build
+Run after any change to confirm all six services are healthy:
+
+```bash
+./scripts/deploy/verify-platform.sh
+```
+
+Or check manually:
+
+| Probe | Command |
+|-------|---------|
+| Backend | `curl -sf http://localhost:8000/health` |
+| Scheduler | `curl -sf http://localhost:8001/health` |
+| Frontend | `curl -sf http://localhost` |
+| Redis | `docker exec trinity-redis redis-cli ping` |
+| MCP Server | `curl -sf http://localhost:8080/health` |
+| Vector | `curl -sf http://localhost:8686/health` |
+
+## Resource Thresholds
+
+Monitor these metrics to catch problems before they cascade:
+
+| Metric | Warning | Critical | Action |
+|--------|---------|----------|--------|
+| Agent context usage | >70% | >90% | Restart the agent |
+| Host CPU | >70% | >90% | Scale down active agents |
+| Host memory | >80% | >95% | Restart idle agents |
+| Disk usage | >70% | >85% | Archive or prune logs |
+| Container restarts | >3/hour | >10/hour | Check logs for crash loop |
+| DB size | >500 MB | >1 GB | Run log archival |
+
+## Common Recovery Patterns
+
+**Agent stuck at >90% context** → Restart the agent container:
+```bash
+docker restart
```
+**"network not found" error when starting an agent** → Backend lost track of the agent network:
+```bash
+docker rm
+docker restart trinity-backend
+```
+
+**Database locked** → Multiple writers contending. Check for duplicate backend processes:
+```bash
+docker ps | grep trinity-backend
+```
+There should be exactly one.
+
+## Backup Strategy
+
+Daily backup (run via cron or manually before upgrades):
+```bash
+docker run --rm -v trinity_trinity-data:/data -v /your/backup/path:/backup alpine \
+ sh -c "cp /data/trinity.db /backup/trinity-$(date +%Y%m%d-%H%M%S).db"
+```
+
+Retain 14 daily backups. The DB contains agent state, schedules, chat history, and credentials metadata.
+
## Next Steps
- [Building Agents](building-agents.md) — Create and deploy with Claude Code + abilities
diff --git a/quickstart.sh b/quickstart.sh
new file mode 100755
index 000000000..ef95ff422
--- /dev/null
+++ b/quickstart.sh
@@ -0,0 +1,175 @@
+#!/bin/bash
+# Trinity Quick Start
+# One-command setup for a new Trinity instance.
+#
+# Usage:
+# ./quickstart.sh — interactive guided setup
+# ./quickstart.sh --defaults — non-interactive with auto-generated secrets
+
+set -e
+
+cd "$(dirname "$0")"
+
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+RED='\033[0;31m'
+BOLD='\033[1m'
+NC='\033[0m'
+
+DEFAULTS_MODE=false
+[ "$1" = "--defaults" ] && DEFAULTS_MODE=true
+
+echo ""
+echo -e "${BOLD}=================================================${NC}"
+echo -e "${BOLD} Trinity Agent Platform — Quick Start${NC}"
+echo -e "${BOLD}=================================================${NC}"
+echo ""
+
+# ── 1. Prerequisites ──────────────────────────────────────────────────────────
+
+echo -e "${BOLD}Checking prerequisites...${NC}"
+
+if ! command -v docker &>/dev/null; then
+ echo -e "${RED}✗ Docker not found. Install Docker Desktop: https://docs.docker.com/get-docker/${NC}"
+ exit 1
+fi
+
+if ! docker info &>/dev/null; then
+ echo -e "${RED}✗ Docker is not running. Start Docker Desktop and try again.${NC}"
+ exit 1
+fi
+echo -e "${GREEN}✓ Docker is running${NC}"
+
+if ! docker compose version &>/dev/null; then
+ echo -e "${RED}✗ Docker Compose v2 not found. Update Docker Desktop or install the compose plugin.${NC}"
+ exit 1
+fi
+echo -e "${GREEN}✓ Docker Compose v2 available${NC}"
+echo ""
+
+# ── 2. Environment ────────────────────────────────────────────────────────────
+
+echo -e "${BOLD}Configuring environment...${NC}"
+
+if [ ! -f .env ]; then
+ cp .env.example .env
+ echo " Created .env from template"
+fi
+
+# Auto-generate missing secrets
+for var in SECRET_KEY INTERNAL_API_SECRET; do
+ val=$(grep -E "^${var}=" .env | cut -d'=' -f2-)
+ if [ -z "$val" ]; then
+ new_val=$(openssl rand -hex 32)
+ sed -i.bak "s|^${var}=.*|${var}=${new_val}|" .env && rm -f .env.bak
+ echo " Auto-generated $var"
+ fi
+done
+
+# Auto-generate CREDENTIAL_ENCRYPTION_KEY if blank
+val=$(grep -E '^CREDENTIAL_ENCRYPTION_KEY=' .env | cut -d'=' -f2-)
+if [ -z "$val" ]; then
+ new_val=$(openssl rand -hex 32)
+ sed -i.bak "s|^CREDENTIAL_ENCRYPTION_KEY=.*|CREDENTIAL_ENCRYPTION_KEY=${new_val}|" .env && rm -f .env.bak
+ echo " Auto-generated CREDENTIAL_ENCRYPTION_KEY (do not change after first boot)"
+fi
+
+# ADMIN_PASSWORD
+admin_pass=$(grep -E '^ADMIN_PASSWORD=' .env | cut -d'=' -f2-)
+if [ -z "$admin_pass" ]; then
+ if [ "$DEFAULTS_MODE" = true ]; then
+ admin_pass=$(openssl rand -base64 16 | tr -d '/+=' | head -c 16)
+ sed -i.bak "s|^ADMIN_PASSWORD=.*|ADMIN_PASSWORD=${admin_pass}|" .env && rm -f .env.bak
+ echo " Auto-generated ADMIN_PASSWORD: ${BOLD}${admin_pass}${NC}"
+ echo -e " ${YELLOW}Save this password — it will not be shown again.${NC}"
+ else
+ echo ""
+ echo -e " ${YELLOW}Set an admin password (minimum 12 characters):${NC}"
+ while true; do
+ read -rsp " ADMIN_PASSWORD: " admin_pass
+ echo ""
+ if [ ${#admin_pass} -ge 12 ]; then
+ break
+ fi
+ echo -e " ${RED}Too short — use at least 12 characters.${NC}"
+ done
+ sed -i.bak "s|^ADMIN_PASSWORD=.*|ADMIN_PASSWORD=${admin_pass}|" .env && rm -f .env.bak
+ echo " ✓ Admin password saved to .env"
+ fi
+fi
+
+# ANTHROPIC_API_KEY
+anthropic_key=$(grep -E '^ANTHROPIC_API_KEY=' .env | cut -d'=' -f2-)
+if [ -z "$anthropic_key" ] && [ "$DEFAULTS_MODE" = false ]; then
+ echo ""
+ echo " Anthropic API key (required for agents to use Claude)."
+ echo " Get one at: https://console.anthropic.com/api-keys"
+ echo " Leave blank to configure later in Settings:"
+ read -rsp " ANTHROPIC_API_KEY (or Enter to skip): " anthropic_key
+ echo ""
+ if [ -n "$anthropic_key" ]; then
+ sed -i.bak "s|^ANTHROPIC_API_KEY=.*|ANTHROPIC_API_KEY=${anthropic_key}|" .env && rm -f .env.bak
+ echo " ✓ Anthropic API key saved"
+ else
+ echo " Skipped — configure later in Settings → Anthropic API Key"
+ fi
+fi
+
+echo ""
+echo -e "${GREEN}✓ Environment configured${NC}"
+echo ""
+
+# ── 3. Base image ─────────────────────────────────────────────────────────────
+
+echo -e "${BOLD}Checking base agent image...${NC}"
+if docker images --format "{{.Repository}}:{{.Tag}}" | grep -q "trinity-agent-base:latest"; then
+ echo -e "${GREEN}✓ trinity-agent-base:latest already exists${NC}"
+else
+ echo " Building trinity-agent-base:latest (5-10 minutes on first run)..."
+ ./scripts/deploy/build-base-image.sh
+ echo -e "${GREEN}✓ Base image built${NC}"
+fi
+echo ""
+
+# ── 4. Start services ─────────────────────────────────────────────────────────
+
+echo -e "${BOLD}Starting Trinity services...${NC}"
+docker compose up -d
+echo ""
+echo "Waiting for services to initialize..."
+sleep 8
+echo ""
+
+# ── 5. Verify ─────────────────────────────────────────────────────────────────
+
+echo -e "${BOLD}Verifying platform health...${NC}"
+./scripts/deploy/verify-platform.sh || true
+echo ""
+
+# ── 6. Summary ────────────────────────────────────────────────────────────────
+
+FRONTEND_PORT=$(grep -E '^FRONTEND_PORT=' .env 2>/dev/null | cut -d'=' -f2 || echo "")
+FRONTEND_PORT=${FRONTEND_PORT:-80}
+WEB_URL="http://localhost"
+[ "$FRONTEND_PORT" != "80" ] && WEB_URL="http://localhost:${FRONTEND_PORT}"
+
+echo ""
+echo -e "${BOLD}=================================================${NC}"
+echo -e "${GREEN}${BOLD} Trinity is ready!${NC}"
+echo -e "${BOLD}=================================================${NC}"
+echo ""
+echo " Web UI: $WEB_URL"
+echo " Backend API: http://localhost:8000/docs"
+echo " MCP Server: http://localhost:8080/mcp"
+echo ""
+echo " Login: admin / [your ADMIN_PASSWORD]"
+echo ""
+echo "Next steps:"
+echo " 1. Open $WEB_URL and log in"
+echo " 2. Go to Settings → Platform API Keys and create an MCP key"
+echo " 3. In Claude Code: /trinity:connect"
+echo " 4. Create your first agent: /trinity:onboard"
+echo ""
+echo "To stop: docker compose stop"
+echo "To check: ./scripts/deploy/verify-platform.sh"
+echo ""
diff --git a/scripts/deploy/start.sh b/scripts/deploy/start.sh
index 46b14fad8..159d58f72 100755
--- a/scripts/deploy/start.sh
+++ b/scripts/deploy/start.sh
@@ -27,6 +27,15 @@ if grep -qE '^CREDENTIAL_ENCRYPTION_KEY=$' .env 2>/dev/null || ! grep -q 'CREDEN
echo "Auto-generated CREDENTIAL_ENCRYPTION_KEY"
fi
+# Check base image before starting — without it, agent creation will silently fail
+if ! docker images --format "{{.Repository}}:{{.Tag}}" | grep -q "trinity-agent-base:latest"; then
+ echo "⚠️ trinity-agent-base:latest not found."
+ echo " Building base agent image first (required for agent creation)..."
+ echo ""
+ ./scripts/deploy/build-base-image.sh
+ echo ""
+fi
+
echo "Starting services..."
docker compose up -d
@@ -56,6 +65,8 @@ echo "To view logs:"
echo " docker compose logs -f"
echo ""
echo "To stop services:"
-echo " docker compose down"
+echo " docker compose stop"
+echo ""
+echo "NOTE: Use 'stop' not 'down' — 'down' destroys agent containers."
echo ""
diff --git a/scripts/deploy/validate.sh b/scripts/deploy/validate.sh
index 08413eb05..fae33ad60 100755
--- a/scripts/deploy/validate.sh
+++ b/scripts/deploy/validate.sh
@@ -10,7 +10,7 @@ echo "====================================="
echo ""
echo "1. Checking directory structure..."
-required_dirs=("docker" "src" "config" "scripts" "deployment")
+required_dirs=("docker" "src" "config" "scripts")
for dir in "${required_dirs[@]}"; do
if [ -d "$dir" ]; then
echo " ✅ $dir/"
@@ -33,11 +33,11 @@ echo ""
echo "3. Checking required files..."
required_files=(
"docker-compose.yml"
- "QUICK_START.md"
+ ".env.example"
"src/backend/main.py"
- "src/audit-logger/audit_logger.py"
"docker/base-image/Dockerfile"
"scripts/deploy/start.sh"
+ "scripts/deploy/build-base-image.sh"
)
for file in "${required_files[@]}"; do
@@ -78,8 +78,9 @@ echo ""
echo "Platform is ready for deployment!"
echo ""
echo "Next steps:"
-echo " 1. Copy env.example to .env and configure"
-echo " 2. Run ./scripts/deploy/start.sh"
-echo " 3. Access web UI at http://localhost:3000"
+echo " 1. Copy .env.example to .env and configure"
+echo " 2. Run ./scripts/deploy/build-base-image.sh"
+echo " 3. Run ./scripts/deploy/start.sh"
+echo " 4. Access web UI at http://localhost"
echo ""
diff --git a/scripts/deploy/verify-platform.sh b/scripts/deploy/verify-platform.sh
index f888986db..f14e75424 100755
--- a/scripts/deploy/verify-platform.sh
+++ b/scripts/deploy/verify-platform.sh
@@ -3,20 +3,21 @@
# Trinity Platform - Verification Script
# Checks that all core services are running and healthy
-set -e
+cd "$(dirname "$0")/../.."
+
+GREEN='\033[0;32m'
+RED='\033[0;31m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
echo "====================================="
echo "Trinity Platform - Health Check"
echo "====================================="
echo ""
-# Color codes
-GREEN='\033[0;32m'
-RED='\033[0;31m'
-YELLOW='\033[1;33m'
-NC='\033[0m' # No Color
+all_running=true
-# Check Docker
+# 1. Docker
echo "1. Checking Docker..."
if ! docker ps &> /dev/null; then
echo -e "${RED}✗ Docker is not running${NC}"
@@ -25,15 +26,12 @@ fi
echo -e "${GREEN}✓ Docker is running${NC}"
echo ""
-# Check core services
+# 2. Core services
echo "2. Checking core services..."
-
-services=("trinity-backend" "trinity-redis" "trinity-frontend" "trinity-audit-logger")
-all_running=true
-
+services=("trinity-backend" "trinity-redis" "trinity-frontend" "trinity-mcp-server" "trinity-scheduler" "trinity-vector")
for service in "${services[@]}"; do
- if docker ps --filter "name=$service" --format "{{.Names}}" | grep -q "$service"; then
- status=$(docker ps --filter "name=$service" --format "{{.Status}}")
+ if docker ps --filter "name=^/${service}$" --format "{{.Names}}" | grep -q "^${service}$"; then
+ status=$(docker ps --filter "name=^/${service}$" --format "{{.Status}}")
echo -e "${GREEN}✓ $service${NC} - $status"
else
echo -e "${RED}✗ $service is not running${NC}"
@@ -42,51 +40,76 @@ for service in "${services[@]}"; do
done
echo ""
-# Check health endpoints
+# 3. Health endpoints
echo "3. Checking health endpoints..."
-# Backend health
-if curl -s http://localhost:8000/health | grep -q "healthy"; then
- echo -e "${GREEN}✓ Backend health endpoint${NC}"
+# Backend
+if curl -sf http://localhost:8000/health | grep -q "healthy\|ok\|status"; then
+ echo -e "${GREEN}✓ Backend health (localhost:8000)${NC}"
else
echo -e "${RED}✗ Backend health check failed${NC}"
all_running=false
fi
-# Audit logger health
-if curl -s http://localhost:8001/health | grep -q "healthy"; then
- echo -e "${GREEN}✓ Audit logger health endpoint${NC}"
+# Scheduler (uses port 8001)
+if curl -sf http://localhost:8001/health | grep -q "healthy\|ok\|status"; then
+ echo -e "${GREEN}✓ Scheduler health (localhost:8001)${NC}"
else
- echo -e "${RED}✗ Audit logger health check failed${NC}"
- all_running=false
+ echo -e "${YELLOW}⚠ Scheduler health check failed (may still be starting)${NC}"
fi
-# Frontend
-if curl -s http://localhost:3000 | grep -q "Claude Agent Manager"; then
- echo -e "${GREEN}✓ Frontend is accessible${NC}"
+# Frontend (port 80 or FRONTEND_PORT)
+FRONTEND_PORT=${FRONTEND_PORT:-$(grep -E '^FRONTEND_PORT=' .env 2>/dev/null | cut -d'=' -f2 || echo "80")}
+FRONTEND_PORT=${FRONTEND_PORT:-80}
+FRONTEND_URL="http://localhost"
+[ "$FRONTEND_PORT" != "80" ] && FRONTEND_URL="http://localhost:${FRONTEND_PORT}"
+
+if curl -sf "$FRONTEND_URL" | grep -q -i "trinity\|html\|app"; then
+ echo -e "${GREEN}✓ Frontend accessible ($FRONTEND_URL)${NC}"
else
- echo -e "${RED}✗ Frontend is not accessible${NC}"
+ echo -e "${RED}✗ Frontend not accessible ($FRONTEND_URL)${NC}"
all_running=false
fi
+
+# MCP Server
+if curl -sf http://localhost:8080/health | grep -q "healthy\|ok\|status"; then
+ echo -e "${GREEN}✓ MCP Server health (localhost:8080)${NC}"
+else
+ echo -e "${YELLOW}⚠ MCP Server health check failed${NC}"
+fi
+
+# Vector
+if curl -sf http://localhost:8686/health | grep -q "ok\|healthy"; then
+ echo -e "${GREEN}✓ Vector log aggregator (localhost:8686)${NC}"
+else
+ echo -e "${YELLOW}⚠ Vector health check failed${NC}"
+fi
echo ""
-# Check base image
+# 4. Base agent image
echo "4. Checking base agent image..."
-if docker images | grep -q "trinity-agent-base"; then
+if docker images --format "{{.Repository}}:{{.Tag}}" | grep -q "trinity-agent-base:latest"; then
echo -e "${GREEN}✓ trinity-agent-base:latest exists${NC}"
else
echo -e "${YELLOW}⚠ trinity-agent-base image not found${NC}"
- echo " Run: ./scripts/deploy/build-base-image.sh"
+ echo " Agents cannot be created until you build it:"
+ echo " ./scripts/deploy/build-base-image.sh"
fi
echo ""
-# Check .env file
+# 5. Configuration
echo "5. Checking configuration..."
if [ -f .env ]; then
echo -e "${GREEN}✓ .env file exists${NC}"
+ # Check critical vars are set
+ for var in SECRET_KEY CREDENTIAL_ENCRYPTION_KEY ADMIN_PASSWORD; do
+ val=$(grep -E "^${var}=" .env 2>/dev/null | cut -d'=' -f2-)
+ if [ -z "$val" ]; then
+ echo -e "${YELLOW}⚠ $var is not set in .env${NC}"
+ fi
+ done
else
- echo -e "${YELLOW}⚠ .env file not found${NC}"
- echo " Run: cp .env.example .env"
+ echo -e "${YELLOW}⚠ .env file not found — run: cp .env.example .env${NC}"
fi
echo ""
@@ -96,16 +119,19 @@ if [ "$all_running" = true ]; then
echo -e "${GREEN}✓ Platform is healthy!${NC}"
echo ""
echo "Access points:"
- echo " - Web UI: http://localhost:3000"
+ echo " - Web UI: $FRONTEND_URL"
echo " - Backend API: http://localhost:8000/docs"
- echo " - Audit Logger: http://localhost:8001/docs"
+ echo " - MCP Server: http://localhost:8080/mcp"
+ echo " - Scheduler: http://localhost:8001/health"
+ echo " - Vector logs: http://localhost:8686/health"
echo ""
- echo "Default login: admin / admin"
+ echo "Login: admin / [your ADMIN_PASSWORD from .env]"
else
echo -e "${RED}✗ Some services are not running${NC}"
echo ""
echo "Try:"
- echo " docker-compose up -d"
+ echo " docker compose logs -f"
+ echo " docker compose restart"
exit 1
fi
echo "====================================="
diff --git a/src/backend/adapters/message_router.py b/src/backend/adapters/message_router.py
index 1a0d6d75c..06158be6d 100644
--- a/src/backend/adapters/message_router.py
+++ b/src/backend/adapters/message_router.py
@@ -12,11 +12,9 @@
- Credential sanitization
"""
-import io
import logging
import os
import re
-import tarfile
import time
import unicodedata
from typing import List, Optional, Tuple
@@ -26,21 +24,13 @@
from services.docker_service import get_agent_container
from services.settings_service import settings_service
from services.task_execution_service import get_task_execution_service
-from services.docker_utils import container_put_archive, container_exec_run
-from services.platform_audit_service import platform_audit_service, AuditEventType
+from services.docker_utils import container_exec_run
from services.telegram_media import process_voice
+from services.upload_service import process_file_uploads, format_file_size, sanitize_filename
from adapters.base import ChannelAdapter, ChannelResponse, FileAttachment, NormalizedMessage, OutboundFile
logger = logging.getLogger(__name__)
-# Try to import python-magic for MIME validation; graceful fallback if unavailable
-try:
- import magic
- _MAGIC_AVAILABLE = True
-except ImportError:
- _MAGIC_AVAILABLE = False
- logger.warning("[ROUTER] python-magic not installed; MIME validation will trust channel metadata")
-
# ---------------------------------------------------------------------------
# Configurable defaults (overridable via settings_service)
@@ -173,13 +163,11 @@ def _check_rate_limit(key: str, max_msgs: Optional[int] = None, window: Optional
def _format_file_size(size_bytes: int) -> str:
- """Format bytes as human-readable string."""
- if size_bytes < 1024:
- return f"{size_bytes} B"
- elif size_bytes < 1024 * 1024:
- return f"{size_bytes / 1024:.0f} KB"
- else:
- return f"{size_bytes / (1024 * 1024):.1f} MB"
+ return format_file_size(size_bytes)
+
+
+def _sanitize_filename(name: str, file_id: str, used_names: set) -> str:
+ return sanitize_filename(name, file_id, used_names)
# Filename sanitization (Issue #487)
@@ -465,8 +453,9 @@ async def _handle_message_inner(self, adapter: ChannelAdapter, message: Normaliz
# Issue #487: workspace-write failures abort execution and surface a
# channel-native error so the user knows the upload didn't land.
upload_dir = None # Track for cleanup
+ image_data: list = []
if message.files:
- file_descriptions, upload_dir, all_writes_failed = await self._handle_file_uploads(
+ file_descriptions, upload_dir, all_writes_failed, image_data = await self._handle_file_uploads(
adapter, message, agent_name, container, session_id,
verified_email=verified_email,
)
@@ -504,10 +493,9 @@ async def _handle_message_inner(self, adapter: ChannelAdapter, message: Normaliz
# Configurable via settings_service (default: WebSearch, WebFetch)
public_allowed_tools = _get_channel_allowed_tools()
- # If user uploaded non-image files, agent needs Read to access them.
- # Images are excluded: Claude Code crashes when reading PNGs with
- # --allowedTools (API returns 400 "Could not process image" and the
- # process exits without flushing stdout, hanging the pipe reader).
+ # Non-image files land at upload_dir in the container; agent needs Read
+ # to access them. Images are delivered as vision content blocks via
+ # --input-format stream-json (#562), so no Read is needed for them.
_IMAGE_MIMES = {"image/png", "image/jpeg", "image/gif", "image/webp", "image/svg+xml"}
has_readable_files = message.files and any(
f.mimetype not in _IMAGE_MIMES for f in message.files
@@ -525,6 +513,7 @@ async def _handle_message_inner(self, adapter: ChannelAdapter, message: Normaliz
source_user_email=source_email,
timeout_seconds=None, # Uses agent's configured timeout (TIMEOUT-001)
allowed_tools=public_allowed_tools,
+ images=image_data or None,
)
if result.status == "failed":
@@ -696,226 +685,34 @@ async def _handle_file_uploads(
verified_email: Optional[str] = None,
) -> tuple:
"""
- Download files via adapter and either:
- - Images: embed as base64 data URI in the prompt (Claude vision)
- - Other files: copy into per-session dir in agent container
-
- Returns (descriptions, upload_dir, all_writes_failed):
- - descriptions: list of context strings for prompt injection
- - upload_dir: container path to clean up after execution, or None
- - all_writes_failed: True iff at least one file attempted a workspace
- write but every such attempt failed; the caller should reply with
- an explicit error and skip agent execution (Issue #487 AC6).
+ Download files via adapter then delegate to the shared upload service.
+
+ Returns (descriptions, upload_dir, all_writes_failed, image_data).
"""
- import base64
-
- MAX_FILE_SIZE = 10 * 1024 * 1024 # 10 MB per file
- MAX_IMAGE_SIZE = 5 * 1024 * 1024 # 5 MB per image for inline base64
- MAX_TOTAL_IMAGE_SIZE = 10 * 1024 * 1024 # 10 MB total across all images
- MAX_FILES = 10 # Max files per message
- UNSUPPORTED_MIMES = {"application/pdf", "application/zip", "application/x-tar",
- "application/gzip", "application/x-rar-compressed",
- "video/", "audio/"}
-
- UPLOAD_BASE = "/home/developer/uploads"
- safe_session_id = re.sub(r"[^a-zA-Z0-9_-]", "_", session_id)
- upload_dir = f"{UPLOAD_BASE}/{safe_session_id}"
- descriptions = []
- dir_created = False
- total_image_bytes = 0
- used_names: set = set()
-
- # Uploader attribution for chat injection (Issue #487 AC3).
- # Prefer the verified email; fall back to the channel-native source id
- # so agents always see who sent the file.
uploader = verified_email or adapter.get_source_identifier(message)
- # Track workspace-write outcomes. A "write" is a real attempt to
- # persist bytes (mkdir / put_archive for files; base64 embed for
- # images). Validation rejections (size/MIME/unsupported) do NOT count
- # as write attempts — those are user errors and the agent should
- # respond normally with the description block.
- write_attempted = 0
- write_succeeded = 0
-
- files = message.files
- for f in files[:MAX_FILES]:
- is_image = f.mimetype.startswith("image/")
-
- # Reject unsupported binary formats (PDF, archives, video, audio)
- if any(f.mimetype.startswith(m) if m.endswith("/") else f.mimetype == m
- for m in UNSUPPORTED_MIMES):
- descriptions.append(f"{f.name} — unsupported format ({f.mimetype}). Text, CSV, JSON, and image files are supported.")
- continue
-
- # Sanitize filename — unicode NFKC, basename, strip unsafe chars,
- # truncate to 200 chars preserving extension, dedup collisions
- # with -1/-2 suffixes (Issue #487 AC2).
- safe_name = _sanitize_filename(f.name, f.id, used_names)
- used_names.add(safe_name)
-
- # Size checks
- size_limit = MAX_IMAGE_SIZE if is_image else MAX_FILE_SIZE
- if f.size > size_limit:
- logger.warning(f"[ROUTER] Skipping {safe_name}: too large ({f.size} bytes)")
- descriptions.append(f"{safe_name} — skipped (exceeds {_format_file_size(size_limit)} limit)")
- continue
-
- # Download via adapter (channel-agnostic)
+ # Download each file via the channel adapter before calling the shared service
+ raw_files = []
+ for f in message.files:
data = await adapter.download_file(f, message)
- if not data:
- logger.warning(f"[ROUTER] Failed to download {safe_name} from {adapter.channel_type}")
- descriptions.append(f"{safe_name} — download failed")
- continue
-
- # Post-download size validation (TOCTOU defense)
- actual_size = len(data)
- if actual_size > size_limit:
- logger.warning(
- f"[ROUTER] Rejecting {safe_name}: actual size ({actual_size}) exceeds limit "
- f"(metadata claimed {f.size})"
- )
- descriptions.append(f"{safe_name} — rejected (actual size exceeds {_format_file_size(size_limit)} limit)")
- continue
-
- # Magic-byte MIME validation (if python-magic available)
- actual_mime = f.mimetype
- if _MAGIC_AVAILABLE:
- try:
- detected_mime = magic.from_buffer(data, mime=True)
- # Allow if detected MIME matches declared, or both are image types
- declared_is_image = f.mimetype.startswith("image/")
- detected_is_image = detected_mime.startswith("image/")
-
- if detected_mime != f.mimetype:
- # Accept if both are images (JPEG vs PNG mislabel is common)
- if declared_is_image and detected_is_image:
- logger.debug(
- f"[ROUTER] MIME mismatch for {safe_name}: "
- f"declared={f.mimetype}, detected={detected_mime} (both images, allowing)"
- )
- actual_mime = detected_mime
- is_image = True # Update in case detection is more accurate
- # Accept text subtypes (text/plain vs text/csv)
- elif f.mimetype.startswith("text/") and detected_mime.startswith("text/"):
- logger.debug(f"[ROUTER] Text subtype variation: {f.mimetype} vs {detected_mime}")
- actual_mime = detected_mime
- else:
- logger.warning(
- f"[ROUTER] Rejecting {safe_name}: MIME mismatch "
- f"(declared={f.mimetype}, detected={detected_mime})"
- )
- descriptions.append(f"{safe_name} — rejected (file type mismatch)")
- continue
- except Exception as e:
- logger.warning(f"[ROUTER] MIME detection failed for {safe_name}: {e}")
- # Fall through — use declared MIME
-
- size_str = _format_file_size(actual_size)
-
- if is_image:
- # Check total inline image budget
- if total_image_bytes + len(data) > MAX_TOTAL_IMAGE_SIZE:
- logger.warning(f"[ROUTER] Skipping {safe_name}: total image budget exceeded")
- descriptions.append(f"{safe_name} ({size_str}) — skipped (total image size limit reached)")
- continue
-
- # Image embedding is the "write" for vision-mode files.
- write_attempted += 1
- total_image_bytes += len(data)
- b64 = base64.b64encode(data).decode()
- descriptions.append(
- f"[File uploaded by {uploader}]: {safe_name} ({size_str}) — image attached inline\n"
- f""
- )
- write_succeeded += 1
- logger.info(f"[ROUTER] Embedded {safe_name} ({size_str}) as base64 for {agent_name}")
-
- # Audit log for image upload
- await platform_audit_service.log(
- event_type=AuditEventType.EXECUTION,
- event_action="file_upload",
- source=adapter.channel_type,
- target_type="agent",
- target_id=agent_name,
- details={
- "filename": safe_name,
- "size_bytes": actual_size,
- "mime_type": actual_mime,
- "storage": "inline_base64",
- "sender_id": message.sender_id,
- "channel_id": message.channel_id,
- "uploader": uploader,
- },
- )
- else:
- # Create per-session upload directory on first non-image file.
- # mkdir is the entry point for container writes — count it as
- # one attempt for the file that triggered it.
- if not dir_created:
- write_attempted += 1
- try:
- await container_exec_run(container, f"mkdir -p {upload_dir}", user="developer")
- dir_created = True
- except Exception as e:
- logger.error(f"[ROUTER] Failed to create {upload_dir} in {agent_name}: {e}")
- descriptions.append(f"[File upload failed]: {safe_name} — could not create workspace upload directory")
- continue
- else:
- write_attempted += 1
-
- try:
- tar_buf = io.BytesIO()
- with tarfile.open(fileobj=tar_buf, mode="w") as tar:
- info = tarfile.TarInfo(name=safe_name)
- info.size = len(data)
- info.uid = 1000 # developer user
- info.gid = 1000
- info.mode = 0o644
- tar.addfile(info, io.BytesIO(data))
- tar_buf.seek(0)
-
- success = await container_put_archive(container, upload_dir, tar_buf.read())
- if not success:
- logger.error(f"[ROUTER] Failed to copy {safe_name} into {agent_name}")
- descriptions.append(f"[File upload failed]: {safe_name} — could not save to agent workspace")
- continue
-
- dest_path = f"{upload_dir}/{safe_name}"
- descriptions.append(
- f"[File uploaded by {uploader}]: {safe_name} ({size_str}) saved to {dest_path}"
- )
- write_succeeded += 1
- logger.info(f"[ROUTER] Copied {safe_name} ({size_str}) to {agent_name}:{dest_path}")
-
- # Audit log for file upload
- await platform_audit_service.log(
- event_type=AuditEventType.EXECUTION,
- event_action="file_upload",
- source=adapter.channel_type,
- target_type="agent",
- target_id=agent_name,
- details={
- "filename": safe_name,
- "size_bytes": actual_size,
- "mime_type": actual_mime,
- "storage": "container_file",
- "dest_path": dest_path,
- "sender_id": message.sender_id,
- "channel_id": message.channel_id,
- "uploader": uploader,
- },
- )
-
- except Exception as e:
- logger.error(f"[ROUTER] Error copying {safe_name} to {agent_name}: {e}")
- descriptions.append(f"[File upload failed]: {safe_name} — workspace write error")
-
- if len(files) > MAX_FILES:
- descriptions.append(f"({len(files) - MAX_FILES} more file(s) skipped — max {MAX_FILES} per message)")
-
- all_writes_failed = write_attempted > 0 and write_succeeded == 0
- return descriptions, upload_dir if dir_created else None, all_writes_failed
+ raw_files.append({
+ "name": f.name,
+ "mimetype": f.mimetype,
+ "size": f.size,
+ "data": data, # None signals download failure to the service
+ "id": f.id,
+ })
+
+ return await process_file_uploads(
+ raw_files=raw_files,
+ agent_name=agent_name,
+ container=container,
+ session_id=session_id,
+ uploader=uploader,
+ source=adapter.channel_type,
+ sender_id=message.sender_id,
+ channel_id=message.channel_id,
+ )
# Singleton instance
diff --git a/src/backend/database.py b/src/backend/database.py
index a5daaf2fb..2b0032355 100644
--- a/src/backend/database.py
+++ b/src/backend/database.py
@@ -111,6 +111,7 @@
from db.activities import ActivityOperations
from db.permissions import PermissionOperations
from db.shared_folders import SharedFolderOperations
+from db.agent_shared_files import AgentSharedFilesOperations
from db.settings import SettingsOperations
from db.public_links import PublicLinkOperations
from db.email_auth import EmailAuthOperations
@@ -265,6 +266,7 @@ def __init__(self):
self._activity_ops = ActivityOperations()
self._permission_ops = PermissionOperations(self._user_ops, self._agent_ops)
self._shared_folder_ops = SharedFolderOperations(self._permission_ops)
+ self._agent_shared_files_ops = AgentSharedFilesOperations()
self._settings_ops = SettingsOperations()
self._public_link_ops = PublicLinkOperations(self._user_ops, self._agent_ops)
self._email_auth_ops = EmailAuthOperations(self._user_ops)
@@ -432,6 +434,56 @@ def set_autonomy_enabled(self, agent_name: str, enabled: bool):
def get_all_agents_autonomy_status(self):
return self._agent_ops.get_all_agents_autonomy_status()
+ # =========================================================================
+ # Agent File Sharing (outbound — FILES-001)
+ # =========================================================================
+
+ def get_file_sharing_enabled(self, agent_name: str):
+ return self._agent_ops.get_file_sharing_enabled(agent_name)
+
+ def set_file_sharing_enabled(self, agent_name: str, enabled: bool):
+ return self._agent_ops.set_file_sharing_enabled(agent_name, enabled)
+
+ def get_public_volume_name(self, agent_name: str):
+ return self._agent_ops.get_public_volume_name(agent_name)
+
+ def get_public_mount_path(self):
+ return self._agent_ops.get_public_mount_path()
+
+ # =========================================================================
+ # Agent Shared Files (outbound file URLs — FILES-001 Step 3)
+ # =========================================================================
+
+ def create_agent_shared_file(self, **kwargs):
+ return self._agent_shared_files_ops.create(**kwargs)
+
+ def get_agent_shared_file(self, file_id: str):
+ return self._agent_shared_files_ops.get_by_id(file_id)
+
+ def get_agent_shared_file_by_token(self, download_token: str):
+ return self._agent_shared_files_ops.get_by_token(download_token)
+
+ def total_shared_file_bytes_for_agent(self, agent_name: str) -> int:
+ return self._agent_shared_files_ops.total_bytes_for_agent(agent_name)
+
+ def list_active_shared_files_for_agent(self, agent_name: str) -> list:
+ return self._agent_shared_files_ops.list_active_for_agent(agent_name)
+
+ def mark_shared_file_downloaded(self, file_id: str) -> None:
+ return self._agent_shared_files_ops.mark_downloaded(file_id)
+
+ def revoke_agent_shared_file(self, file_id: str) -> bool:
+ return self._agent_shared_files_ops.revoke(file_id)
+
+ def validate_agent_session(self, agent_name: str, session_token: str):
+ return self._public_link_ops.validate_agent_session(agent_name, session_token)
+
+ def delete_shared_files_for_agent(self, agent_name: str) -> list:
+ return self._agent_shared_files_ops.delete_for_agent(agent_name)
+
+ def delete_expired_and_revoked_shared_files(self, revoke_grace_hours: int = 24) -> list:
+ return self._agent_shared_files_ops.delete_expired_and_revoked(revoke_grace_hours=revoke_grace_hours)
+
# =========================================================================
# Batch Metadata Query (N+1 Fix) - delegated to db/agents.py
# =========================================================================
@@ -1484,6 +1536,9 @@ def get_slack_agent_name_for_channel(self, team_id, slack_channel_id):
def get_slack_dm_default_agent(self, team_id):
return self._slack_channel_ops.get_dm_default_agent(team_id)
+ def set_slack_dm_default(self, team_id, agent_name):
+ return self._slack_channel_ops.set_dm_default(team_id, agent_name)
+
def get_slack_agents_for_workspace(self, team_id):
return self._slack_channel_ops.get_agents_for_workspace(team_id)
@@ -1781,6 +1836,10 @@ def get_audit_stats(self, start_time: str = None, end_time: str = None):
"""Aggregate counts by event_type and actor_type for the dashboard."""
return self._audit_ops.get_audit_stats(start_time=start_time, end_time=end_time)
+ def prune_audit_log(self, retention_days: int) -> int:
+ """Delete audit_log entries older than ``retention_days``. Returns count removed."""
+ return self._audit_ops.prune_audit_log(retention_days)
+
# Global database manager instance
db = DatabaseManager()
diff --git a/src/backend/db/agent_settings/__init__.py b/src/backend/db/agent_settings/__init__.py
index 15a63daad..dc6c8f5d4 100644
--- a/src/backend/db/agent_settings/__init__.py
+++ b/src/backend/db/agent_settings/__init__.py
@@ -9,6 +9,7 @@
- AvatarMixin: Avatar identity management
- MetadataMixin: Batch queries, rename operations
- GitPATMixin: Per-agent GitHub PAT management (#347)
+- FileSharingMixin: Per-agent outbound file-sharing toggle (FILES-001)
"""
from .sharing import SharingMixin
@@ -19,6 +20,7 @@
from .metadata import MetadataMixin
from .access_policy import AccessPolicyMixin
from .git_pat import GitPATMixin
+from .file_sharing import FileSharingMixin
__all__ = [
'SharingMixin',
@@ -29,4 +31,5 @@
'MetadataMixin',
'AccessPolicyMixin',
'GitPATMixin',
+ 'FileSharingMixin',
]
diff --git a/src/backend/db/agent_settings/file_sharing.py b/src/backend/db/agent_settings/file_sharing.py
new file mode 100644
index 000000000..5552a0481
--- /dev/null
+++ b/src/backend/db/agent_settings/file_sharing.py
@@ -0,0 +1,63 @@
+"""
+Agent file-sharing toggle (amazing-file-outbound, FILES-001).
+
+Per-agent opt-in flag that controls whether the agent gets a Docker volume
+mounted at /home/developer/public/. When enabled, the agent can call the
+`share_file` MCP tool to mint a public download URL for files it writes there.
+
+This mixin only manages the toggle + volume-name convention. Actual volume
+creation happens in services/agent_service/crud.py mirroring the
+shared-folders pattern.
+"""
+
+from db.connection import get_db_connection
+
+
+class FileSharingMixin:
+ """Mixin for the per-agent file-sharing opt-in toggle."""
+
+ # =========================================================================
+ # Toggle state
+ # =========================================================================
+
+ def get_file_sharing_enabled(self, agent_name: str) -> bool:
+ """Whether the agent has file sharing enabled. Default: False."""
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ SELECT COALESCE(file_sharing_enabled, 0) AS file_sharing_enabled
+ FROM agent_ownership WHERE agent_name = ?
+ """,
+ (agent_name,),
+ )
+ row = cursor.fetchone()
+ return bool(row["file_sharing_enabled"]) if row else False
+
+ def set_file_sharing_enabled(self, agent_name: str, enabled: bool) -> bool:
+ """Flip the toggle. Returns True if a row was updated."""
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ UPDATE agent_ownership SET file_sharing_enabled = ?
+ WHERE agent_name = ?
+ """,
+ (1 if enabled else 0, agent_name),
+ )
+ conn.commit()
+ return cursor.rowcount > 0
+
+ # =========================================================================
+ # Volume / mount conventions
+ # =========================================================================
+
+ @staticmethod
+ def get_public_volume_name(agent_name: str) -> str:
+ """Docker volume name for the agent's publish dir."""
+ return f"agent-{agent_name}-public"
+
+ @staticmethod
+ def get_public_mount_path() -> str:
+ """Where the volume is mounted inside the agent container."""
+ return "/home/developer/public"
diff --git a/src/backend/db/agent_settings/metadata.py b/src/backend/db/agent_settings/metadata.py
index e939c6944..4ab6d23c8 100644
--- a/src/backend/db/agent_settings/metadata.py
+++ b/src/backend/db/agent_settings/metadata.py
@@ -205,6 +205,14 @@ def rename_agent(self, old_name: str, new_name: str) -> bool:
(new_name, old_name)
)
+ # Shared files (outbound — amazing-file-outbound).
+ # FK has ON UPDATE CASCADE as belt-and-suspenders; this keeps
+ # the explicit cascade list complete for visibility.
+ cursor.execute(
+ "UPDATE agent_shared_files SET agent_name = ? WHERE agent_name = ?",
+ (new_name, old_name)
+ )
+
conn.commit()
return True
diff --git a/src/backend/db/agent_shared_files.py b/src/backend/db/agent_shared_files.py
new file mode 100644
index 000000000..2f18b8189
--- /dev/null
+++ b/src/backend/db/agent_shared_files.py
@@ -0,0 +1,231 @@
+"""
+agent_shared_files database operations (amazing-file-outbound, FILES-001).
+
+Thin CRUD over the table. No business logic here — that belongs in
+services/agent_shared_files_service.py.
+
+MVP scope (Step 3):
+- create — insert a new shared-file row
+- get_by_token — token-based lookup (Step 4 download endpoint will use this)
+- total_bytes_for_agent — quota enforcement helper
+
+Methods needed by later steps (list_by_agent, revoke, mark_downloaded,
+delete_expired) are added later.
+
+One-time link semantics are deferred (see amazing-file-outbound.md §9).
+The `one_time` / `consumed_at` columns exist in the schema but are not
+written or read by this layer — kept so the feature can be re-enabled
+without another migration.
+"""
+
+from typing import Optional, Dict, Any
+
+from .connection import get_db_connection
+
+
+class AgentSharedFilesOperations:
+ """CRUD for agent_shared_files."""
+
+ # -------------------------------------------------------------------------
+ # Write
+ # -------------------------------------------------------------------------
+
+ def create(
+ self,
+ *,
+ file_id: str,
+ agent_name: str,
+ filename: str,
+ stored_filename: str,
+ size_bytes: int,
+ mime_type: Optional[str],
+ download_token: str,
+ created_by: str,
+ created_at: str,
+ expires_at: str,
+ ) -> str:
+ """Insert a new shared-file row. Returns the file_id."""
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ INSERT INTO agent_shared_files (
+ id, agent_name, filename, stored_filename, size_bytes,
+ mime_type, download_token, created_by, created_at, expires_at
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+ """,
+ (
+ file_id,
+ agent_name,
+ filename,
+ stored_filename,
+ size_bytes,
+ mime_type,
+ download_token,
+ created_by,
+ created_at,
+ expires_at,
+ ),
+ )
+ conn.commit()
+ return file_id
+
+ # -------------------------------------------------------------------------
+ # Read
+ # -------------------------------------------------------------------------
+
+ def get_by_id(self, file_id: str) -> Optional[Dict[str, Any]]:
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ "SELECT * FROM agent_shared_files WHERE id = ?", (file_id,)
+ )
+ row = cursor.fetchone()
+ return dict(row) if row else None
+
+ def get_by_token(self, download_token: str) -> Optional[Dict[str, Any]]:
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ "SELECT * FROM agent_shared_files WHERE download_token = ?",
+ (download_token,),
+ )
+ row = cursor.fetchone()
+ return dict(row) if row else None
+
+ def mark_downloaded(self, file_id: str) -> None:
+ """Increment download_count + bump last_downloaded_at."""
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ UPDATE agent_shared_files
+ SET download_count = download_count + 1,
+ last_downloaded_at = datetime('now')
+ WHERE id = ?
+ """,
+ (file_id,),
+ )
+ conn.commit()
+
+ def revoke(self, file_id: str) -> bool:
+ """
+ Mark a share as revoked. Idempotent — a second revoke is a no-op.
+ Returns True if a row transitioned from active to revoked.
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ UPDATE agent_shared_files
+ SET revoked_at = datetime('now')
+ WHERE id = ? AND revoked_at IS NULL
+ """,
+ (file_id,),
+ )
+ conn.commit()
+ return cursor.rowcount > 0
+
+ def delete_expired_and_revoked(self, revoke_grace_hours: int = 24) -> list:
+ """
+ Delete rows where the share is:
+ - expired (expires_at < now) — any state, OR
+ - revoked more than `revoke_grace_hours` ago
+
+ Returns a list of `stored_filename` values whose rows were
+ deleted, so the caller can unlink the on-disk files under
+ /data/agent-files/. Called by the cleanup service (C4 / Step 7).
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ SELECT id, stored_filename
+ FROM agent_shared_files
+ WHERE datetime(expires_at) < datetime('now')
+ OR (revoked_at IS NOT NULL
+ AND datetime(revoked_at) < datetime('now', ?))
+ """,
+ (f"-{revoke_grace_hours} hours",),
+ )
+ rows = cursor.fetchall()
+ if not rows:
+ return []
+ stored = [row["stored_filename"] for row in rows]
+ ids = [row["id"] for row in rows]
+ placeholders = ",".join("?" * len(ids))
+ cursor.execute(
+ f"DELETE FROM agent_shared_files WHERE id IN ({placeholders})",
+ ids,
+ )
+ conn.commit()
+ return stored
+
+ def delete_for_agent(self, agent_name: str) -> list:
+ """
+ Delete every shared-file row for an agent and return the
+ `stored_filename` values so the caller can unlink the files
+ on disk.
+
+ Used by the agent-delete handler. We can't rely on the FK
+ `ON DELETE CASCADE` because the platform's SQLite connections
+ don't `PRAGMA foreign_keys = ON`; deletion of child rows is
+ done explicitly everywhere else in the codebase too
+ (`rename_agent` in `db/agent_settings/metadata.py` follows the
+ same manual-cascade pattern).
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ "SELECT stored_filename FROM agent_shared_files WHERE agent_name = ?",
+ (agent_name,),
+ )
+ stored = [row["stored_filename"] for row in cursor.fetchall()]
+ cursor.execute(
+ "DELETE FROM agent_shared_files WHERE agent_name = ?",
+ (agent_name,),
+ )
+ conn.commit()
+ return stored
+
+ def list_active_for_agent(self, agent_name: str) -> list:
+ """
+ Active (non-revoked, non-expired) shares for an agent,
+ newest first. Used by the Sharing panel.
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ SELECT id, agent_name, filename, stored_filename, size_bytes,
+ mime_type, download_token, created_by, created_at,
+ expires_at, revoked_at, download_count, last_downloaded_at
+ FROM agent_shared_files
+ WHERE agent_name = ?
+ AND revoked_at IS NULL
+ AND datetime(expires_at) > datetime('now')
+ ORDER BY created_at DESC
+ """,
+ (agent_name,),
+ )
+ return [dict(row) for row in cursor.fetchall()]
+
+ def total_bytes_for_agent(self, agent_name: str) -> int:
+ """
+ Sum of size_bytes for rows that still count toward the quota:
+ not revoked and not expired.
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ SELECT COALESCE(SUM(size_bytes), 0) AS total
+ FROM agent_shared_files
+ WHERE agent_name = ?
+ AND revoked_at IS NULL
+ AND datetime(expires_at) > datetime('now')
+ """,
+ (agent_name,),
+ )
+ row = cursor.fetchone()
+ return int(row["total"]) if row else 0
diff --git a/src/backend/db/agents.py b/src/backend/db/agents.py
index 324ed0789..69c026208 100644
--- a/src/backend/db/agents.py
+++ b/src/backend/db/agents.py
@@ -26,6 +26,7 @@
MetadataMixin,
AccessPolicyMixin,
GitPATMixin,
+ FileSharingMixin,
)
from utils.helpers import utc_now_iso
@@ -42,6 +43,7 @@ class AgentOperations(
MetadataMixin,
AccessPolicyMixin,
GitPATMixin,
+ FileSharingMixin,
):
"""Agent ownership, access control, and settings database operations.
diff --git a/src/backend/db/audit.py b/src/backend/db/audit.py
index f1c2f37f0..5de93c926 100644
--- a/src/backend/db/audit.py
+++ b/src/backend/db/audit.py
@@ -252,6 +252,39 @@ def get_audit_stats(
"by_actor_type": by_actor_type,
}
+ # ---------------------------------------------------------------------
+ # Retention
+ # ---------------------------------------------------------------------
+
+ def prune_audit_log(self, retention_days: int) -> int:
+ """Delete entries older than ``retention_days``. Returns rows removed.
+
+ The append-only trigger ``audit_log_no_delete`` blocks DELETEs of
+ rows whose ``timestamp > datetime('now', '-365 days')``. Callers
+ must not pass ``retention_days < 365`` — the trigger would raise
+ on every candidate row and the bulk DELETE would abort.
+
+ Note (architectural invariant #16): we intentionally use SQLite's
+ ``datetime('now', ?)`` here — not ``iso_cutoff()`` — so the prune
+ WHERE filter and the trigger's WHEN clause apply the *same*
+ format-mismatched comparison. Aligning with the trigger avoids
+ IntegrityError on the day-of-cutoff boundary. Fixing the trigger
+ to use ISO-Z form is tracked separately.
+ """
+ if retention_days < 365:
+ raise ValueError(
+ "retention_days must be >= 365 (audit_log_no_delete trigger floor)"
+ )
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ "DELETE FROM audit_log WHERE timestamp < datetime('now', ?)",
+ (f"-{int(retention_days)} days",),
+ )
+ removed = cursor.rowcount
+ conn.commit()
+ return int(removed)
+
# ---------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------
diff --git a/src/backend/db/migrations.py b/src/backend/db/migrations.py
index 9f6690ff1..78ad5b086 100644
--- a/src/backend/db/migrations.py
+++ b/src/backend/db/migrations.py
@@ -47,6 +47,30 @@
logger = logging.getLogger(__name__)
+def _safe_add_column(cursor, table: str, column: str, ddl: str, *, log_msg: str | None = None) -> bool:
+ """Idempotent, concurrent-safe ALTER TABLE ADD COLUMN.
+
+ Two uvicorn workers starting simultaneously can both pass the PRAGMA
+ check before either commits its ALTER (#456 / #389). Catch the
+ duplicate-column race and treat it as success.
+
+ Returns True if this caller added the column, False if it was already
+ present (either by a prior run or another worker winning the race).
+ """
+ cursor.execute(f"PRAGMA table_info({table})")
+ if column in {row[1] for row in cursor.fetchall()}:
+ return False
+ if log_msg:
+ print(log_msg)
+ try:
+ cursor.execute(ddl)
+ return True
+ except sqlite3.OperationalError as e:
+ if "duplicate column name" in str(e).lower():
+ return False
+ raise
+
+
def run_all_migrations(cursor, conn):
"""Run all pending migrations in order. Called twice from init_database().
@@ -143,10 +167,6 @@ def _migrate_agent_sharing_table(cursor, conn):
def _migrate_schedule_executions_observability(cursor, conn):
"""Add observability columns to schedule_executions table."""
- cursor.execute("PRAGMA table_info(schedule_executions)")
- columns = {row[1] for row in cursor.fetchall()}
-
- # Add new columns if they don't exist
new_columns = [
("context_used", "INTEGER"),
("context_max", "INTEGER"),
@@ -156,144 +176,156 @@ def _migrate_schedule_executions_observability(cursor, conn):
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to schedule_executions...")
- cursor.execute(f"ALTER TABLE schedule_executions ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "schedule_executions",
+ col_name,
+ f"ALTER TABLE schedule_executions ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to schedule_executions...",
+ )
conn.commit()
def _migrate_mcp_api_keys_agent_scope(cursor, conn):
"""Add agent_name and scope columns to mcp_api_keys table for agent-to-agent collaboration."""
- cursor.execute("PRAGMA table_info(mcp_api_keys)")
- columns = {row[1] for row in cursor.fetchall()}
-
- # Add new columns if they don't exist
new_columns = [
("agent_name", "TEXT"), # Which agent owns this key (NULL for user keys)
("scope", "TEXT DEFAULT 'user'") # "user" or "agent"
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to mcp_api_keys for agent collaboration...")
- cursor.execute(f"ALTER TABLE mcp_api_keys ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "mcp_api_keys",
+ col_name,
+ f"ALTER TABLE mcp_api_keys ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to mcp_api_keys for agent collaboration...",
+ )
conn.commit()
def _migrate_agent_ownership_system_flag(cursor, conn):
"""Add is_system column to agent_ownership table for system agent protection."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "is_system" not in columns:
- print("Adding is_system column to agent_ownership for system agent protection...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN is_system INTEGER DEFAULT 0")
- conn.commit()
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "is_system",
+ "ALTER TABLE agent_ownership ADD COLUMN is_system INTEGER DEFAULT 0",
+ log_msg="Adding is_system column to agent_ownership for system agent protection...",
+ )
+ conn.commit()
def _migrate_agent_ownership_platform_key(cursor, conn):
"""Add use_platform_api_key column to agent_ownership table for per-agent API key control."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "use_platform_api_key" not in columns:
- print("Adding use_platform_api_key column to agent_ownership for per-agent API key control...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN use_platform_api_key INTEGER DEFAULT 1")
- conn.commit()
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "use_platform_api_key",
+ "ALTER TABLE agent_ownership ADD COLUMN use_platform_api_key INTEGER DEFAULT 1",
+ log_msg="Adding use_platform_api_key column to agent_ownership for per-agent API key control...",
+ )
+ conn.commit()
def _migrate_agent_git_config_source_branch(cursor, conn):
"""Add source_branch and source_mode columns to agent_git_config for GitHub source tracking."""
- cursor.execute("PRAGMA table_info(agent_git_config)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("source_branch", "TEXT DEFAULT 'main'"), # Branch to pull from
("source_mode", "INTEGER DEFAULT 0") # 1 = track source branch directly, 0 = legacy working branch
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to agent_git_config for GitHub source tracking...")
- cursor.execute(f"ALTER TABLE agent_git_config ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "agent_git_config",
+ col_name,
+ f"ALTER TABLE agent_git_config ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to agent_git_config for GitHub source tracking...",
+ )
conn.commit()
def _migrate_agent_ownership_autonomy(cursor, conn):
"""Add autonomy_enabled column to agent_ownership table for autonomous scheduling control."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "autonomy_enabled" not in columns:
- print("Adding autonomy_enabled column to agent_ownership for autonomous scheduling...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN autonomy_enabled INTEGER DEFAULT 0")
- conn.commit()
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "autonomy_enabled",
+ "ALTER TABLE agent_ownership ADD COLUMN autonomy_enabled INTEGER DEFAULT 0",
+ log_msg="Adding autonomy_enabled column to agent_ownership for autonomous scheduling...",
+ )
+ conn.commit()
def _migrate_agent_ownership_resource_limits(cursor, conn):
"""Add memory_limit and cpu_limit columns to agent_ownership table for per-agent resource configuration."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("memory_limit", "TEXT"), # e.g., "4g", "8g", "16g"
("cpu_limit", "TEXT") # e.g., "2", "4", "8"
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to agent_ownership for resource configuration...")
- cursor.execute(f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ col_name,
+ f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to agent_ownership for resource configuration...",
+ )
conn.commit()
def _migrate_agent_ownership_full_capabilities(cursor, conn):
"""Add full_capabilities column to agent_ownership table for container security settings."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "full_capabilities" not in columns:
- print("Adding full_capabilities column to agent_ownership for container security settings...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN full_capabilities INTEGER DEFAULT 0")
- conn.commit()
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "full_capabilities",
+ "ALTER TABLE agent_ownership ADD COLUMN full_capabilities INTEGER DEFAULT 0",
+ log_msg="Adding full_capabilities column to agent_ownership for container security settings...",
+ )
+ conn.commit()
def _migrate_agent_ownership_read_only_mode(cursor, conn):
"""Add read_only_mode and read_only_config columns to agent_ownership table for code protection."""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("read_only_mode", "INTEGER DEFAULT 0"), # 0 = disabled, 1 = enabled
("read_only_config", "TEXT") # JSON config for blocked/allowed patterns
]
for col_name, col_def in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to agent_ownership for read-only mode...")
- cursor.execute(f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_def}")
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ col_name,
+ f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_def}",
+ log_msg=f"Adding {col_name} column to agent_ownership for read-only mode...",
+ )
conn.commit()
def _migrate_agent_schedules_execution_config(cursor, conn):
"""Add timeout_seconds and allowed_tools columns to agent_schedules table for per-schedule execution config."""
- cursor.execute("PRAGMA table_info(agent_schedules)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("timeout_seconds", "INTEGER DEFAULT 900"), # Default 15 minutes
("allowed_tools", "TEXT") # JSON array of allowed tools, NULL = all tools
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to agent_schedules for execution configuration...")
- cursor.execute(f"ALTER TABLE agent_schedules ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "agent_schedules",
+ col_name,
+ f"ALTER TABLE agent_schedules ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to agent_schedules for execution configuration...",
+ )
conn.commit()
@@ -332,9 +364,6 @@ def _migrate_execution_origin_tracking(cursor, conn):
- source_mcp_key_id: MCP API key ID used (for MCP calls)
- source_mcp_key_name: MCP API key name (denormalized)
"""
- cursor.execute("PRAGMA table_info(schedule_executions)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("source_user_id", "INTEGER"),
("source_user_email", "TEXT"),
@@ -344,9 +373,13 @@ def _migrate_execution_origin_tracking(cursor, conn):
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to schedule_executions for origin tracking...")
- cursor.execute(f"ALTER TABLE schedule_executions ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "schedule_executions",
+ col_name,
+ f"ALTER TABLE schedule_executions ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to schedule_executions for origin tracking...",
+ )
conn.commit()
@@ -357,13 +390,13 @@ def _migrate_execution_session_tracking(cursor, conn):
Enables "Continue Execution as Chat" feature by storing Claude Code's
session_id, which can be used with --resume to continue the session.
"""
- cursor.execute("PRAGMA table_info(schedule_executions)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "claude_session_id" not in columns:
- print("Adding claude_session_id column to schedule_executions for session resume...")
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN claude_session_id TEXT")
-
+ _safe_add_column(
+ cursor,
+ "schedule_executions",
+ "claude_session_id",
+ "ALTER TABLE schedule_executions ADD COLUMN claude_session_id TEXT",
+ log_msg="Adding claude_session_id column to schedule_executions for session resume...",
+ )
conn.commit()
@@ -396,13 +429,13 @@ def _migrate_agent_ownership_subscription_id(cursor, conn):
Links agents to their assigned subscription for Claude Max/Pro authentication.
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "subscription_id" not in columns:
- print("Adding subscription_id column to agent_ownership for subscription management...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN subscription_id TEXT REFERENCES subscription_credentials(id)")
-
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "subscription_id",
+ "ALTER TABLE agent_ownership ADD COLUMN subscription_id TEXT REFERENCES subscription_credentials(id)",
+ log_msg="Adding subscription_id column to agent_ownership for subscription management...",
+ )
conn.commit()
@@ -590,13 +623,13 @@ def _migrate_agent_ownership_parallel_capacity(cursor, conn):
Configures per-agent parallel execution capacity for the /task endpoint.
Range: 1-10, Default: 3.
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "max_parallel_tasks" not in columns:
- print("Adding max_parallel_tasks column to agent_ownership for parallel capacity...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN max_parallel_tasks INTEGER DEFAULT 3")
-
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "max_parallel_tasks",
+ "ALTER TABLE agent_ownership ADD COLUMN max_parallel_tasks INTEGER DEFAULT 3",
+ log_msg="Adding max_parallel_tasks column to agent_ownership for parallel capacity...",
+ )
conn.commit()
@@ -606,22 +639,20 @@ def _migrate_schedule_model_selection(cursor, conn):
- agent_schedules.model: Model to use when schedule fires (NULL = agent default)
- schedule_executions.model_used: Records which model was actually used
"""
- # Add model to agent_schedules
- cursor.execute("PRAGMA table_info(agent_schedules)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "model" not in columns:
- print("Adding model column to agent_schedules for model selection...")
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN model TEXT")
-
- # Add model_used to schedule_executions
- cursor.execute("PRAGMA table_info(schedule_executions)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "model_used" not in columns:
- print("Adding model_used column to schedule_executions for model audit...")
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN model_used TEXT")
-
+ _safe_add_column(
+ cursor,
+ "agent_schedules",
+ "model",
+ "ALTER TABLE agent_schedules ADD COLUMN model TEXT",
+ log_msg="Adding model column to agent_schedules for model selection...",
+ )
+ _safe_add_column(
+ cursor,
+ "schedule_executions",
+ "model_used",
+ "ALTER TABLE schedule_executions ADD COLUMN model_used TEXT",
+ log_msg="Adding model_used column to schedule_executions for model audit...",
+ )
conn.commit()
@@ -672,18 +703,19 @@ def _migrate_agent_avatar_columns(cursor, conn):
- avatar_identity_prompt: User's character description for avatar generation
- avatar_updated_at: ISO timestamp for cache-busting
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
new_columns = [
("avatar_identity_prompt", "TEXT"),
("avatar_updated_at", "TEXT"),
]
for col_name, col_type in new_columns:
- if col_name not in columns:
- print(f"Adding {col_name} column to agent_ownership for avatar support...")
- cursor.execute(f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_type}")
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ col_name,
+ f"ALTER TABLE agent_ownership ADD COLUMN {col_name} {col_type}",
+ log_msg=f"Adding {col_name} column to agent_ownership for avatar support...",
+ )
conn.commit()
@@ -731,13 +763,13 @@ def _migrate_agent_ownership_default_avatar(cursor, conn):
Tracks whether an agent's avatar was auto-generated (default) vs custom.
Allows re-runs to skip custom avatars and overwrite stale defaults.
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "is_default_avatar" not in columns:
- print("Adding is_default_avatar column to agent_ownership for default avatar support...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN is_default_avatar INTEGER DEFAULT 0")
-
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "is_default_avatar",
+ "ALTER TABLE agent_ownership ADD COLUMN is_default_avatar INTEGER DEFAULT 0",
+ log_msg="Adding is_default_avatar column to agent_ownership for default avatar support...",
+ )
conn.commit()
@@ -748,13 +780,13 @@ def _migrate_agent_ownership_execution_timeout(cursor, conn):
scheduler, MCP, paid endpoints) read from this setting when no explicit timeout
is provided. Default: 900 seconds (15 minutes).
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "execution_timeout_seconds" not in columns:
- print("Adding execution_timeout_seconds column to agent_ownership for per-agent timeout...")
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN execution_timeout_seconds INTEGER DEFAULT 900")
-
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "execution_timeout_seconds",
+ "ALTER TABLE agent_ownership ADD COLUMN execution_timeout_seconds INTEGER DEFAULT 900",
+ log_msg="Adding execution_timeout_seconds column to agent_ownership for per-agent timeout...",
+ )
conn.commit()
@@ -816,11 +848,13 @@ def _migrate_chat_messages_source_column(cursor, conn):
Distinguishes voice messages from text messages.
Values: 'text' (default), 'voice'.
"""
- try:
- cursor.execute("ALTER TABLE chat_messages ADD COLUMN source TEXT DEFAULT 'text'")
- conn.commit()
- except Exception:
- pass # Column already exists
+ _safe_add_column(
+ cursor,
+ "chat_messages",
+ "source",
+ "ALTER TABLE chat_messages ADD COLUMN source TEXT DEFAULT 'text'",
+ )
+ conn.commit()
def _migrate_agent_ownership_guardrails(cursor, conn):
@@ -830,11 +864,13 @@ def _migrate_agent_ownership_guardrails(cursor, conn):
extra_bash_deny, extra_path_deny) as a JSON blob. NULL means "use
platform baseline only".
"""
- cursor.execute("PRAGMA table_info(agent_ownership)")
- columns = {row[1] for row in cursor.fetchall()}
- if "guardrails_config" not in columns:
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN guardrails_config TEXT")
- conn.commit()
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "guardrails_config",
+ "ALTER TABLE agent_ownership ADD COLUMN guardrails_config TEXT",
+ )
+ conn.commit()
def _migrate_agent_ownership_voice_prompt(cursor, conn):
@@ -842,11 +878,13 @@ def _migrate_agent_ownership_voice_prompt(cursor, conn):
Stores the per-agent voice personality prompt for Gemini Live API.
"""
- try:
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN voice_system_prompt TEXT")
- conn.commit()
- except Exception:
- pass # Column already exists
+ _safe_add_column(
+ cursor,
+ "agent_ownership",
+ "voice_system_prompt",
+ "ALTER TABLE agent_ownership ADD COLUMN voice_system_prompt TEXT",
+ )
+ conn.commit()
def _migrate_slack_channel_agents(cursor, conn):
"""Add multi-agent Slack support: workspace table + channel-agent bindings.
@@ -922,25 +960,22 @@ def _migrate_subscription_usage_tracking(cursor, conn):
- chat_sessions: subscription_id (active when session started)
- schedule_executions: subscription_id (snapshotted at record time)
"""
- cursor.execute("PRAGMA table_info(chat_messages)")
- chat_msg_cols = {row[1] for row in cursor.fetchall()}
-
- if "subscription_id" not in chat_msg_cols:
- cursor.execute("ALTER TABLE chat_messages ADD COLUMN subscription_id TEXT")
- if "output_tokens" not in chat_msg_cols:
- cursor.execute("ALTER TABLE chat_messages ADD COLUMN output_tokens INTEGER")
-
- cursor.execute("PRAGMA table_info(chat_sessions)")
- chat_sess_cols = {row[1] for row in cursor.fetchall()}
-
- if "subscription_id" not in chat_sess_cols:
- cursor.execute("ALTER TABLE chat_sessions ADD COLUMN subscription_id TEXT")
-
- cursor.execute("PRAGMA table_info(schedule_executions)")
- exec_cols = {row[1] for row in cursor.fetchall()}
-
- if "subscription_id" not in exec_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN subscription_id TEXT")
+ _safe_add_column(
+ cursor, "chat_messages", "subscription_id",
+ "ALTER TABLE chat_messages ADD COLUMN subscription_id TEXT",
+ )
+ _safe_add_column(
+ cursor, "chat_messages", "output_tokens",
+ "ALTER TABLE chat_messages ADD COLUMN output_tokens INTEGER",
+ )
+ _safe_add_column(
+ cursor, "chat_sessions", "subscription_id",
+ "ALTER TABLE chat_sessions ADD COLUMN subscription_id TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "subscription_id",
+ "ALTER TABLE schedule_executions ADD COLUMN subscription_id TEXT",
+ )
# Indexes for usage queries
cursor.execute(
@@ -961,14 +996,14 @@ def _migrate_execution_fan_out_id(cursor, conn):
Links individual execution records back to a parent fan-out operation
for observability and result aggregation.
"""
- cursor.execute("PRAGMA table_info(schedule_executions)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "fan_out_id" not in columns:
- print("Adding fan_out_id column to schedule_executions for fan-out linkage...")
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN fan_out_id TEXT")
- cursor.execute("CREATE INDEX IF NOT EXISTS idx_executions_fan_out ON schedule_executions(fan_out_id)")
-
+ _safe_add_column(
+ cursor,
+ "schedule_executions",
+ "fan_out_id",
+ "ALTER TABLE schedule_executions ADD COLUMN fan_out_id TEXT",
+ log_msg="Adding fan_out_id column to schedule_executions for fan-out linkage...",
+ )
+ cursor.execute("CREATE INDEX IF NOT EXISTS idx_executions_fan_out ON schedule_executions(fan_out_id)")
conn.commit()
@@ -1030,20 +1065,24 @@ def _migrate_access_control(cursor, conn):
- access_requests: new table for pending per-agent access requests
"""
# 1. agent_ownership columns
- cursor.execute("PRAGMA table_info(agent_ownership)")
- ao_cols = {row[1] for row in cursor.fetchall()}
- if "require_email" not in ao_cols:
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN require_email INTEGER DEFAULT 0")
- if "open_access" not in ao_cols:
- cursor.execute("ALTER TABLE agent_ownership ADD COLUMN open_access INTEGER DEFAULT 0")
+ _safe_add_column(
+ cursor, "agent_ownership", "require_email",
+ "ALTER TABLE agent_ownership ADD COLUMN require_email INTEGER DEFAULT 0",
+ )
+ _safe_add_column(
+ cursor, "agent_ownership", "open_access",
+ "ALTER TABLE agent_ownership ADD COLUMN open_access INTEGER DEFAULT 0",
+ )
# 2. telegram_chat_links columns
- cursor.execute("PRAGMA table_info(telegram_chat_links)")
- tcl_cols = {row[1] for row in cursor.fetchall()}
- if "verified_email" not in tcl_cols:
- cursor.execute("ALTER TABLE telegram_chat_links ADD COLUMN verified_email TEXT")
- if "verified_at" not in tcl_cols:
- cursor.execute("ALTER TABLE telegram_chat_links ADD COLUMN verified_at TEXT")
+ _safe_add_column(
+ cursor, "telegram_chat_links", "verified_email",
+ "ALTER TABLE telegram_chat_links ADD COLUMN verified_email TEXT",
+ )
+ _safe_add_column(
+ cursor, "telegram_chat_links", "verified_at",
+ "ALTER TABLE telegram_chat_links ADD COLUMN verified_at TEXT",
+ )
# 3. access_requests table
cursor.execute("""
@@ -1104,14 +1143,12 @@ def _migrate_email_whitelist_default_role(cursor, conn):
buggy default going forward. It does NOT touch `users.role`, so users who
have already logged in and been promoted keep their current role.
"""
- cursor.execute("PRAGMA table_info(email_whitelist)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "default_role" not in columns:
- cursor.execute(
- "ALTER TABLE email_whitelist ADD COLUMN default_role TEXT NOT NULL DEFAULT 'user'"
- )
-
+ _safe_add_column(
+ cursor,
+ "email_whitelist",
+ "default_role",
+ "ALTER TABLE email_whitelist ADD COLUMN default_role TEXT NOT NULL DEFAULT 'user'",
+ )
conn.commit()
@@ -1158,20 +1195,20 @@ def _migrate_backlog_support(cursor, conn):
- Partial index on (agent_name, queued_at) WHERE status='queued' for cheap FIFO claim.
"""
# schedule_executions columns
- cursor.execute("PRAGMA table_info(schedule_executions)")
- se_cols = {row[1] for row in cursor.fetchall()}
- if "queued_at" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN queued_at TEXT")
- if "backlog_metadata" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN backlog_metadata TEXT")
+ _safe_add_column(
+ cursor, "schedule_executions", "queued_at",
+ "ALTER TABLE schedule_executions ADD COLUMN queued_at TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "backlog_metadata",
+ "ALTER TABLE schedule_executions ADD COLUMN backlog_metadata TEXT",
+ )
# agent_ownership column
- cursor.execute("PRAGMA table_info(agent_ownership)")
- ao_cols = {row[1] for row in cursor.fetchall()}
- if "max_backlog_depth" not in ao_cols:
- cursor.execute(
- "ALTER TABLE agent_ownership ADD COLUMN max_backlog_depth INTEGER DEFAULT 50"
- )
+ _safe_add_column(
+ cursor, "agent_ownership", "max_backlog_depth",
+ "ALTER TABLE agent_ownership ADD COLUMN max_backlog_depth INTEGER DEFAULT 50",
+ )
# Partial index for efficient FIFO claim of queued rows
cursor.execute(
@@ -1197,25 +1234,29 @@ def _migrate_scheduler_retry_support(cursor, conn):
retries amplified load during multi-hour subscription outages.
"""
# agent_schedules columns
- cursor.execute("PRAGMA table_info(agent_schedules)")
- as_cols = {row[1] for row in cursor.fetchall()}
-
- if "max_retries" not in as_cols:
- # Default 0 for existing schedules (opt-in); schema default is also 0 since #476
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN max_retries INTEGER DEFAULT 0")
- if "retry_delay_seconds" not in as_cols:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN retry_delay_seconds INTEGER DEFAULT 60")
+ # Default 0 for existing schedules (opt-in); schema default is also 0 since #476.
+ _safe_add_column(
+ cursor, "agent_schedules", "max_retries",
+ "ALTER TABLE agent_schedules ADD COLUMN max_retries INTEGER DEFAULT 0",
+ )
+ _safe_add_column(
+ cursor, "agent_schedules", "retry_delay_seconds",
+ "ALTER TABLE agent_schedules ADD COLUMN retry_delay_seconds INTEGER DEFAULT 60",
+ )
# schedule_executions columns
- cursor.execute("PRAGMA table_info(schedule_executions)")
- se_cols = {row[1] for row in cursor.fetchall()}
-
- if "attempt_number" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN attempt_number INTEGER DEFAULT 1")
- if "retry_of_execution_id" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN retry_of_execution_id TEXT")
- if "retry_scheduled_at" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN retry_scheduled_at TEXT")
+ _safe_add_column(
+ cursor, "schedule_executions", "attempt_number",
+ "ALTER TABLE schedule_executions ADD COLUMN attempt_number INTEGER DEFAULT 1",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "retry_of_execution_id",
+ "ALTER TABLE schedule_executions ADD COLUMN retry_of_execution_id TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "retry_scheduled_at",
+ "ALTER TABLE schedule_executions ADD COLUMN retry_scheduled_at TEXT",
+ )
# Index for finding pending retries on startup
cursor.execute(
@@ -1239,28 +1280,36 @@ def _migrate_validation_support(cursor, conn):
- schedule_executions.validates_execution_id: FK to execution being validated (for validation records)
"""
# agent_schedules columns
- cursor.execute("PRAGMA table_info(agent_schedules)")
- as_cols = {row[1] for row in cursor.fetchall()}
-
- if "validation_enabled" not in as_cols:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN validation_enabled INTEGER DEFAULT 0")
- if "validation_prompt" not in as_cols:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN validation_prompt TEXT")
- if "validation_timeout_seconds" not in as_cols:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN validation_timeout_seconds INTEGER DEFAULT 120")
+ _safe_add_column(
+ cursor, "agent_schedules", "validation_enabled",
+ "ALTER TABLE agent_schedules ADD COLUMN validation_enabled INTEGER DEFAULT 0",
+ )
+ _safe_add_column(
+ cursor, "agent_schedules", "validation_prompt",
+ "ALTER TABLE agent_schedules ADD COLUMN validation_prompt TEXT",
+ )
+ _safe_add_column(
+ cursor, "agent_schedules", "validation_timeout_seconds",
+ "ALTER TABLE agent_schedules ADD COLUMN validation_timeout_seconds INTEGER DEFAULT 120",
+ )
# schedule_executions columns
- cursor.execute("PRAGMA table_info(schedule_executions)")
- se_cols = {row[1] for row in cursor.fetchall()}
-
- if "business_status" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN business_status TEXT")
- if "validated_at" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN validated_at TEXT")
- if "validation_execution_id" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN validation_execution_id TEXT")
- if "validates_execution_id" not in se_cols:
- cursor.execute("ALTER TABLE schedule_executions ADD COLUMN validates_execution_id TEXT")
+ _safe_add_column(
+ cursor, "schedule_executions", "business_status",
+ "ALTER TABLE schedule_executions ADD COLUMN business_status TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "validated_at",
+ "ALTER TABLE schedule_executions ADD COLUMN validated_at TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "validation_execution_id",
+ "ALTER TABLE schedule_executions ADD COLUMN validation_execution_id TEXT",
+ )
+ _safe_add_column(
+ cursor, "schedule_executions", "validates_execution_id",
+ "ALTER TABLE schedule_executions ADD COLUMN validates_execution_id TEXT",
+ )
# Indexes for validation queries
cursor.execute(
@@ -1280,26 +1329,22 @@ def _migrate_group_auth_mode(cursor, conn):
Supports: 'none' (default, current behavior), 'any_verified' (at least one verified member).
"""
# Add group_auth_mode to agent_ownership
- cursor.execute("PRAGMA table_info(agent_ownership)")
- ao_cols = {row[1] for row in cursor.fetchall()}
- if "group_auth_mode" not in ao_cols:
- cursor.execute(
- "ALTER TABLE agent_ownership ADD COLUMN group_auth_mode TEXT DEFAULT 'none'"
- )
+ if _safe_add_column(
+ cursor, "agent_ownership", "group_auth_mode",
+ "ALTER TABLE agent_ownership ADD COLUMN group_auth_mode TEXT DEFAULT 'none'",
+ ):
logger.info("Added group_auth_mode column to agent_ownership")
- # Add verified_by_email to telegram_group_configs
- cursor.execute("PRAGMA table_info(telegram_group_configs)")
- tg_cols = {row[1] for row in cursor.fetchall()}
- if "verified_by_email" not in tg_cols:
- cursor.execute(
- "ALTER TABLE telegram_group_configs ADD COLUMN verified_by_email TEXT"
- )
+ # Add verified_by_email + verified_at to telegram_group_configs
+ if _safe_add_column(
+ cursor, "telegram_group_configs", "verified_by_email",
+ "ALTER TABLE telegram_group_configs ADD COLUMN verified_by_email TEXT",
+ ):
logger.info("Added verified_by_email column to telegram_group_configs")
- if "verified_at" not in tg_cols:
- cursor.execute(
- "ALTER TABLE telegram_group_configs ADD COLUMN verified_at TEXT"
- )
+ if _safe_add_column(
+ cursor, "telegram_group_configs", "verified_at",
+ "ALTER TABLE telegram_group_configs ADD COLUMN verified_at TEXT",
+ ):
logger.info("Added verified_at column to telegram_group_configs")
conn.commit()
@@ -1378,13 +1423,13 @@ def _migrate_audit_log_table(cursor, conn):
def _migrate_agent_git_config_pat(cursor, conn):
"""Add github_pat_encrypted column to agent_git_config for per-agent GitHub PAT (#347)."""
- cursor.execute("PRAGMA table_info(agent_git_config)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "github_pat_encrypted" not in columns:
- print("Adding github_pat_encrypted column to agent_git_config for per-agent GitHub PAT...")
- cursor.execute("ALTER TABLE agent_git_config ADD COLUMN github_pat_encrypted TEXT")
-
+ _safe_add_column(
+ cursor,
+ "agent_git_config",
+ "github_pat_encrypted",
+ "ALTER TABLE agent_git_config ADD COLUMN github_pat_encrypted TEXT",
+ log_msg="Adding github_pat_encrypted column to agent_git_config for per-agent GitHub PAT...",
+ )
conn.commit()
@@ -1395,51 +1440,47 @@ def _migrate_sync_health(cursor, conn):
- agent_git_config gets auto_sync_enabled and freeze_schedules_if_sync_failing
so operators can opt in to the 15-min heartbeat and pause schedules when
sync breaks.
- Idempotent: guards every change with PRAGMA table_info / table-existence checks.
+ Idempotent and concurrent-safe: ALTER TABLE goes through _safe_add_column
+ which catches the duplicate-column race between workers (#456); CREATE
+ TABLE uses IF NOT EXISTS, which SQLite handles atomically.
"""
# 1. New columns on agent_git_config
- cursor.execute("PRAGMA table_info(agent_git_config)")
- cols = {row[1] for row in cursor.fetchall()}
- if "auto_sync_enabled" not in cols:
- print("Adding auto_sync_enabled column to agent_git_config for auto-sync cadence (#389)...")
- cursor.execute(
- "ALTER TABLE agent_git_config ADD COLUMN auto_sync_enabled INTEGER DEFAULT 0"
- )
- if "freeze_schedules_if_sync_failing" not in cols:
- print(
- "Adding freeze_schedules_if_sync_failing column to agent_git_config (#389)..."
- )
- cursor.execute(
- "ALTER TABLE agent_git_config "
- "ADD COLUMN freeze_schedules_if_sync_failing INTEGER DEFAULT 0"
- )
+ _safe_add_column(
+ cursor,
+ "agent_git_config",
+ "auto_sync_enabled",
+ "ALTER TABLE agent_git_config ADD COLUMN auto_sync_enabled INTEGER DEFAULT 0",
+ log_msg="Adding auto_sync_enabled column to agent_git_config for auto-sync cadence (#389)...",
+ )
+ _safe_add_column(
+ cursor,
+ "agent_git_config",
+ "freeze_schedules_if_sync_failing",
+ "ALTER TABLE agent_git_config ADD COLUMN freeze_schedules_if_sync_failing INTEGER DEFAULT 0",
+ log_msg="Adding freeze_schedules_if_sync_failing column to agent_git_config (#389)...",
+ )
- # 2. New agent_sync_state table
+ # 2. New agent_sync_state table (CREATE TABLE IF NOT EXISTS is atomic)
cursor.execute(
- "SELECT name FROM sqlite_master WHERE type='table' AND name='agent_sync_state'"
- )
- if cursor.fetchone() is None:
- print("Creating agent_sync_state table for sync-health observability (#389)...")
- cursor.execute(
- """
- CREATE TABLE agent_sync_state (
- agent_name TEXT PRIMARY KEY,
- last_sync_at TEXT,
- last_sync_status TEXT,
- consecutive_failures INTEGER DEFAULT 0,
- last_error_summary TEXT,
- last_remote_sha_main TEXT,
- last_remote_sha_working TEXT,
- ahead_main INTEGER DEFAULT 0,
- behind_main INTEGER DEFAULT 0,
- ahead_working INTEGER DEFAULT 0,
- behind_working INTEGER DEFAULT 0,
- last_check_at TEXT,
- updated_at TEXT NOT NULL,
- FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
- )
- """
+ """
+ CREATE TABLE IF NOT EXISTS agent_sync_state (
+ agent_name TEXT PRIMARY KEY,
+ last_sync_at TEXT,
+ last_sync_status TEXT,
+ consecutive_failures INTEGER DEFAULT 0,
+ last_error_summary TEXT,
+ last_remote_sha_main TEXT,
+ last_remote_sha_working TEXT,
+ ahead_main INTEGER DEFAULT 0,
+ behind_main INTEGER DEFAULT 0,
+ ahead_working INTEGER DEFAULT 0,
+ behind_working INTEGER DEFAULT 0,
+ last_check_at TEXT,
+ updated_at TEXT NOT NULL,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
)
+ """
+ )
cursor.execute(
"CREATE INDEX IF NOT EXISTS idx_sync_state_status "
@@ -1455,12 +1496,13 @@ def _migrate_proactive_messaging(cursor, conn):
Users must explicitly opt-in to receive proactive messages from agents.
Default is 0 (no proactive messages allowed).
"""
- cursor.execute("PRAGMA table_info(agent_sharing)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "allow_proactive" not in columns:
- print("Adding allow_proactive column to agent_sharing for proactive messaging consent...")
- cursor.execute("ALTER TABLE agent_sharing ADD COLUMN allow_proactive INTEGER DEFAULT 0")
+ _safe_add_column(
+ cursor,
+ "agent_sharing",
+ "allow_proactive",
+ "ALTER TABLE agent_sharing ADD COLUMN allow_proactive INTEGER DEFAULT 0",
+ log_msg="Adding allow_proactive column to agent_sharing for proactive messaging consent...",
+ )
# Create index for efficient lookup of proactive-enabled shares
cursor.execute(
@@ -1630,19 +1672,72 @@ def _migrate_agent_git_config_branch_ownership(cursor, conn):
def _migrate_agent_schedules_webhook(cursor, conn):
"""Add webhook_token and webhook_enabled columns to agent_schedules (WEBHOOK-001, #291)."""
- cursor.execute("PRAGMA table_info(agent_schedules)")
- columns = {row[1] for row in cursor.fetchall()}
-
- if "webhook_token" not in columns:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN webhook_token TEXT")
- if "webhook_enabled" not in columns:
- cursor.execute("ALTER TABLE agent_schedules ADD COLUMN webhook_enabled INTEGER DEFAULT 0")
+ _safe_add_column(
+ cursor, "agent_schedules", "webhook_token",
+ "ALTER TABLE agent_schedules ADD COLUMN webhook_token TEXT",
+ )
+ _safe_add_column(
+ cursor, "agent_schedules", "webhook_enabled",
+ "ALTER TABLE agent_schedules ADD COLUMN webhook_enabled INTEGER DEFAULT 0",
+ )
# Partial unique index — only applies to rows where a token is set
cursor.execute(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_schedules_webhook_token "
"ON agent_schedules(webhook_token) WHERE webhook_token IS NOT NULL"
)
+ conn.commit()
+
+
+def _migrate_agent_shared_files(cursor, conn):
+ """Create agent_shared_files table and add file_sharing_enabled to agent_ownership.
+
+ FILES-001 / amazing-file-outbound: outbound file sharing via public URL that
+ inherits the agent's channel-access policy. Agents call the `share_file` MCP
+ tool; backend extracts the named file via Docker SDK `get_archive` and mints
+ a token-scoped download URL under /api/files/{id}.
+
+ FK uses ON UPDATE CASCADE so rows self-heal if an agent is renamed
+ (defense-in-depth alongside the explicit UPDATE in
+ db.agent_settings.metadata.rename_agent).
+ """
+ cursor.execute("""
+ CREATE TABLE IF NOT EXISTS agent_shared_files (
+ id TEXT PRIMARY KEY,
+ agent_name TEXT NOT NULL,
+ filename TEXT NOT NULL,
+ stored_filename TEXT NOT NULL,
+ size_bytes INTEGER NOT NULL,
+ mime_type TEXT,
+ download_token TEXT UNIQUE NOT NULL,
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL,
+ revoked_at TEXT,
+ one_time INTEGER DEFAULT 0,
+ consumed_at TEXT,
+ download_count INTEGER DEFAULT 0,
+ last_downloaded_at TEXT,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
+ ON DELETE CASCADE ON UPDATE CASCADE
+ )
+ """)
+ cursor.execute(
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_agent ON agent_shared_files(agent_name)"
+ )
+ cursor.execute(
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_token ON agent_shared_files(download_token)"
+ )
+ cursor.execute(
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_expires ON agent_shared_files(expires_at) WHERE revoked_at IS NULL"
+ )
+
+ cursor.execute("PRAGMA table_info(agent_ownership)")
+ columns = {row[1] for row in cursor.fetchall()}
+ if "file_sharing_enabled" not in columns:
+ cursor.execute(
+ "ALTER TABLE agent_ownership ADD COLUMN file_sharing_enabled INTEGER DEFAULT 0"
+ )
conn.commit()
@@ -1699,4 +1794,5 @@ def _migrate_agent_schedules_webhook(cursor, conn):
("sync_health", _migrate_sync_health),
("whatsapp_bindings", _migrate_whatsapp_bindings),
("agent_schedules_webhook", _migrate_agent_schedules_webhook),
+ ("agent_shared_files", _migrate_agent_shared_files),
]
diff --git a/src/backend/db/public_links.py b/src/backend/db/public_links.py
index f3340f99f..353bb703d 100644
--- a/src/backend/db/public_links.py
+++ b/src/backend/db/public_links.py
@@ -319,6 +319,42 @@ def validate_session(self, link_id: str, session_token: str) -> tuple[bool, Opti
return True, email
+ def validate_agent_session(
+ self, agent_name: str, session_token: str
+ ) -> tuple[bool, Optional[str]]:
+ """
+ Validate a session token against ANY public link for the agent.
+
+ A session verified on any of an agent's public links counts as
+ valid for cross-resource access — specifically, FILES-001
+ downloads (/api/files/{id}) reuse this instead of minting a
+ separate verification flow.
+
+ Returns: (is_valid, email_if_valid)
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute(
+ """
+ SELECT v.email, v.session_expires_at
+ FROM public_link_verifications v
+ JOIN agent_public_links l ON v.link_id = l.id
+ WHERE l.agent_name = ?
+ AND v.session_token = ?
+ AND v.verified = 1
+ """,
+ (agent_name, session_token),
+ )
+ row = cursor.fetchone()
+
+ if not row:
+ return False, None
+
+ email, session_expires = row
+ if _utcnow() > _parse_aware(session_expires):
+ return False, None
+ return True, email
+
def count_recent_verification_requests(self, email: str, minutes: int = 10) -> int:
"""Count verification requests for an email in the last N minutes."""
cutoff = (datetime.utcnow() - timedelta(minutes=minutes)).isoformat()
diff --git a/src/backend/db/schedules.py b/src/backend/db/schedules.py
index d7bdca377..3eedd7dd6 100644
--- a/src/backend/db/schedules.py
+++ b/src/backend/db/schedules.py
@@ -25,7 +25,6 @@
# when Phase 3 fails an execution. Used to scope the residual-race WARNING log
# below so it doesn't misfire on other legitimate FAILED→SUCCESS transitions
# (e.g. Phase 0 auto-terminate, Phase 1 stale cleanup, startup recovery).
-_STALE_SLOT_ERROR_PATTERN = "Stale execution — slot TTL expired"
class ScheduleOperations:
@@ -1004,66 +1003,65 @@ def update_execution_status(
) -> bool:
"""Update execution status when completed.
+ CAS contract (RELIABILITY-005): SUCCESS writes are unconditional — the agent's
+ own completion result always wins. Non-success terminal writes (FAILED, CANCELLED)
+ are guarded so they cannot overwrite an already-terminal status, preventing
+ cleanup paths from silently clobbering a real completion.
+
Args:
claude_session_id: Claude Code session ID for --resume support (EXEC-023)
"""
+ # Terminal states that a non-success write must not overwrite.
+ _TERMINAL = (
+ TaskExecutionStatus.SUCCESS,
+ TaskExecutionStatus.FAILED,
+ TaskExecutionStatus.CANCELLED,
+ TaskExecutionStatus.SKIPPED,
+ )
+
with get_db_connection() as conn:
cursor = conn.cursor()
- # Get started_at for duration calculation and current status/error
- # for #378 residual-race observability (see log below).
cursor.execute(
- "SELECT started_at, status, error FROM schedule_executions WHERE id = ?",
+ "SELECT started_at FROM schedule_executions WHERE id = ?",
(execution_id,),
)
row = cursor.fetchone()
if not row:
return False
- # #378: warn when SUCCESS overwrites a Phase-3 phantom-stale FAILED.
- # This lets us observe residual races in production without
- # changing update semantics (agent's response still wins). Scoped
- # to the stale-slot error pattern so other legitimate FAILED→SUCCESS
- # transitions (Phase 0/1 recovery, startup recovery) don't misfire.
- current_status = row["status"] if "status" in row.keys() else None
- current_error = row["error"] if "error" in row.keys() else None
- if (
- status == TaskExecutionStatus.SUCCESS
- and current_status == TaskExecutionStatus.FAILED
- and current_error
- and _STALE_SLOT_ERROR_PATTERN in current_error
- ):
- logger.warning(
- f"[DB] SUCCESS overwrote Phase-3 stale-slot FAILED for execution "
- f"{execution_id} — residual race condition (#378). Prior error: "
- f"{current_error[:200]}"
- )
-
- # Use parse_iso_timestamp to handle both 'Z' and non-'Z' timestamps
started_at = parse_iso_timestamp(row["started_at"])
completed_at = parse_iso_timestamp(utc_now_iso())
duration_ms = int((completed_at - started_at).total_seconds() * 1000)
- cursor.execute("""
- UPDATE schedule_executions
- SET status = ?, completed_at = ?, duration_ms = ?, response = ?, error = ?,
- context_used = ?, context_max = ?, cost = ?, tool_calls = ?, execution_log = ?,
- claude_session_id = ?
- WHERE id = ?
- """, (
- status,
- to_utc_iso(completed_at), # Use UTC with 'Z' suffix
- duration_ms,
- response,
- error,
- context_used,
- context_max,
- cost,
- tool_calls,
- execution_log,
- claude_session_id,
- execution_id
- ))
+ if status == TaskExecutionStatus.SUCCESS:
+ # Agent's own completion result always wins.
+ cursor.execute("""
+ UPDATE schedule_executions
+ SET status = ?, completed_at = ?, duration_ms = ?, response = ?, error = ?,
+ context_used = ?, context_max = ?, cost = ?, tool_calls = ?,
+ execution_log = ?, claude_session_id = ?
+ WHERE id = ?
+ """, (
+ status, to_utc_iso(completed_at), duration_ms, response, error,
+ context_used, context_max, cost, tool_calls, execution_log,
+ claude_session_id, execution_id,
+ ))
+ else:
+ # Non-success terminal write: block if already terminal so cleanup
+ # paths cannot overwrite a real completion (RELIABILITY-005).
+ cursor.execute("""
+ UPDATE schedule_executions
+ SET status = ?, completed_at = ?, duration_ms = ?, response = ?, error = ?,
+ context_used = ?, context_max = ?, cost = ?, tool_calls = ?,
+ execution_log = ?, claude_session_id = ?
+ WHERE id = ? AND status NOT IN (?, ?, ?, ?)
+ """, (
+ status, to_utc_iso(completed_at), duration_ms, response, error,
+ context_used, context_max, cost, tool_calls, execution_log,
+ claude_session_id, execution_id, *_TERMINAL,
+ ))
+
conn.commit()
return cursor.rowcount > 0
@@ -1532,14 +1530,17 @@ def mark_stale_executions_failed(self, timeout_minutes: int = 30) -> int:
started_at = parse_iso_timestamp(row["started_at"])
duration_ms = int((completed_at - started_at).total_seconds() * 1000)
# SQL literal matches TaskExecutionStatus.FAILED
+ # RELIABILITY-005: guard the UPDATE so a SUCCESS that arrived
+ # between the SELECT and this UPDATE is never overwritten.
cursor.execute("""
UPDATE schedule_executions
SET status = ?,
completed_at = ?,
duration_ms = ?,
error = 'Marked as failed by cleanup: exceeded ' || ? || '-minute timeout'
- WHERE id = ?
- """, (TaskExecutionStatus.FAILED, now, duration_ms, str(timeout_minutes), row["id"]))
+ WHERE id = ? AND status = ?
+ """, (TaskExecutionStatus.FAILED, now, duration_ms, str(timeout_minutes),
+ row["id"], TaskExecutionStatus.RUNNING))
conn.commit()
return len(stale_rows)
@@ -1580,14 +1581,17 @@ def mark_no_session_executions_failed(self, timeout_seconds: int = 60) -> int:
for row in no_session_rows:
started_at = parse_iso_timestamp(row["started_at"])
duration_ms = int((completed_at - started_at).total_seconds() * 1000)
+ # RELIABILITY-005: guard the UPDATE so a SUCCESS that arrived
+ # between the SELECT and this UPDATE is never overwritten.
cursor.execute("""
UPDATE schedule_executions
SET status = ?,
completed_at = ?,
duration_ms = ?,
error = 'Silent launch failure: no Claude session created within ' || ? || ' seconds'
- WHERE id = ?
- """, (TaskExecutionStatus.FAILED, now, duration_ms, str(timeout_seconds), row["id"]))
+ WHERE id = ? AND status = ?
+ """, (TaskExecutionStatus.FAILED, now, duration_ms, str(timeout_seconds),
+ row["id"], TaskExecutionStatus.RUNNING))
conn.commit()
return len(no_session_rows)
diff --git a/src/backend/db/schema.py b/src/backend/db/schema.py
index 1c7292358..67076d2e3 100644
--- a/src/backend/db/schema.py
+++ b/src/backend/db/schema.py
@@ -12,6 +12,7 @@
- Activities: agent_activities
- Permissions: agent_permissions
- Shared Folders: agent_shared_folder_config
+- Shared Files (outbound): agent_shared_files
- Settings: system_settings
- Public Links: agent_public_links, public_link_verifications, public_link_usage
- Public Chat: public_chat_sessions, public_chat_messages, public_user_memory
@@ -70,6 +71,7 @@
open_access INTEGER DEFAULT 0,
group_auth_mode TEXT DEFAULT 'none',
guardrails_config TEXT,
+ file_sharing_enabled INTEGER DEFAULT 0,
FOREIGN KEY (owner_id) REFERENCES users(id),
FOREIGN KEY (subscription_id) REFERENCES subscription_credentials(id)
)
@@ -292,6 +294,31 @@
)
""",
+ # -------------------------------------------------------------------------
+ # Shared Files (outbound agent-to-user file sharing via public URL)
+ # -------------------------------------------------------------------------
+ "agent_shared_files": """
+ CREATE TABLE IF NOT EXISTS agent_shared_files (
+ id TEXT PRIMARY KEY,
+ agent_name TEXT NOT NULL,
+ filename TEXT NOT NULL,
+ stored_filename TEXT NOT NULL,
+ size_bytes INTEGER NOT NULL,
+ mime_type TEXT,
+ download_token TEXT UNIQUE NOT NULL,
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL,
+ revoked_at TEXT,
+ one_time INTEGER DEFAULT 0,
+ consumed_at TEXT,
+ download_count INTEGER DEFAULT 0,
+ last_downloaded_at TEXT,
+ FOREIGN KEY (agent_name) REFERENCES agent_ownership(agent_name)
+ ON DELETE CASCADE ON UPDATE CASCADE
+ )
+ """,
+
# -------------------------------------------------------------------------
# Settings Tables
# -------------------------------------------------------------------------
@@ -817,6 +844,11 @@
"CREATE INDEX IF NOT EXISTS idx_shared_folders_expose ON agent_shared_folder_config(expose_enabled)",
"CREATE INDEX IF NOT EXISTS idx_shared_folders_consume ON agent_shared_folder_config(consume_enabled)",
+ # Shared files (outbound) indexes
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_agent ON agent_shared_files(agent_name)",
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_token ON agent_shared_files(download_token)",
+ "CREATE INDEX IF NOT EXISTS idx_agent_files_expires ON agent_shared_files(expires_at) WHERE revoked_at IS NULL",
+
# Public links indexes
"CREATE INDEX IF NOT EXISTS idx_public_links_token ON agent_public_links(token)",
"CREATE INDEX IF NOT EXISTS idx_public_links_agent ON agent_public_links(agent_name)",
diff --git a/src/backend/db/slack_channels.py b/src/backend/db/slack_channels.py
index 4779ecad2..f962d6dbb 100644
--- a/src/backend/db/slack_channels.py
+++ b/src/backend/db/slack_channels.py
@@ -197,6 +197,37 @@ def get_dm_default_agent(self, team_id: str) -> Optional[str]:
row = cursor.fetchone()
return row[0] if row else None
+ def set_dm_default(self, team_id: str, agent_name: str) -> bool:
+ """Make ``agent_name`` the DM-default for the workspace.
+
+ Single transaction: clear all existing flags, then set on the target.
+ Avoids any window where two agents would both look like the default
+ (the schema has no exclusivity constraint, so the read-side falls
+ back to ``LIMIT 1`` and would pick non-deterministically).
+
+ Returns True if the target row was updated, False if the agent is
+ not bound in this workspace (caller should 404).
+ """
+ with get_db_connection() as conn:
+ cursor = conn.cursor()
+ cursor.execute("BEGIN")
+ try:
+ cursor.execute(
+ "UPDATE slack_channel_agents SET is_dm_default = 0 WHERE team_id = ?",
+ (team_id,),
+ )
+ cursor.execute(
+ """UPDATE slack_channel_agents SET is_dm_default = 1
+ WHERE team_id = ? AND agent_name = ?""",
+ (team_id, agent_name),
+ )
+ changed = cursor.rowcount > 0
+ conn.commit()
+ except Exception:
+ conn.rollback()
+ raise
+ return changed
+
def get_agents_for_workspace(self, team_id: str) -> List[dict]:
"""Get all agent-channel bindings for a workspace."""
with get_db_connection() as conn:
@@ -229,7 +260,14 @@ def get_channel_for_agent(self, team_id: str, agent_name: str) -> Optional[dict]
return self._row_to_channel_agent(row)
def unbind_agent(self, team_id: str, agent_name: str) -> bool:
- """Remove an agent's channel binding."""
+ """Remove an agent's channel binding.
+
+ Pure delete — does not auto-promote a new DM default. The router
+ layer is responsible for refusing to unbind the current DM default
+ while other agents are still bound (#584). When the unbind target
+ is the only agent in the workspace, the binding is removed cleanly
+ and the workspace ends up with no Slack agents at all.
+ """
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
diff --git a/src/backend/db_models.py b/src/backend/db_models.py
index 692a863fd..a4ab6fd7a 100644
--- a/src/backend/db_models.py
+++ b/src/backend/db_models.py
@@ -427,12 +427,21 @@ class VerificationResponse(BaseModel):
error: Optional[str] = None
+class WebFileUpload(BaseModel):
+ """A file attachment sent via web chat (base64-encoded)."""
+ name: str
+ mimetype: str
+ size: int
+ data_base64: str # raw base64 or data: URI from FileReader.readAsDataURL()
+
+
class PublicChatRequest(BaseModel):
"""Request to chat via a public link."""
message: str
session_token: Optional[str] = None # Required if link requires email verification
session_id: Optional[str] = None # For anonymous links (stored in localStorage)
async_mode: bool = False # When true, return execution_id immediately for SSE streaming
+ files: Optional[List[WebFileUpload]] = None # File attachments (#364)
class PublicChatResponse(BaseModel):
diff --git a/src/backend/main.py b/src/backend/main.py
index cde4b0039..b07d18f94 100644
--- a/src/backend/main.py
+++ b/src/backend/main.py
@@ -64,6 +64,7 @@
from routers.ops import router as ops_router
from routers.public_links import router as public_links_router, set_websocket_manager as set_public_links_ws_manager
from routers.public import router as public_router
+from routers.files import router as files_router # FILES-001 — outbound file downloads
from routers.setup import router as setup_router, get_setup_token as get_setup_setup_token
from routers.telemetry import router as telemetry_router
from routers.logs import router as logs_router
@@ -90,6 +91,7 @@
from routers.debug import router as debug_router # #306 soak instrumentation
from routers.messages import router as messages_router # Proactive Messaging (#321)
from routers.webhooks import router as webhooks_router # Webhook triggers (WEBHOOK-001, #291)
+from routers.ws_tickets import router as ws_tickets_router # /ws ticket auth (#550)
# Import activity service
from services.activity_service import activity_service
@@ -100,6 +102,9 @@
# Import log archive service
from services.log_archive_service import log_archive_service
+# Import audit retention service (#552)
+from services.audit_retention_service import audit_retention_service
+
# Import operator queue sync service
from services.operator_queue_service import operator_queue_service, set_websocket_manager as set_opqueue_sync_ws_manager
from services.sync_health_service import sync_health_service
@@ -365,6 +370,13 @@ async def lifespan(app: FastAPI):
except Exception as e:
print(f"Error starting log archive service: {e}")
+ # Initialize audit retention service (#552)
+ try:
+ audit_retention_service.start()
+ print("Audit retention service started")
+ except Exception as e:
+ print(f"Error starting audit retention service: {e}")
+
# PERF-269: Stagger background services to reduce SQLite write contention
# Start operator queue sync service (OPS-001) — polls every 5s
try:
@@ -393,32 +405,31 @@ async def _start_sync_health_delayed():
print(f"Error starting sync health service: {e}")
asyncio.create_task(_start_sync_health_delayed())
- # BACKLOG-001: Register backlog drain as a slot-release callback and spawn
- # a 60s maintenance task. The maintenance loop handles two things:
+ # BACKLOG-001 / CAPACITY-CONSOLIDATE (#428): instantiate the unified
+ # CapacityManager (this also wires the slot-release → backlog-drain
+ # callback internally) and spawn the 60s maintenance loop. The
+ # maintenance loop handles two things:
# 1. Expire queued rows older than 24h (-> FAILED)
# 2. Drain orphans — queued work that missed its release callback
# (e.g. backend restarted between enqueue and drain).
try:
- from services.slot_service import get_slot_service
- from services.backlog_service import get_backlog_service
- _backlog = get_backlog_service()
- get_slot_service().register_on_release(_backlog.on_slot_released)
+ from services.capacity_manager import get_capacity_manager
+ capacity = get_capacity_manager()
- async def _backlog_maintenance_loop():
+ async def _capacity_maintenance_loop():
# First tick after a short delay so startup stays snappy.
await asyncio.sleep(15)
while True:
try:
- await _backlog.expire_stale(max_age_hours=24)
- await _backlog.drain_orphans_all()
+ await capacity.run_maintenance(max_age_hours=24)
except Exception as exc:
- logger.warning(f"[Backlog] maintenance tick failed: {exc}")
+ logger.warning(f"[Capacity] maintenance tick failed: {exc}")
await asyncio.sleep(60)
- asyncio.create_task(_backlog_maintenance_loop())
- print("Backlog service wired (callback + 60s maintenance)")
+ asyncio.create_task(_capacity_maintenance_loop())
+ print("CapacityManager initialised; maintenance loop running (60s)")
except Exception as e:
- print(f"Error wiring backlog service: {e}")
+ print(f"Error wiring CapacityManager: {e}")
# Recover orphaned regular task executions (Issue #128)
try:
@@ -550,6 +561,13 @@ async def _backlog_maintenance_loop():
except Exception as e:
print(f"Error stopping log archive service: {e}")
+ # Shutdown audit retention service (#552)
+ try:
+ audit_retention_service.stop()
+ print("Audit retention service stopped")
+ except Exception as e:
+ print(f"Error stopping audit retention service: {e}")
+
# Shutdown cleanup service
try:
cleanup_service.stop()
@@ -699,6 +717,7 @@ async def add_security_headers(request: Request, call_next):
app.include_router(ops_router)
app.include_router(public_links_router)
app.include_router(public_router)
+app.include_router(files_router) # FILES-001: /api/files/{id} — token-gated downloads
app.include_router(setup_router)
app.include_router(telemetry_router)
app.include_router(logs_router)
@@ -728,51 +747,50 @@ async def add_security_headers(request: Request, call_next):
app.include_router(users_router) # User Management (ROLE-001)
app.include_router(debug_router) # #306 soak dashboard
app.include_router(webhooks_router) # Webhook Triggers (WEBHOOK-001, #291)
+app.include_router(ws_tickets_router) # WebSocket auth tickets (#550)
# WebSocket endpoint
@app.websocket("/ws")
async def websocket_endpoint(
websocket: WebSocket,
- token: str = Query(default=None),
+ ticket: str = Query(default=None),
last_event_id: Optional[str] = Query(default=None, alias="last-event-id"),
):
"""
WebSocket endpoint for real-time updates.
- Security (#178, C-002): Authentication is REQUIRED before accepting the
- connection. The JWT token MUST be provided via query parameter:
- /ws?token=
+ Security (#178/C-002 + #550): authentication is REQUIRED before
+ ``websocket.accept()``. Clients first call ``POST /api/ws/ticket``
+ to mint a single-use 30-second opaque ticket, then connect to:
- Connections without a valid token are rejected before websocket.accept()
- to prevent any unauthenticated data leakage.
+ /ws?ticket=
+
+ Switching from a long-lived JWT in the URL to an opaque single-use
+ ticket closes the JWT-leak surface (nginx logs, browser history,
+ upstream proxies) flagged by the April 2026 remediation pentest
+ (finding 3.2.1) and mitigates CSWSH — a malicious page can't mint
+ a ticket on the victim's behalf because the ticket endpoint requires
+ the JWT in an ``Authorization`` header.
Reconnect replay (#306): clients may pass ``last-event-id=``
to receive events missed during a disconnect. Malformed or too-old ids
- produce a ``{"type": "resync_required"}`` message — the client must then
- fetch current state via REST.
+ produce a ``{"type": "resync_required"}`` message — the client must
+ then fetch current state via REST.
"""
- from jose import JWTError, jwt as jose_jwt
- from config import SECRET_KEY, ALGORITHM
from services.event_bus import validate_last_event_id
+ from services.ws_ticket_service import consume_ticket
- # Reject immediately if no token provided — before accept()
- if not token:
- await websocket.close(code=4001, reason="Authentication required: provide ?token=")
+ if not ticket:
+ await websocket.close(code=4001, reason="Authentication required: provide ?ticket=")
return
- # Validate token before accepting the connection
- try:
- payload = jose_jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
- username = payload.get("sub")
- if not username:
- await websocket.close(code=4001, reason="Invalid token: missing subject")
- return
- except JWTError:
- await websocket.close(code=4001, reason="Invalid authentication token")
+ payload = consume_ticket(ticket)
+ if not payload or not payload.get("sub"):
+ await websocket.close(code=4001, reason="Invalid or expired WebSocket ticket")
return
- # Token validated — now accept the connection
+ # Ticket validated — now accept the connection
await manager.connect(websocket, last_event_id=validate_last_event_id(last_event_id))
try:
diff --git a/src/backend/models.py b/src/backend/models.py
index 0c7f6a026..dcd8b1f64 100644
--- a/src/backend/models.py
+++ b/src/backend/models.py
@@ -1,12 +1,13 @@
"""
Pydantic models for the Trinity backend API.
"""
-from pydantic import BaseModel
+from pydantic import BaseModel, Field
from typing import Dict, List, Optional
from datetime import datetime
from enum import Enum
from utils.helpers import to_utc_iso
+from db_models import WebFileUpload # noqa: F401 — re-exported for router imports
class AgentConfig(BaseModel):
@@ -95,6 +96,7 @@ class ParallelTaskRequest(BaseModel):
chat_session_id: Optional[str] = None # Explicit chat session ID to save messages to (for continuing existing sessions)
resume_session_id: Optional[str] = None # Claude Code session ID to resume (EXEC-023)
inject_result: Optional[bool] = False # If true and self-task, inject result as message in originating chat session (SELF-EXEC-001)
+ files: Optional[List[WebFileUpload]] = None # File attachments (#364)
# ============================================================================
@@ -184,17 +186,30 @@ class ExecutionSource(str, Enum):
class TaskExecutionStatus(str, Enum):
"""
- Canonical status values for task/schedule executions persisted to the database.
-
- Used across: TaskExecutionService, db/schedules.py, scheduler/database.py, chat.py, cleanup_service.py.
- NOT used by: ExecutionQueue (uses QueueItemStatus).
+ Canonical status values for task/schedule executions (RELIABILITY-005).
+
+ State machine — allowed transitions and authorized writers:
+
+ [create] → QUEUED writer: TaskExecutionService / BacklogService
+ QUEUED → RUNNING writer: BacklogService (drain) / TaskExecutionService
+ RUNNING → SUCCESS writer: TaskExecutionService (agent HTTP response — always wins)
+ RUNNING → FAILED writer: TaskExecutionService / CleanupService (guarded: no overwrite of terminal)
+ RUNNING → CANCELLED writer: terminate handler (guarded)
+ RUNNING → PENDING_RETRY writer: scheduler retry handler (#271)
+ PENDING_RETRY → RUNNING writer: scheduler retry dispatch
+ any → SKIPPED writer: TaskExecutionService (capacity overflow path)
+
+ CAS invariant (db/schedules.py update_execution_status): SUCCESS writes are
+ unconditional; all other terminal writes are blocked if the row is already
+ in a terminal state, preventing cleanup paths from overwriting a real completion.
"""
- QUEUED = "queued" # BACKLOG-001: Persisted async task waiting for a free slot
+ QUEUED = "queued" # Persisted async task waiting for a free slot (BACKLOG-001)
RUNNING = "running"
SUCCESS = "success"
FAILED = "failed"
CANCELLED = "cancelled"
SKIPPED = "skipped"
+ PENDING_RETRY = "pending_retry" # Awaiting retry dispatch (#271)
class BusinessStatus(str, Enum):
@@ -406,3 +421,58 @@ class GithubPatPropagationResult(BaseModel):
updated: List[str]
skipped: List[AgentPropagationStatus]
failed: List[AgentPropagationStatus]
+
+
+# =============================================================================
+# Outbound File Sharing (FILES-001)
+# =============================================================================
+
+class ShareFileRequest(BaseModel):
+ """Body for POST /api/internal/agent-files/share (internal, agent-server path)."""
+ agent_name: str = Field(..., max_length=128)
+ filename: str = Field(..., min_length=1, max_length=255)
+ display_name: Optional[str] = Field(default=None, max_length=255)
+ expires_in: Optional[int] = None
+ # NOTE: `one_time` is deferred — the schema retains the columns
+ # so we can re-enable it later without a migration.
+
+
+class ShareFileMcpRequest(BaseModel):
+ """Body for POST /api/agents/{agent_name}/shared-files (MCP path).
+
+ The agent_name lives in the URL, so the body only needs the
+ per-share parameters.
+ """
+ filename: str = Field(..., min_length=1, max_length=255)
+ display_name: Optional[str] = Field(default=None, max_length=255)
+ expires_in: Optional[int] = None
+
+
+class ShareFileResponse(BaseModel):
+ """Response payload for a successful share."""
+ file_id: str
+ url: str
+ expires_at: str
+ size_bytes: int
+ mime_type: Optional[str] = None
+
+
+class SharedFileInfo(BaseModel):
+ """One row in the owner's file-sharing panel."""
+ file_id: str
+ filename: str
+ size_bytes: int
+ mime_type: Optional[str] = None
+ url: str
+ created_at: str
+ expires_at: str
+ download_count: int
+ last_downloaded_at: Optional[str] = None
+
+
+class SharedFilesList(BaseModel):
+ """Response for GET /api/agents/{name}/shared-files."""
+ agent_name: str
+ files: List[SharedFileInfo]
+ total_bytes: int
+ quota_bytes: int
diff --git a/src/backend/routers/agent_config.py b/src/backend/routers/agent_config.py
index cf4f42a92..038fcca3e 100644
--- a/src/backend/routers/agent_config.py
+++ b/src/backend/routers/agent_config.py
@@ -363,7 +363,7 @@ async def get_agent_capacity(
- slots: List of active slot details
"""
from db_models import AgentCapacity, SlotInfo
- from services.slot_service import get_slot_service
+ from services.capacity_manager import get_capacity_manager
# Check access
if not db.can_user_access_agent(current_user.username, agent_name):
@@ -376,9 +376,9 @@ async def get_agent_capacity(
# Get max_parallel_tasks from database
max_tasks = db.get_max_parallel_tasks(agent_name)
- # Get slot state from Redis
- slot_service = get_slot_service()
- slot_state = await slot_service.get_slot_state(agent_name, max_tasks)
+ # CAPACITY-CONSOLIDATE (#428): per-agent slot state via CapacityManager.
+ capacity = get_capacity_manager()
+ slot_state = await capacity.get_slot_state(agent_name, max_tasks)
# Convert to response model
slots = [
diff --git a/src/backend/routers/agent_files.py b/src/backend/routers/agent_files.py
index 2fa4d0617..86d95ca6a 100644
--- a/src/backend/routers/agent_files.py
+++ b/src/backend/routers/agent_files.py
@@ -24,6 +24,14 @@
preview_agent_file_logic,
update_agent_file_logic,
get_agent_metrics_logic,
+ get_file_sharing_status_logic,
+ set_file_sharing_status_logic,
+)
+from models import ShareFileMcpRequest, ShareFileResponse, SharedFileInfo, SharedFilesList
+from services.agent_shared_files_service import (
+ create_share,
+ build_download_url,
+ MAX_AGENT_QUOTA_BYTES,
)
from services.platform_audit_service import platform_audit_service, AuditEventType
@@ -367,3 +375,160 @@ async def get_folder_consumers(
):
"""Get list of agents that can consume this agent's shared folder."""
return await get_folder_consumers_logic(agent_name, current_user)
+
+
+# ============================================================================
+# File Sharing (outbound) Endpoints — FILES-001 Step 2
+# ============================================================================
+
+
+@router.get("/{agent_name}/file-sharing")
+async def get_agent_file_sharing(
+ agent_name: str,
+ request: Request,
+ current_user: User = Depends(get_current_user),
+):
+ """
+ Get the outbound file-sharing status for an agent.
+
+ Returns:
+ - enabled: bool — whether the toggle is on
+ - volume_attached: bool — whether /home/developer/public is currently mounted
+ - restart_required: bool — true when enabled != volume_attached
+ - file_count / total_bytes / quota_bytes — placeholder zeros in Step 2;
+ wired to agent_shared_files in Step 3
+ """
+ return await get_file_sharing_status_logic(agent_name, current_user)
+
+
+@router.put("/{agent_name}/file-sharing")
+async def set_agent_file_sharing(
+ agent_name: str,
+ body: dict,
+ request: Request,
+ current_user: User = Depends(get_current_user),
+):
+ """
+ Enable or disable outbound file sharing for an agent (owner-only).
+
+ Body:
+ - enabled: True/False
+
+ Flipping the flag does NOT mount/unmount immediately — it sets
+ restart_required. The next stop/start cycle triggers container
+ recreation with the correct volume configuration.
+ """
+ return await set_file_sharing_status_logic(agent_name, body, current_user)
+
+
+@router.post(
+ "/{agent_name}/shared-files",
+ response_model=ShareFileResponse,
+ status_code=201,
+)
+async def share_agent_file(
+ agent_name: str,
+ body: ShareFileMcpRequest,
+ current_user: User = Depends(get_current_user),
+):
+ """
+ Mint a public download URL for a file the agent has written to its
+ publish dir (/home/developer/public/). Called by the `share_file`
+ MCP tool.
+
+ Auth: owner/admin of the agent, OR agent-scoped MCP key whose
+ agent_name matches the path. User-scoped MCP keys of non-owners
+ are rejected.
+ """
+ # Owner gate (the agent's owner always passes)
+ if not db.can_user_share_agent(current_user.username, agent_name):
+ raise HTTPException(
+ status_code=403,
+ detail="Only the owner or admin can share files from this agent.",
+ )
+
+ # Defense in depth: if this is an agent-scoped key, it must be for
+ # the same agent. Prevents Agent A's key from being used to share
+ # files from Agent B's volume even when both are owned by the same user.
+ actor_agent = getattr(current_user, "agent_name", None)
+ if actor_agent and actor_agent != agent_name:
+ raise HTTPException(
+ status_code=403,
+ detail="Agent-scoped MCP key cannot share files for a different agent.",
+ )
+
+ result = await create_share(
+ agent_name=agent_name,
+ filename=body.filename,
+ display_name=body.display_name,
+ expires_in=body.expires_in,
+ created_by=actor_agent or current_user.username,
+ )
+ return ShareFileResponse(**result)
+
+
+@router.get(
+ "/{agent_name}/shared-files",
+ response_model=SharedFilesList,
+)
+async def list_agent_shared_files(
+ agent_name: str,
+ current_user: User = Depends(get_current_user),
+):
+ """
+ List active (non-revoked, non-expired) shared files for an agent.
+ Restricted to owner + admin (C7) — the list includes full download
+ URLs with tokens, so anyone who can see the list can effectively
+ reuse the shares. That's a capability that belongs with `share_file`
+ and `revoke` (both owner-only), not with shared-user read access.
+ """
+ if not db.can_user_share_agent(current_user.username, agent_name):
+ raise HTTPException(status_code=403, detail="Only the owner or admin can view shared files")
+
+ rows = db.list_active_shared_files_for_agent(agent_name)
+ files = [
+ SharedFileInfo(
+ file_id=row["id"],
+ filename=row["filename"],
+ size_bytes=row["size_bytes"],
+ mime_type=row["mime_type"],
+ url=build_download_url(row["id"], row["download_token"]),
+ created_at=row["created_at"],
+ expires_at=row["expires_at"],
+ download_count=row["download_count"] or 0,
+ last_downloaded_at=row["last_downloaded_at"],
+ )
+ for row in rows
+ ]
+ total_bytes = db.total_shared_file_bytes_for_agent(agent_name)
+ return SharedFilesList(
+ agent_name=agent_name,
+ files=files,
+ total_bytes=total_bytes,
+ quota_bytes=MAX_AGENT_QUOTA_BYTES,
+ )
+
+
+@router.delete(
+ "/{agent_name}/shared-files/{file_id}",
+ status_code=204,
+)
+async def revoke_agent_shared_file(
+ agent_name: str,
+ file_id: str,
+ current_user: User = Depends(get_current_user),
+):
+ """
+ Revoke a shared file. Owner/admin only. Idempotent — revoking a
+ revoked or missing file returns 204 either way.
+ """
+ if not db.can_user_share_agent(current_user.username, agent_name):
+ raise HTTPException(status_code=403, detail="Only the owner can revoke shares")
+
+ row = db.get_agent_shared_file(file_id)
+ if row and row["agent_name"] != agent_name:
+ # Preventing cross-agent revoke via URL manipulation
+ raise HTTPException(status_code=404, detail="File not found for this agent")
+
+ db.revoke_agent_shared_file(file_id)
+ return None
diff --git a/src/backend/routers/agents.py b/src/backend/routers/agents.py
index 71f7ffb3c..a09fc8d8a 100644
--- a/src/backend/routers/agents.py
+++ b/src/backend/routers/agents.py
@@ -288,15 +288,15 @@ async def get_all_agent_slots(
- timestamp: ISO timestamp of response
"""
from db_models import BulkSlotState
- from services.slot_service import get_slot_service
+ from services.capacity_manager import get_capacity_manager
from datetime import datetime
# Get all agents with their capacities
agent_capacities = db.get_all_agents_parallel_capacity()
- # Get slot states from Redis
- slot_service = get_slot_service()
- slot_states = await slot_service.get_all_slot_states(agent_capacities)
+ # CAPACITY-CONSOLIDATE (#428): bulk capacity meter via CapacityManager.
+ capacity = get_capacity_manager()
+ slot_states = await capacity.get_all_states(agent_capacities)
return BulkSlotState(
agents=slot_states,
@@ -441,9 +441,11 @@ async def delete_agent_endpoint(agent_name: str, request: Request, current_user:
# BACKLOG-001: Cancel any queued backlog items before deleting the agent
# so they don't sit around in schedule_executions pointing at a dead agent.
+ # CAPACITY-CONSOLIDATE (#428): single CapacityManager.cancel_all_overflow
+ # covers both in-memory queue and persistent backlog.
try:
- from services.backlog_service import get_backlog_service
- await get_backlog_service().cancel_all_backlog(
+ from services.capacity_manager import get_capacity_manager
+ await get_capacity_manager().cancel_all_overflow(
agent_name, reason="agent_deleted"
)
except Exception as e:
@@ -488,6 +490,30 @@ async def delete_agent_endpoint(agent_name: str, request: Request, current_user:
except Exception as e:
logger.warning(f"Failed to delete shared folder config for agent {agent_name}: {e}")
+ # Delete per-agent public volume + shared-file rows + on-disk bytes
+ # (FILES-001). Backend connections don't PRAGMA foreign_keys=ON, so
+ # we can't rely on the FK ON DELETE CASCADE — follow the same explicit
+ # pattern used elsewhere in the codebase (see db.agent_settings.metadata
+ # :rename_agent which also manually updates all 16 child tables).
+ try:
+ stored_filenames = db.delete_shared_files_for_agent(agent_name)
+ for stored in stored_filenames:
+ try:
+ path = Path("/data/agent-files") / stored
+ if path.exists():
+ path.unlink()
+ except Exception as e:
+ logger.warning(f"Failed to unlink shared file {stored}: {e}")
+
+ public_volume_name = db.get_public_volume_name(agent_name)
+ try:
+ public_volume = await volume_get(public_volume_name)
+ await volume_remove(public_volume)
+ except docker.errors.NotFound:
+ pass
+ except Exception as e:
+ logger.warning(f"Failed to delete public volume for agent {agent_name}: {e}")
+
# Delete agent tags (ORG-001)
try:
db.delete_agent_tags(agent_name)
diff --git a/src/backend/routers/chat.py b/src/backend/routers/chat.py
index 5f7a30fc6..fd25b3e02 100644
--- a/src/backend/routers/chat.py
+++ b/src/backend/routers/chat.py
@@ -16,8 +16,12 @@
from dependencies import get_current_user, get_authorized_agent, get_owned_agent
from services.docker_service import get_agent_container
from services.activity_service import activity_service
-from services.execution_queue import get_execution_queue, QueueFullError, AgentBusyError
-from services.slot_service import get_slot_service
+from services.upload_service import process_file_uploads, decode_web_file, WEB_MAX_FILES, WEB_MAX_FILE_SIZE, WEB_MAX_IMAGE_SIZE, WEB_MAX_TOTAL_IMAGE_SIZE
+from services.capacity_manager import (
+ CapacityFull,
+ PersistentTaskPayload,
+ get_capacity_manager,
+)
from services.task_execution_service import (
get_task_execution_service,
agent_post_with_retry,
@@ -108,20 +112,37 @@ async def chat_with_agent(
else:
source = ExecutionSource.USER
- # Create execution request and submit to queue
- queue = get_execution_queue()
- execution = queue.create_execution(
- agent_name=name,
- message=request.message,
- source=source,
- source_agent=x_source_agent,
- source_user_id=str(current_user.id),
- source_user_email=current_user.email or current_user.username
- )
-
+ # CAPACITY-CONSOLIDATE (#428): single CapacityManager.acquire call replaces
+ # the prior ExecutionQueue.submit + SlotService.acquire_slot pair. /chat
+ # shares the agent's parallel pool with /task (same `max_parallel_tasks`)
+ # and spills to an in-memory queue (depth 3, preserved from the original
+ # ExecutionQueue MAX_QUEUE_SIZE) when the pool is full. The agent's Claude
+ # subprocess is the actual serial bottleneck downstream.
+ import uuid as _uuid
+ capacity = get_capacity_manager()
+ chat_execution_id = str(_uuid.uuid4())
+ chat_timeout = db.get_execution_timeout(name)
+ max_parallel_tasks = db.get_max_parallel_tasks(name)
try:
- queue_result, execution = await queue.submit(execution, wait_if_busy=True)
- logger.info(f"[Chat] Agent '{name}' execution {execution.id}: {queue_result}")
+ capacity_result = await capacity.acquire(
+ agent_name=name,
+ execution_id=chat_execution_id,
+ max_concurrent=max_parallel_tasks,
+ message_preview=request.message[:100] if request.message else "",
+ timeout_seconds=chat_timeout,
+ overflow_policy="queue_in_memory",
+ source=source,
+ source_agent=x_source_agent,
+ source_user_id=str(current_user.id),
+ source_user_email=current_user.email or current_user.username,
+ message=request.message,
+ )
+ queue_result = (
+ "running"
+ if capacity_result.state == "admitted"
+ else f"queued:{capacity_result.queue_position}"
+ )
+ logger.info(f"[Chat] Agent '{name}' execution {chat_execution_id}: {queue_result}")
await platform_audit_service.log(
event_type=AuditEventType.EXECUTION,
event_action="chat_started",
@@ -136,47 +157,34 @@ async def chat_with_agent(
endpoint=f"/api/agents/{name}/chat",
request_id=None,
details={
- "execution_id": execution.id,
+ "execution_id": chat_execution_id,
"queue_result": queue_result,
"source": source.value if hasattr(source, "value") else str(source),
"message_length": len(request.message) if request.message else 0,
},
)
- except QueueFullError as e:
- logger.warning(f"[Chat] Agent '{name}' queue full, rejecting request")
+ except CapacityFull as e:
+ logger.warning(f"[Chat] Agent '{name}' at capacity, rejecting request (reason={e.reason})")
raise HTTPException(
status_code=429,
detail={
"error": "Agent queue is full",
"agent": name,
- "queue_length": e.queue_length,
+ "queue_length": e.depth or 0,
"retry_after": 30,
- "message": f"Agent '{name}' is busy with {e.queue_length} queued requests. Please try again later."
+ "message": f"Agent '{name}' is busy. Please try again later."
}
)
# Track queue position for observability
- is_queued = queue_result.startswith("queued:")
-
- # Issue #98: Acquire a capacity slot so chat executions are visible in the
- # capacity meter. The queue still enforces serial chat; the slot makes the
- # resource usage visible to SlotService (single source of truth for load).
- slot_service = get_slot_service()
- chat_slot_acquired = False
- try:
- chat_timeout = db.get_execution_timeout(name)
- max_parallel_tasks = db.get_max_parallel_tasks(name)
- chat_slot_acquired = await slot_service.acquire_slot(
- agent_name=name,
- execution_id=execution.id,
- max_parallel_tasks=max_parallel_tasks,
- message_preview=request.message[:100] if request.message else "",
- timeout_seconds=chat_timeout,
- )
- if not chat_slot_acquired:
- logger.warning(f"[Chat] Agent '{name}' at capacity, could not acquire slot for chat {execution.id}")
- except Exception as e:
- logger.warning(f"[Chat] Failed to acquire slot for chat execution {execution.id}: {e}")
+ is_queued = capacity_result.state == "queued_in_memory"
+ # Backwards-compat names: existing code below references `execution.id`.
+ # Map the new chat_execution_id onto the old shape so the rest of the
+ # function stays diff-minimal.
+ class _ExecutionLite:
+ def __init__(self, eid: str):
+ self.id = eid
+ execution = _ExecutionLite(chat_execution_id)
# Create execution record for ALL chat calls (user, MCP, and agent-to-agent)
# This ensures every execution appears in the Tasks tab for unified tracking (#96)
@@ -455,13 +463,19 @@ async def chat_with_agent(
error=error_msg
)
- # SUB-003: Auto-switch subscription on rate-limit errors from agent
+ # SUB-003 (#441): Auto-switch on rate-limit (429) OR auth-class
+ # failures (503 from agent server, or auth indicators in the error).
+ from services.subscription_auto_switch import (
+ handle_subscription_failure,
+ is_auth_failure,
+ )
+
if agent_status_code == 429:
try:
- from services.subscription_auto_switch import handle_rate_limit_error
- switch_result = await handle_rate_limit_error(
+ switch_result = await handle_subscription_failure(
agent_name=name,
error_message=error_msg,
+ failure_kind="rate_limit",
)
if switch_result:
# Auto-switch happened — inform the caller
@@ -485,16 +499,41 @@ async def chat_with_agent(
# Preserve 429 from agent so frontend can show clear message
raise HTTPException(status_code=429, detail=error_msg)
+ if agent_status_code == 503 or is_auth_failure(error_msg):
+ try:
+ switch_result = await handle_subscription_failure(
+ agent_name=name,
+ error_message=error_msg,
+ failure_kind="auth",
+ )
+ if switch_result:
+ # Auto-switch happened — surface as 503 + retry hint so the
+ # frontend gets the same retry UX as the 429 path.
+ raise HTTPException(
+ status_code=503,
+ detail={
+ "error": error_msg,
+ "auto_switch": switch_result,
+ "message": (
+ f"Authentication failure on subscription. Auto-switched to "
+ f"'{switch_result['new_subscription']}'. Please retry."
+ ),
+ "retry_after": 15,
+ }
+ )
+ except HTTPException:
+ raise
+ except Exception as e:
+ logger.error(f"[SUB-003] Auto-switch check failed for '{name}': {e}")
+
raise HTTPException(
status_code=503,
detail=f"Failed to communicate with agent: {error_msg}"
)
finally:
- # Always release the queue slot when done
- await queue.complete(name, success=execution_success)
- # Issue #98: Release the capacity slot acquired for this chat execution
- if chat_slot_acquired:
- await slot_service.release_slot(name, execution.id)
+ # CAPACITY-CONSOLIDATE (#428): single release covers both the
+ # SlotService N-ary counter and the in-memory overflow bookkeeping.
+ await capacity.release(name, execution.id)
async def _persist_chat_session(
@@ -578,6 +617,7 @@ async def _run_async_task_with_persistence(
subscription_id: Optional[str] = None,
is_self_task: bool = False,
self_task_activity_id: Optional[str] = None,
+ images: Optional[list] = None,
):
"""
Async /task background wrapper (issue #95).
@@ -628,6 +668,7 @@ async def _run_async_task_with_persistence(
"timeout_seconds": request.timeout_seconds,
},
slot_already_held=True, # Router pre-acquired to preserve 429-upfront contract
+ images=images or [],
)
execution_time_ms = int((datetime.utcnow() - start_time).total_seconds() * 1000)
@@ -814,6 +855,44 @@ async def execute_parallel_task(
source = ExecutionSource.USER
triggered_by = "manual"
+ # (#364) File upload processing — done synchronously before the async/sync
+ # fork so bytes are decoded and written to the container before we return
+ # execution_id (async) or before execute_task runs (sync).
+ _upload_dir = None
+ _image_data: list = []
+ if request.files:
+ uploader = current_user.email or current_user.username
+ raw_files = [
+ {
+ "name": f.name,
+ "mimetype": f.mimetype,
+ "size": f.size,
+ "data": decode_web_file(f.dict()),
+ "id": f"f{i}",
+ }
+ for i, f in enumerate(request.files)
+ ]
+ file_descs, _upload_dir, all_writes_failed, _image_data = await process_file_uploads(
+ raw_files=raw_files,
+ agent_name=name,
+ container=container,
+ session_id=str(current_user.id),
+ uploader=uploader,
+ source="web",
+ max_files=WEB_MAX_FILES,
+ max_file_size=WEB_MAX_FILE_SIZE,
+ max_image_size=WEB_MAX_IMAGE_SIZE,
+ max_total_image_size=WEB_MAX_TOTAL_IMAGE_SIZE,
+ )
+ if all_writes_failed:
+ raise HTTPException(
+ status_code=502,
+ detail="File upload failed: could not write to agent workspace."
+ )
+ if file_descs:
+ file_block = "\n".join(file_descs)
+ request.message = f"{request.message}\n\n{file_block}"
+
# SUB-004: Look up subscription for usage tracking (best-effort)
try:
_task_subscription_id = db.get_agent_subscription_id(name)
@@ -903,66 +982,53 @@ async def execute_parallel_task(
}
)
- # Async mode: pre-acquire slot synchronously so at-capacity returns 429 upfront
- # (preserves existing client contract), then delegate the lifecycle to
- # TaskExecutionService via _run_async_task_with_persistence (issue #95).
+ # Async mode: pre-acquire capacity synchronously so at-capacity returns 429
+ # upfront (preserves existing client contract), then delegate the lifecycle
+ # to TaskExecutionService via _run_async_task_with_persistence (#95).
+ # CAPACITY-CONSOLIDATE (#428): one CapacityManager.acquire call replaces
+ # the prior slot_service.acquire_slot + backlog.enqueue dance.
if request.async_mode:
- slot_service = get_slot_service()
+ capacity = get_capacity_manager()
max_parallel_tasks = db.get_max_parallel_tasks(name)
effective_timeout = request.timeout_seconds
if effective_timeout is None:
effective_timeout = db.get_execution_timeout(name)
- slot_acquired = await slot_service.acquire_slot(
- agent_name=name,
- execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
- max_parallel_tasks=max_parallel_tasks,
- message_preview=request.message[:100] if request.message else "",
- timeout_seconds=effective_timeout,
- )
- if not slot_acquired:
- # BACKLOG-001: Spill to persistent backlog instead of returning 429.
- # True 429 only if the backlog is also at its configured depth.
- from services.backlog_service import get_backlog_service
- backlog = get_backlog_service()
- enqueued = await backlog.enqueue(
+
+ try:
+ cap_result = await capacity.acquire(
agent_name=name,
- execution_id=execution_id,
- request=request,
- effective_timeout=effective_timeout,
- user_id=current_user.id,
- user_email=current_user.email or current_user.username,
- subscription_id=_task_subscription_id,
- x_source_agent=x_source_agent,
- x_mcp_key_id=x_mcp_key_id,
- x_mcp_key_name=x_mcp_key_name,
- triggered_by=triggered_by,
- collaboration_activity_id=collaboration_activity_id,
- # #496: thread self-task fields so SELF-EXEC-001 (#264)
- # inject_result still works when a self-task overflows to backlog.
- is_self_task=is_self_task,
- self_task_activity_id=self_task_activity_id,
+ execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
+ max_concurrent=max_parallel_tasks,
+ message_preview=request.message[:100] if request.message else "",
+ timeout_seconds=effective_timeout,
+ overflow_policy="queue_persistent",
+ overflow_payload=PersistentTaskPayload(
+ request=request,
+ effective_timeout=effective_timeout,
+ user_id=current_user.id,
+ user_email=current_user.email or current_user.username,
+ subscription_id=_task_subscription_id,
+ x_source_agent=x_source_agent,
+ x_mcp_key_id=x_mcp_key_id,
+ x_mcp_key_name=x_mcp_key_name,
+ triggered_by=triggered_by,
+ collaboration_activity_id=collaboration_activity_id,
+ # #496: thread self-task fields so SELF-EXEC-001 (#264)
+ # inject_result still works when a self-task overflows.
+ is_self_task=is_self_task,
+ self_task_activity_id=self_task_activity_id,
+ ),
)
- if enqueued:
- logger.info(
- f"[Task Async] Agent '{name}' at capacity — execution {execution_id} queued to backlog"
- )
- return {
- "status": "queued",
- "execution_id": execution_id,
- "agent_name": name,
- "message": (
- f"Agent at capacity; task queued. Poll GET "
- f"/api/agents/{name}/executions/{execution_id} for results."
- ),
- "async_mode": True,
- }
-
- # Backlog also full: surface true 429.
+ except CapacityFull as e:
+ # Both capacity AND backlog are full — surface 429 with prior shape.
if execution_id:
db.update_execution_status(
execution_id=execution_id,
status=TaskExecutionStatus.FAILED,
- error=f"Agent at capacity ({max_parallel_tasks}/{max_parallel_tasks} parallel tasks running) and backlog is full"
+ error=(
+ f"Agent at capacity ({max_parallel_tasks}/{max_parallel_tasks} parallel tasks running) "
+ f"and backlog is full"
+ ),
)
raise HTTPException(
status_code=429,
@@ -970,7 +1036,23 @@ async def execute_parallel_task(
f"Agent '{name}' is at capacity ({max_parallel_tasks} parallel tasks) "
f"and its backlog is full. Try again later."
),
+ ) from e
+
+ if cap_result.state == "queued_persistent":
+ logger.info(
+ f"[Task Async] Agent '{name}' at capacity — execution {execution_id} queued to backlog"
)
+ return {
+ "status": "queued",
+ "execution_id": execution_id,
+ "agent_name": name,
+ "message": (
+ f"Agent at capacity; task queued. Poll GET "
+ f"/api/agents/{name}/executions/{execution_id} for results."
+ ),
+ "async_mode": True,
+ }
+ slot_acquired = True # admitted — preserved for downstream finally semantics
# Issue #279: done callback surfaces unhandled BG task exceptions.
def _on_task_done(task: asyncio.Task):
@@ -991,6 +1073,7 @@ def _on_task_done(task: asyncio.Task):
subscription_id=_task_subscription_id,
is_self_task=is_self_task,
self_task_activity_id=self_task_activity_id,
+ images=_image_data,
)
)
bg_task.add_done_callback(_on_task_done)
@@ -1004,63 +1087,66 @@ def _on_task_done(task: asyncio.Task):
"async_mode": True,
}
- # ---- Sync mode: pre-acquire slot to mirror async branch (issue #498).
+ # ---- Sync mode: pre-acquire capacity to mirror async branch (issue #498).
# On success, delegate to TaskExecutionService with slot_already_held=True
# so service finally still releases. On at-capacity, spill to the same
# backlog the async path uses and long-poll on this connection until the
# execution reaches a terminal status.
- sync_slot_service = get_slot_service()
+ # CAPACITY-CONSOLIDATE (#428): single CapacityManager.acquire call.
+ capacity = get_capacity_manager()
sync_max_parallel_tasks = db.get_max_parallel_tasks(name)
sync_effective_timeout = request.timeout_seconds
if sync_effective_timeout is None:
sync_effective_timeout = db.get_execution_timeout(name)
- sync_slot_acquired = await sync_slot_service.acquire_slot(
- agent_name=name,
- execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
- max_parallel_tasks=sync_max_parallel_tasks,
- message_preview=request.message[:100] if request.message else "",
- timeout_seconds=sync_effective_timeout,
- )
-
- if not sync_slot_acquired:
- # Issue #498: spill sync calls to the SAME backlog the async path uses
- # (BACKLOG-001), then await on the open HTTP connection. The drain
- # callback fires _run_async_task_with_persistence; that helper signals
- # _sync_waiters from its finally so we wake immediately on terminal.
- from services.backlog_service import get_backlog_service
- sync_backlog = get_backlog_service()
- sync_enqueued = await sync_backlog.enqueue(
+ try:
+ sync_cap_result = await capacity.acquire(
agent_name=name,
- execution_id=execution_id,
- request=request,
- effective_timeout=sync_effective_timeout,
- user_id=current_user.id,
- user_email=current_user.email or current_user.username,
- subscription_id=_task_subscription_id,
- x_source_agent=x_source_agent,
- x_mcp_key_id=x_mcp_key_id,
- x_mcp_key_name=x_mcp_key_name,
- triggered_by=triggered_by,
- collaboration_activity_id=collaboration_activity_id,
- is_self_task=is_self_task,
- self_task_activity_id=self_task_activity_id,
+ execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
+ max_concurrent=sync_max_parallel_tasks,
+ message_preview=request.message[:100] if request.message else "",
+ timeout_seconds=sync_effective_timeout,
+ overflow_policy="queue_persistent",
+ overflow_payload=PersistentTaskPayload(
+ request=request,
+ effective_timeout=sync_effective_timeout,
+ user_id=current_user.id,
+ user_email=current_user.email or current_user.username,
+ subscription_id=_task_subscription_id,
+ x_source_agent=x_source_agent,
+ x_mcp_key_id=x_mcp_key_id,
+ x_mcp_key_name=x_mcp_key_name,
+ triggered_by=triggered_by,
+ collaboration_activity_id=collaboration_activity_id,
+ is_self_task=is_self_task,
+ self_task_activity_id=self_task_activity_id,
+ ),
)
- if not sync_enqueued:
- # Backlog ALSO full → preserve existing terminal-failure semantics.
- if execution_id:
- db.update_execution_status(
- execution_id=execution_id,
- status=TaskExecutionStatus.FAILED,
- error=f"Agent at capacity ({sync_max_parallel_tasks}/{sync_max_parallel_tasks} parallel tasks running) and backlog is full",
- )
- raise HTTPException(
- status_code=429,
- detail=(
- f"Agent '{name}' is at capacity ({sync_max_parallel_tasks} parallel tasks) "
- f"and its backlog is full. Try again later."
+ except CapacityFull as e:
+ # Both capacity AND backlog are full → preserve terminal-failure semantics.
+ if execution_id:
+ db.update_execution_status(
+ execution_id=execution_id,
+ status=TaskExecutionStatus.FAILED,
+ error=(
+ f"Agent at capacity ({sync_max_parallel_tasks}/{sync_max_parallel_tasks} parallel tasks running) "
+ f"and backlog is full"
),
)
+ raise HTTPException(
+ status_code=429,
+ detail=(
+ f"Agent '{name}' is at capacity ({sync_max_parallel_tasks} parallel tasks) "
+ f"and its backlog is full. Try again later."
+ ),
+ ) from e
+
+ sync_slot_acquired = sync_cap_result.state == "admitted"
+
+ if not sync_slot_acquired:
+ # Spilled to backlog — long-poll on the open HTTP connection. The drain
+ # callback fires _run_async_task_with_persistence; that helper signals
+ # _sync_waiters from its finally so we wake immediately on terminal.
# Long-poll cap: queue wait + execution time, both bounded individually
# by effective_timeout via slot TTL and TaskExecutionService internals.
@@ -1159,6 +1245,7 @@ def _on_task_done(task: asyncio.Task):
system_prompt=request.system_prompt,
execution_id=execution_id,
slot_already_held=True, # Issue #498: router pre-acquired
+ images=_image_data,
)
# Complete collaboration activity based on result
@@ -1734,19 +1821,19 @@ async def terminate_agent_execution(
detail=result.get("detail", "Termination failed")
)
- # Clear queue state and release capacity slot if termination succeeded
+ # Clear capacity state if termination succeeded.
+ # CAPACITY-CONSOLIDATE (#428): single force_release covers both the
+ # SlotService N-ary counter and the in-memory overflow queue.
if result.get("status") in ["terminated", "already_finished"]:
- queue = get_execution_queue()
- await queue.force_release(name)
- logger.info(f"[Terminate] Released queue for agent '{name}' after terminating execution {execution_id}")
-
- # Issue #98: Also release any capacity slot held by this execution
try:
- term_slot_service = get_slot_service()
- await term_slot_service.release_slot(name, execution_id)
- logger.info(f"[Terminate] Released capacity slot for agent '{name}' execution {execution_id}")
+ capacity = get_capacity_manager()
+ fr = await capacity.force_release(name)
+ logger.info(
+ f"[Terminate] Force-released capacity for agent '{name}' "
+ f"(was_running={fr.was_running}, slots_cleared={fr.slots_cleared})"
+ )
except Exception as e:
- logger.warning(f"[Terminate] Failed to release slot for {name}: {e}")
+ logger.warning(f"[Terminate] Failed to force-release capacity for {name}: {e}")
# Update database execution record if provided
if task_execution_id:
diff --git a/src/backend/routers/credentials.py b/src/backend/routers/credentials.py
index c331b677e..9749b0b37 100644
--- a/src/backend/routers/credentials.py
+++ b/src/backend/routers/credentials.py
@@ -22,6 +22,7 @@
from config import OAUTH_CONFIGS, BACKEND_URL
from dependencies import get_current_user, require_admin, get_authorized_agent_by_name, get_owned_agent_by_name
from services.docker_service import get_agent_container, get_agent_status_from_container
+from services.mcp_validator import validate_mcp_config, McpValidationError
from services.platform_audit_service import platform_audit_service, AuditEventType
logger = logging.getLogger(__name__)
@@ -30,7 +31,15 @@
# Allowlist of credential file paths that can be injected into agents.
# Blocks arbitrary file writes (pentest 3.2.6 / #183).
-ALLOWED_CREDENTIAL_PATHS = {".env", ".mcp.json", ".mcp.json.template", ".credentials.enc"}
+#
+# `.mcp.json` is allowlisted but its CONTENT is structure-validated by
+# `services.mcp_validator.validate_mcp_config` before the inject is
+# accepted (#598, Layer 2 of AISEC-C2 closure). Bare allowlisting is
+# sufficient for `.env` and `.credentials.enc` because their content is
+# either KEY=VALUE pairs or AES-GCM ciphertext — neither defines
+# executable behavior. `.mcp.json.template` remains BLOCKED because the
+# envsubst flow it feeds doesn't sanitize attacker-controlled JSON.
+ALLOWED_CREDENTIAL_PATHS = {".env", ".credentials.enc", ".mcp.json"}
# ============================================================================
@@ -200,6 +209,22 @@ async def inject_credentials(
f"Allowed: {sorted(ALLOWED_CREDENTIAL_PATHS)}"
)
+ # AISEC-C2 / #590 Layer 2 (#598): structure-validate .mcp.json content
+ # before forwarding to the agent. Closes the RCE-by-config bypass while
+ # restoring the legitimate post-deploy MCP server editing flow.
+ if ".mcp.json" in request_body.files:
+ try:
+ validate_mcp_config(request_body.files[".mcp.json"])
+ except McpValidationError as e:
+ logger.warning(
+ f".mcp.json injection blocked by validator for agent {agent_name} "
+ f"by user {current_user.email}: {e}"
+ )
+ raise HTTPException(
+ status_code=400,
+ detail=f"Invalid .mcp.json: {e}",
+ )
+
try:
async with httpx.AsyncClient() as client:
response = await client.post(
diff --git a/src/backend/routers/docs.py b/src/backend/routers/docs.py
index 44d75b21b..cd7019c70 100644
--- a/src/backend/routers/docs.py
+++ b/src/backend/routers/docs.py
@@ -6,11 +6,14 @@
- Serving markdown content
- Documentation index
"""
-from fastapi import APIRouter, HTTPException
+from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import FileResponse, JSONResponse
from pathlib import Path
import json
+from models import User
+from dependencies import get_current_user
+
router = APIRouter(prefix="/api/docs", tags=["Documentation"])
# Documentation directory - relative to trinity root
@@ -31,7 +34,7 @@ def get_docs_dir() -> Path:
@router.get("/index")
-async def get_docs_index():
+async def get_docs_index(current_user: User = Depends(get_current_user)):
"""Get documentation index/navigation structure."""
docs_dir = get_docs_dir()
if not docs_dir:
@@ -49,7 +52,10 @@ async def get_docs_index():
@router.get("/content/{slug:path}")
-async def get_doc_content(slug: str):
+async def get_doc_content(
+ slug: str,
+ current_user: User = Depends(get_current_user),
+):
"""Get documentation content by slug (supports .md and .json files)."""
docs_dir = get_docs_dir()
if not docs_dir:
@@ -100,7 +106,7 @@ async def get_doc_content(slug: str):
@router.get("/list")
-async def list_docs():
+async def list_docs(current_user: User = Depends(get_current_user)):
"""List all available documentation files."""
docs_dir = get_docs_dir()
if not docs_dir:
diff --git a/src/backend/routers/files.py b/src/backend/routers/files.py
new file mode 100644
index 000000000..406498856
--- /dev/null
+++ b/src/backend/routers/files.py
@@ -0,0 +1,237 @@
+"""
+Public download endpoint for outbound agent file sharing (FILES-001 Step 4).
+
+Resolves the token-scoped URLs minted by POST /api/internal/agent-files/share.
+Unauthenticated — the 192-bit `sig` token IS the auth credential, minted at
+share time and known only to the recipient.
+
+Error matrix:
+- 404 — file_id does not exist
+- 401 — download_token missing or wrong (constant-time compare)
+- 410 — revoked or expired
+- 429 — IP rate limit
+- 500 — storage file missing on disk
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import secrets
+from datetime import datetime, timezone
+from typing import Optional
+from urllib.parse import quote
+
+from fastapi import APIRouter, HTTPException, Request, Response
+from fastapi.responses import StreamingResponse
+
+from database import db
+from services.agent_shared_files_service import STORAGE_ROOT
+from services.platform_audit_service import AuditEventType, platform_audit_service
+from routers.auth import get_redis_client
+from routers.public import _get_client_ip
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/files", tags=["files"])
+
+_CHUNK_SIZE = 64 * 1024
+
+# C5: File-download rate limit has its OWN bucket so heavy download
+# traffic can't exhaust the shared public_link_lookups bucket used by
+# /api/public/* endpoints. Same 60/min per IP default; configurable via
+# redis key prefix.
+_DOWNLOAD_RATE_LIMIT = 60 # requests per window per IP
+_DOWNLOAD_RATE_WINDOW = 60 # window in seconds
+
+
+def _check_file_download_rate_limit(client_ip: str) -> None:
+ """
+ Rate-limit GETs to /api/files/{id} per client IP.
+
+ Fails open if Redis is unavailable (logs a warning) — same convention
+ as public-link rate limiting. Uses a dedicated `file_downloads:{ip}`
+ bucket so it can't starve other public endpoints.
+ """
+ r = get_redis_client()
+ if r is None:
+ logger.warning("File download rate limiting unavailable — Redis not connected")
+ return
+ key = f"file_downloads:{client_ip}"
+ try:
+ attempts = r.get(key)
+ if attempts and int(attempts) >= _DOWNLOAD_RATE_LIMIT:
+ raise HTTPException(status_code=429, detail="Too many requests. Please try again later.")
+ pipe = r.pipeline()
+ pipe.incr(key)
+ pipe.expire(key, _DOWNLOAD_RATE_WINDOW)
+ pipe.execute()
+ except HTTPException:
+ raise
+ except Exception as e:
+ logger.warning(f"File download rate limit check failed: {e}")
+
+
+def _iter_file(path: str):
+ """Stream a file in chunks without loading it into memory."""
+ with open(path, "rb") as fh:
+ while True:
+ chunk = fh.read(_CHUNK_SIZE)
+ if not chunk:
+ break
+ yield chunk
+
+
+def _format_disposition(filename: str) -> str:
+ """
+ RFC 6266 Content-Disposition with UTF-8 fallback.
+ Always `attachment` — never inline (defense against XSS via agent-uploaded HTML).
+ """
+ ascii_name = filename.encode("ascii", "replace").decode("ascii")
+ safe_ascii = ascii_name.replace('"', "").replace("\\", "")
+ utf8_encoded = quote(filename, safe="")
+ return f'attachment; filename="{safe_ascii}"; filename*=UTF-8\'\'{utf8_encoded}'
+
+
+def _parse_expires(value: str) -> datetime:
+ """Parse the ISO timestamp stored in expires_at, guaranteeing tz-aware."""
+ dt = datetime.fromisoformat(value)
+ if dt.tzinfo is None:
+ dt = dt.replace(tzinfo=timezone.utc)
+ return dt
+
+
+async def _validate_download_request(
+ file_id: str,
+ request: Request,
+ sig: Optional[str],
+ download_token_alias: Optional[str],
+) -> tuple:
+ """
+ Shared validation for GET and HEAD requests.
+
+ Returns ``(row, storage_path, headers, mime_type, client_ip)`` on success.
+ Raises HTTPException on any failure (401 / 404 / 410 / 500 / 429).
+ """
+ client_ip = _get_client_ip(request)
+ _check_file_download_rate_limit(client_ip) # 429 on limit (C5 — dedicated bucket)
+
+ # Accept either `sig` (preferred) or `download_token` (legacy alias)
+ token = sig or download_token_alias
+ if not token:
+ raise HTTPException(status_code=401, detail="sig required")
+
+ row = db.get_agent_shared_file(file_id)
+ if not row:
+ raise HTTPException(status_code=404, detail="not found")
+
+ # Constant-time compare to prevent timing oracles
+ if not secrets.compare_digest(token, row["download_token"]):
+ raise HTTPException(status_code=401, detail="invalid download_token")
+
+ if row["revoked_at"]:
+ raise HTTPException(status_code=410, detail="revoked")
+
+ try:
+ expires = _parse_expires(row["expires_at"])
+ except ValueError:
+ logger.error("[files] malformed expires_at on file_id=%s: %r", file_id, row.get("expires_at"))
+ raise HTTPException(status_code=500, detail="storage error")
+ if datetime.now(timezone.utc) > expires:
+ raise HTTPException(status_code=410, detail="expired")
+
+ storage_path = os.path.join(STORAGE_ROOT, row["stored_filename"])
+ if not os.path.exists(storage_path):
+ logger.error(
+ "[files] orphan DB row for file_id=%s — stored_filename=%s missing on disk",
+ file_id, row["stored_filename"],
+ )
+ raise HTTPException(status_code=500, detail="storage error")
+
+ headers = {
+ "Content-Disposition": _format_disposition(row["filename"]),
+ "X-Content-Type-Options": "nosniff",
+ "Cache-Control": "private, no-store",
+ "Content-Length": str(row["size_bytes"]),
+ }
+ mime_type = row["mime_type"] or "application/octet-stream"
+ return row, storage_path, headers, mime_type, client_ip
+
+
+@router.get("/{file_id}")
+async def download_shared_file(
+ file_id: str,
+ request: Request,
+ sig: Optional[str] = None,
+ download_token: Optional[str] = None,
+):
+ """
+ Serve a file previously registered via POST /api/internal/agent-files/share.
+
+ Query parameters:
+ - sig (required): 192-bit token minted at share time, sole auth credential.
+
+ `download_token` is accepted as a legacy alias but deprecated —
+ Trinity's credential sanitizer redacts `...TOKEN...=value` query
+ pairs from agent responses, stripping the token in transit. New
+ URLs emit `?sig=...`.
+ """
+ row, storage_path, headers, mime_type, client_ip = await _validate_download_request(
+ file_id, request, sig, download_token,
+ )
+ agent_name = row["agent_name"]
+
+ # Counters — best-effort
+ try:
+ db.mark_shared_file_downloaded(file_id)
+ except Exception as e: # pragma: no cover
+ logger.warning("[files] failed to mark_downloaded for %s: %s", file_id, e)
+
+ # Audit — best-effort
+ try:
+ await platform_audit_service.log(
+ event_type=AuditEventType.EXECUTION,
+ event_action="file_share_download",
+ source="public",
+ actor_ip=client_ip,
+ target_type="agent",
+ target_id=agent_name,
+ details={
+ "file_id": file_id,
+ "filename": row["filename"],
+ "size_bytes": row["size_bytes"],
+ "mime_type": row["mime_type"],
+ "user_agent": (request.headers.get("user-agent") or "")[:200],
+ },
+ endpoint=str(request.url.path),
+ )
+ except Exception as e: # pragma: no cover
+ logger.warning("[files] audit log failed for %s: %s", file_id, e)
+
+ return StreamingResponse(
+ _iter_file(storage_path),
+ media_type=mime_type,
+ headers=headers,
+ )
+
+
+@router.head("/{file_id}")
+async def head_shared_file(
+ file_id: str,
+ request: Request,
+ sig: Optional[str] = None,
+ download_token: Optional[str] = None,
+):
+ """
+ HEAD handler for link previewers / CDNs that probe before GET.
+
+ Runs the same validation as GET (rate limit, token, expiry, revoke,
+ storage presence) and returns the same headers — but no body, no
+ download counter bump, no audit row. Follows RFC 7231 §4.3.2:
+ HEAD is identical to GET except the server MUST NOT return a
+ message-body.
+ """
+ _row, _storage_path, headers, mime_type, _client_ip = await _validate_download_request(
+ file_id, request, sig, download_token,
+ )
+ return Response(status_code=200, headers=headers, media_type=mime_type)
diff --git a/src/backend/routers/git.py b/src/backend/routers/git.py
index 720a005b2..8ac17711c 100644
--- a/src/backend/routers/git.py
+++ b/src/backend/routers/git.py
@@ -347,7 +347,7 @@ async def initialize_github_sync(
else:
# Database record exists but git not initialized - clean up orphaned record
print(f"Warning: Found orphaned git config for {agent_name}. Cleaning up and allowing re-initialization.")
- db.execute_query("DELETE FROM agent_git_config WHERE agent_name = ?", (agent_name,))
+ db.delete_git_config(agent_name)
# Get GitHub PAT from settings
github_pat = get_github_pat()
diff --git a/src/backend/routers/internal.py b/src/backend/routers/internal.py
index 5a2a0f37f..bfa174a27 100644
--- a/src/backend/routers/internal.py
+++ b/src/backend/routers/internal.py
@@ -16,7 +16,7 @@
from typing import Optional, Dict, List
import logging
-from models import ActivityState, ActivityType
+from models import ActivityState, ActivityType, ShareFileRequest, ShareFileResponse
from services.activity_service import activity_service
from services.task_execution_service import get_task_execution_service
from services.platform_audit_service import platform_audit_service, AuditEventType
@@ -59,6 +59,27 @@ async def internal_health():
return {"status": "ok"}
+# ---------------------------------------------------------------------------
+# Scheduler pre-check (#454, SCHED-COND-001)
+# ---------------------------------------------------------------------------
+
+
+@router.post("/agents/{agent_name}/pre-check")
+async def internal_agent_pre_check(agent_name: str):
+ """Run the agent's optional pre-check hook (SCHED-COND-001 / #454).
+
+ Thin passthrough — all logic lives in
+ ``services/pre_check_service.py`` (Invariant #1: Router → Service
+ → DB). See that module for the full contract.
+ """
+ from services.pre_check_service import run_pre_check, AgentNotFound
+
+ try:
+ return await run_pre_check(agent_name)
+ except AgentNotFound:
+ raise HTTPException(status_code=404, detail="Agent not found")
+
+
@router.get("/agents/{agent_name}/sync-health-status")
async def internal_agent_sync_health(agent_name: str):
"""#389: lightweight read used by the dedicated scheduler before dispatching.
@@ -501,3 +522,29 @@ async def log_audit_entry(request: InternalAuditRequest):
except Exception as e:
logger.error(f"Failed to log audit entry: {e}")
raise HTTPException(status_code=500, detail=str(e))
+
+
+# =============================================================================
+# Agent Shared Files (outbound — FILES-001 Step 3)
+# =============================================================================
+
+@router.post("/agent-files/share", response_model=ShareFileResponse)
+async def agent_files_share(payload: ShareFileRequest):
+ """
+ Mint a public download URL for a file the agent wrote to its publish dir.
+
+ Authentication: X-Internal-Secret (already enforced by router dependency).
+ Agent identity: carried by `payload.agent_name`. The agent server is
+ responsible for passing its own name here — same trust model as
+ /internal/execute-task (forging requires the internal secret).
+ """
+ from services.agent_shared_files_service import create_share
+
+ result = await create_share(
+ agent_name=payload.agent_name,
+ filename=payload.filename,
+ display_name=payload.display_name,
+ expires_in=payload.expires_in,
+ created_by=payload.agent_name,
+ )
+ return ShareFileResponse(**result)
diff --git a/src/backend/routers/public.py b/src/backend/routers/public.py
index d47134346..fa68f301e 100644
--- a/src/backend/routers/public.py
+++ b/src/backend/routers/public.py
@@ -12,7 +12,7 @@
import httpx
import logging
from typing import Optional, List
-from fastapi import APIRouter, HTTPException, Request
+from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
@@ -26,12 +26,15 @@
PublicChatResponse,
PublicChatMessage
)
+from dependencies import get_current_user
+from models import User
from routers.auth import check_login_rate_limit, record_login_attempt, get_redis_client
from services.docker_service import get_agent_container
from services.email_service import email_service
from services.task_execution_service import get_task_execution_service
from services.platform_prompt_service import format_user_memory_block
from services.settings_service import get_anthropic_api_key
+from services.upload_service import process_file_uploads, decode_web_file, WEB_MAX_FILES, WEB_MAX_FILE_SIZE, WEB_MAX_IMAGE_SIZE, WEB_MAX_TOTAL_IMAGE_SIZE
class PublicChatHistoryResponse(BaseModel):
@@ -487,6 +490,42 @@ async def public_chat(
detail="Agent is not available. Please try again later."
)
+ # (#364) File upload processing for public chat.
+ # Rate-limited by existing IP check above. Files must be processed
+ # synchronously before the async/sync fork so bytes are in the container.
+ _pub_image_data: list = []
+ _pub_file_descs: list = []
+ if chat_request.files:
+ uploader = verified_email or f"anonymous ({client_ip})"
+ raw_files = [
+ {
+ "name": f.name,
+ "mimetype": f.mimetype,
+ "size": f.size,
+ "data": decode_web_file(f.dict()),
+ "id": f"f{i}",
+ }
+ for i, f in enumerate(chat_request.files)
+ ]
+ file_descs, _, all_writes_failed, _pub_image_data = await process_file_uploads(
+ raw_files=raw_files,
+ agent_name=agent_name,
+ container=container,
+ session_id=session_identifier,
+ uploader=uploader,
+ source="public",
+ max_files=WEB_MAX_FILES,
+ max_file_size=WEB_MAX_FILE_SIZE,
+ max_image_size=WEB_MAX_IMAGE_SIZE,
+ max_total_image_size=WEB_MAX_TOTAL_IMAGE_SIZE,
+ )
+ if all_writes_failed:
+ raise HTTPException(
+ status_code=502,
+ detail="File upload failed: could not write to agent workspace."
+ )
+ _pub_file_descs = file_descs
+
# Get or create chat session
chat_session = db.get_or_create_public_chat_session(
link_id=link["id"],
@@ -494,7 +533,19 @@ async def public_chat(
identifier_type=identifier_type
)
- # Store user message
+ # Build context from prior history before storing the new user message.
+ # Must happen first — storing the user message then reading it back would
+ # include the current message in both "Previous conversation:" and
+ # "Current message:", sending it to the agent twice on every turn.
+ context_prompt = db.build_public_chat_context(
+ session_id=chat_session.id,
+ new_message=chat_request.message,
+ max_turns=10
+ )
+ if _pub_file_descs:
+ context_prompt = f"{context_prompt}\n\n" + "\n".join(_pub_file_descs)
+
+ # Store user message (after context is built so it doesn't appear twice)
db.add_public_chat_message(
session_id=chat_session.id,
role="user",
@@ -508,13 +559,6 @@ async def public_chat(
ip_address=client_ip
)
- # Build context-enriched prompt with conversation history
- context_prompt = db.build_public_chat_context(
- session_id=chat_session.id,
- new_message=chat_request.message,
- max_turns=10
- )
-
# MEM-001: Fetch per-user memory for email-verified sessions and inject into system prompt
memory_system_prompt = None
if identifier_type == "email" and verified_email:
@@ -550,6 +594,7 @@ async def public_chat(
identifier_type=identifier_type,
verified_email=verified_email,
memory_system_prompt=memory_system_prompt,
+ images=_pub_image_data,
))
return {
@@ -568,6 +613,7 @@ async def public_chat(
source_user_email=source_email,
timeout_seconds=900,
system_prompt=memory_system_prompt,
+ images=_pub_image_data,
)
if result.status == "failed":
@@ -866,6 +912,7 @@ async def _execute_public_chat_background(
identifier_type: str,
verified_email: str = None,
memory_system_prompt: str = None,
+ images: list = None,
):
"""
Background task for async public chat execution.
@@ -884,6 +931,7 @@ async def _execute_public_chat_background(
timeout_seconds=900,
execution_id=execution_id,
system_prompt=memory_system_prompt,
+ images=images or [],
)
if result.status == "success" and result.response:
@@ -1072,3 +1120,74 @@ async def public_execution_status(
"response": execution.response if execution.status in ("success", "failed") else None,
"error": execution.error if execution.status == "failed" else None,
}
+
+
+@router.get("/sessions/{token}")
+async def get_public_link_sessions(
+ token: str,
+ limit: int = 20,
+ current_user: User = Depends(get_current_user)
+):
+ """
+ List the authenticated user's chat sessions for the agent behind this public link.
+
+ Requires JWT. Returns the caller's own sessions ordered most-recent first,
+ capped at `limit` (default 20). Does not require agent sharing — the public
+ link token acts as the access credential for this read-only history view.
+ """
+ link = _validate_public_link(token)
+ agent_name = link["agent_name"]
+
+ sessions = db.get_agent_chat_sessions(
+ agent_name=agent_name,
+ user_id=current_user.id,
+ )
+ page = sessions[:limit]
+
+ result = []
+ for s in page:
+ entry = s.model_dump()
+ # Attach a preview from the most recent message in the session
+ recent = db.get_chat_messages(s.id, limit=1)
+ entry["preview"] = recent[0].content[:120] if recent else None
+ result.append(entry)
+
+ return {
+ "session_count": len(result),
+ "sessions": result,
+ }
+
+
+@router.get("/sessions/{token}/{session_id}")
+async def get_public_link_session_detail(
+ token: str,
+ session_id: str,
+ limit: int = 100,
+ current_user: User = Depends(get_current_user)
+):
+ """
+ Get messages for a specific chat session via a public link token.
+
+ Requires JWT. The session must belong to the authenticated user and to
+ the agent referenced by the public link token.
+ """
+ link = _validate_public_link(token)
+ agent_name = link["agent_name"]
+
+ session = db.get_chat_session(session_id)
+ if not session:
+ raise HTTPException(status_code=404, detail="Session not found")
+
+ if session.agent_name != agent_name:
+ raise HTTPException(status_code=403, detail="Session does not belong to this agent")
+
+ if session.user_id != current_user.id:
+ raise HTTPException(status_code=403, detail="You don't have access to this session")
+
+ messages = db.get_chat_messages(session_id, limit=limit)
+
+ return {
+ "session": session.model_dump(),
+ "message_count": len(messages),
+ "messages": [m.model_dump() for m in messages],
+ }
diff --git a/src/backend/routers/slack.py b/src/backend/routers/slack.py
index 5eeed22cc..4132c8b78 100644
--- a/src/backend/routers/slack.py
+++ b/src/backend/routers/slack.py
@@ -405,6 +405,10 @@ async def get_agent_slack_channel(
for ws in workspaces:
binding = db.get_slack_channel_for_agent(ws["team_id"], name)
if binding:
+ # Count agents in the workspace so the UI can decide whether
+ # to allow unbinding — the DM-default agent cannot be unbound
+ # while other agents are still bound (#584).
+ workspace_agents = db.get_slack_agents_for_workspace(ws["team_id"])
return {
"bound": True,
"channel_name": binding["slack_channel_name"],
@@ -412,6 +416,7 @@ async def get_agent_slack_channel(
"workspace_team_id": ws["team_id"],
"workspace_name": ws["team_name"],
"is_dm_default": binding.get("is_dm_default", False),
+ "workspace_agent_count": len(workspace_agents),
"created_at": binding.get("created_at"),
}
@@ -490,14 +495,121 @@ async def delete_agent_slack_channel(
name: str,
current_user: User = Depends(get_current_user)
):
- """Unbind an agent from its Slack channel."""
+ """Unbind an agent from its Slack channel.
+
+ Refuses to unbind the workspace's current DM-default agent while any
+ other agents are still bound (#584). The owner must promote a different
+ agent first via ``PUT /api/agents/{name}/slack/channel/dm-default``.
+ When the agent is the only one bound, unbind is allowed — the workspace
+ ends up with no Slack agents, which is a clean cascade.
+ """
if not db.can_user_share_agent(current_user.username, name):
raise HTTPException(status_code=403, detail="Only owners can manage Slack channels")
workspaces = db.get_all_slack_workspaces()
for ws in workspaces:
+ binding = db.get_slack_channel_for_agent(ws["team_id"], name)
+ if not binding:
+ continue
+
+ # Refuse to drop the DM default while siblings remain.
+ if binding.get("is_dm_default"):
+ workspace_agents = db.get_slack_agents_for_workspace(ws["team_id"])
+ if len(workspace_agents) > 1:
+ raise HTTPException(
+ status_code=409,
+ detail=(
+ "Cannot unbind the DM-default agent while other agents "
+ "are bound to this workspace. Set another agent as DM "
+ "default first (PUT /api/agents/{other}/slack/channel/"
+ "dm-default) and try again."
+ ),
+ )
+
if db.unbind_slack_agent(ws["team_id"], name):
logger.info(f"Agent {name} unbound from Slack in workspace {ws['team_name']}")
return {"unbound": True, "workspace_name": ws["team_name"]}
raise HTTPException(status_code=404, detail="Agent is not bound to any Slack channel")
+
+
+@auth_router.put("/api/agents/{name}/slack/channel/dm-default")
+async def set_agent_as_slack_dm_default(
+ name: str,
+ current_user: User = Depends(get_current_user)
+):
+ """Make this agent the DM-default for its Slack workspace.
+
+ DMs to the bot (no channel context, no @mention) route to whichever
+ agent in the workspace is flagged ``is_dm_default=1``. Until #584 the
+ flag was only auto-set for the first agent ever connected and had no
+ setter, so workspaces with multiple agents were stuck. This endpoint
+ flips it; ``unbind`` auto-promotes the oldest remaining agent so the
+ workspace is never left with zero defaults.
+ """
+ if not db.can_user_share_agent(current_user.username, name):
+ raise HTTPException(status_code=403, detail="Only owners can manage Slack channels")
+
+ # Find the workspace where this agent is bound. There should be at
+ # most one — agents are bound 1:1 per workspace today.
+ workspace = None
+ for ws in db.get_all_slack_workspaces():
+ if db.get_slack_channel_for_agent(ws["team_id"], name):
+ workspace = ws
+ break
+
+ if not workspace:
+ raise HTTPException(
+ status_code=404,
+ detail="Agent is not bound to any Slack channel",
+ )
+
+ team_id = workspace["team_id"]
+ previous = db.get_slack_dm_default_agent(team_id)
+ if previous == name:
+ # Idempotent — already the default.
+ return {
+ "status": "unchanged",
+ "team_id": team_id,
+ "workspace_name": workspace.get("team_name"),
+ "previous": previous,
+ "new_default": name,
+ }
+
+ if not db.set_slack_dm_default(team_id, name):
+ # set_dm_default returns False only if the agent isn't bound — we
+ # already verified that above, so this is a real "row vanished"
+ # race. Surface as 404.
+ raise HTTPException(status_code=404, detail="Agent binding not found")
+
+ logger.info(
+ "Slack DM default for workspace %s changed: %s → %s (by %s)",
+ workspace.get("team_name"), previous, name, current_user.username,
+ )
+
+ # Audit
+ try:
+ await platform_audit_service.log(
+ event_type=AuditEventType.AGENT_LIFECYCLE,
+ event_action="slack_dm_default_changed",
+ source="api",
+ actor_user=current_user,
+ target_type="agent",
+ target_id=name,
+ details={
+ "team_id": team_id,
+ "workspace_name": workspace.get("team_name"),
+ "previous": previous,
+ "new_default": name,
+ },
+ )
+ except Exception as e: # pragma: no cover
+ logger.warning("Failed to audit slack_dm_default_changed: %s", e)
+
+ return {
+ "status": "updated",
+ "team_id": team_id,
+ "workspace_name": workspace.get("team_name"),
+ "previous": previous,
+ "new_default": name,
+ }
diff --git a/src/backend/routers/subscriptions.py b/src/backend/routers/subscriptions.py
index 72c04d07d..4b673ddeb 100644
--- a/src/backend/routers/subscriptions.py
+++ b/src/backend/routers/subscriptions.py
@@ -377,7 +377,10 @@ async def get_auto_switch_setting(
):
"""Get the auto-switch subscriptions setting."""
require_admin(current_user)
- enabled = db.get_setting_value("auto_switch_subscriptions", default="false") == "true"
+ # #441: default flipped to "true" (opt-out). Must match the default in
+ # services/subscription_auto_switch.handle_subscription_failure so the UI
+ # toggle and the runtime gate read the same value on a clean install.
+ enabled = db.get_setting_value("auto_switch_subscriptions", default="true") == "true"
return {"enabled": enabled}
diff --git a/src/backend/routers/voice.py b/src/backend/routers/voice.py
index 96f05247d..3c9c3807b 100644
--- a/src/backend/routers/voice.py
+++ b/src/backend/routers/voice.py
@@ -5,7 +5,7 @@
Endpoints:
POST /api/agents/{name}/voice/start - Initialize voice session
POST /api/agents/{name}/voice/stop - End voice session and save transcript
- WS /ws/voice/{voice_session_id} - Audio streaming bridge
+ WS /ws/voice/{voice_session_id} - Audio streaming bridge (audio + tool_call + tool_result)
"""
import asyncio
@@ -14,6 +14,7 @@
import logging
from typing import Optional
+import httpx
from fastapi import APIRouter, Depends, HTTPException, WebSocket, WebSocketDisconnect, Query
from pydantic import BaseModel
@@ -23,6 +24,7 @@
from config import GEMINI_API_KEY, VOICE_ENABLED
from services.gemini_voice import voice_service
from services.docker_service import get_agent_container
+from services.platform_audit_service import platform_audit_service, AuditEventType, AuditActorType
logger = logging.getLogger(__name__)
@@ -33,6 +35,7 @@
class VoiceStartRequest(BaseModel):
session_id: Optional[str] = None # Existing chat session to continue
+ voice_name: Optional[str] = None # Gemini voice name (e.g. "Kore", "Puck")
class VoiceStartResponse(BaseModel):
@@ -92,10 +95,8 @@ async def voice_start(
if context_summary:
combined_prompt += f"\n\n## Conversation so far:\n{context_summary}"
- # Get voice name preference
- voice_name = _get_voice_name(name)
+ voice_name = request.voice_name or _get_voice_name(name)
- # Create the voice session
session = voice_service.create_session(
agent_name=name,
chat_session_id=chat_session_id,
@@ -239,6 +240,37 @@ async def on_status(state: str):
except Exception:
pass
+ async def on_tool_call(tool_name: str, args: dict):
+ try:
+ await websocket.send_json({
+ "type": "tool_call",
+ "tool": tool_name,
+ "args": args,
+ })
+ except Exception:
+ pass
+ await platform_audit_service.log(
+ event_type=AuditEventType.EXECUTION,
+ event_action="voice_tool_call",
+ actor_type=AuditActorType.USER,
+ actor_id=str(session.user_id),
+ actor_email=session.user_email,
+ target_type="agent",
+ target_id=session.agent_name,
+ details={"tool": tool_name, "prompt_preview": str(args.get("prompt", ""))[:100]},
+ source="api",
+ )
+
+ async def on_tool_result(tool_name: str, result: str):
+ try:
+ await websocket.send_json({
+ "type": "tool_result",
+ "tool": tool_name,
+ "result_preview": result[:200],
+ })
+ except Exception:
+ pass
+
# Start the Gemini connection in a background task
gemini_task = asyncio.create_task(
voice_service.connect_and_stream(
@@ -246,6 +278,8 @@ async def on_status(state: str):
on_audio_out=on_audio_out,
on_transcript=on_transcript,
on_status=on_status,
+ on_tool_call=on_tool_call,
+ on_tool_result=on_tool_result,
)
)
@@ -294,7 +328,7 @@ async def _get_voice_system_prompt(agent_name: str) -> str:
Priority:
1. Per-agent voice_system_prompt from DB (set via API)
2. voice-agent-system-prompt.md from agent container's working directory
- 3. Error if neither exists
+ 3. Auto-generated from agent template info (description + voice behaviour hints)
"""
# 1. Check DB override
prompt = db.get_voice_system_prompt(agent_name)
@@ -312,18 +346,34 @@ async def _get_voice_system_prompt(agent_name: str) -> str:
user="developer",
)
output = result.output.decode("utf-8").strip() if hasattr(result, 'output') else str(result).strip()
- if output and "No such file" not in output:
+ if output and "No such file" not in output and len(output) > 10:
return output
except Exception as e:
logger.debug(f"Could not read voice-agent-system-prompt.md from {agent_name}: {e}")
- # 3. No prompt found — error
- raise HTTPException(
- status_code=400,
- detail=f"No voice system prompt configured for agent '{agent_name}'. "
- f"Create a 'voice-agent-system-prompt.md' file in the agent's working directory "
- f"(/home/developer/voice-agent-system-prompt.md) or set one via the API."
+ # 3. Auto-generate from template info
+ description = None
+ if container:
+ try:
+ async with httpx.AsyncClient(timeout=5.0) as client:
+ resp = await client.get(f"http://agent-{agent_name}:8000/api/template/info")
+ if resp.status_code == 200:
+ info = resp.json()
+ description = info.get("description") or info.get("summary")
+ except Exception:
+ pass
+
+ display_name = agent_name.replace("-", " ").title()
+ lines = [f"You are {display_name}, an AI agent."]
+ if description:
+ lines.append(f"\n{description}")
+ lines.append(
+ "\n## Voice Behaviour\n"
+ "You are in a voice conversation. Keep responses concise and natural for speech. "
+ "No bullet points, markdown formatting, or code blocks — speak as you would in conversation. "
+ "One idea at a time. Use the run_task tool when you need to look something up or take an action."
)
+ return "\n".join(lines)
def _get_voice_name(agent_name: str) -> str:
diff --git a/src/backend/routers/ws_tickets.py b/src/backend/routers/ws_tickets.py
new file mode 100644
index 000000000..1d374cc49
--- /dev/null
+++ b/src/backend/routers/ws_tickets.py
@@ -0,0 +1,36 @@
+"""
+WebSocket auth ticket endpoint (#550).
+
+Browser clients call ``POST /api/ws/ticket`` (JWT in ``Authorization``
+header) and receive a short-lived opaque ticket they then present on
+``/ws?ticket=``. See ``services/ws_ticket_service`` for the
+single-use Redis-backed exchange.
+"""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Depends, HTTPException
+
+from dependencies import get_current_user
+from models import User
+from services.ws_ticket_service import mint_ticket
+
+router = APIRouter(prefix="/api/ws", tags=["websocket"])
+
+
+@router.post("/ticket")
+async def create_ws_ticket(current_user: User = Depends(get_current_user)) -> dict:
+ """Mint a single-use 30-second WebSocket auth ticket for the caller.
+
+ The ticket is opaque and unrelated to the JWT. Clients must call
+ this endpoint immediately before opening the WebSocket; if the
+ ticket isn't consumed within 30 seconds it expires and a new one
+ must be requested.
+ """
+ try:
+ ticket = mint_ticket(current_user.username, scope="user")
+ except RuntimeError as exc:
+ # Redis down — fail closed. Surface as 503 so the client retries.
+ raise HTTPException(status_code=503, detail=str(exc))
+
+ return {"ticket": ticket, "expires_in": 30}
diff --git a/src/backend/services/agent_service/__init__.py b/src/backend/services/agent_service/__init__.py
index 7c4cb91db..11699907d 100644
--- a/src/backend/services/agent_service/__init__.py
+++ b/src/backend/services/agent_service/__init__.py
@@ -80,6 +80,11 @@
inject_read_only_hooks,
remove_read_only_hooks,
)
+from .file_sharing import (
+ get_file_sharing_status_logic,
+ set_file_sharing_status_logic,
+ check_public_folder_mount_matches,
+)
__all__ = [
# Helpers
@@ -143,4 +148,8 @@
"set_read_only_status_logic",
"inject_read_only_hooks",
"remove_read_only_hooks",
+ # File Sharing (FILES-001 Step 2)
+ "get_file_sharing_status_logic",
+ "set_file_sharing_status_logic",
+ "check_public_folder_mount_matches",
]
diff --git a/src/backend/services/agent_service/crud.py b/src/backend/services/agent_service/crud.py
index e86c530b7..645b44985 100644
--- a/src/backend/services/agent_service/crud.py
+++ b/src/backend/services/agent_service/crud.py
@@ -543,6 +543,36 @@ async def create_agent_internal(
# Source agent hasn't started yet or doesn't have shared volume
pass
+ # FILES-001 Step 2: if file sharing is enabled, create and mount the
+ # per-agent public volume (symmetric to the shared-folders expose flow).
+ if db.get_file_sharing_enabled(config.name):
+ public_volume_name = db.get_public_volume_name(config.name)
+ public_volume_created = False
+ try:
+ await volume_get(public_volume_name)
+ except docker.errors.NotFound:
+ await volume_create(
+ name=public_volume_name,
+ labels={
+ 'trinity.platform': 'agent-public',
+ 'trinity.agent-name': config.name,
+ },
+ )
+ public_volume_created = True
+
+ if public_volume_created:
+ try:
+ await containers_run(
+ 'alpine',
+ command='chown 1000:1000 /public',
+ volumes={public_volume_name: {'bind': '/public', 'mode': 'rw'}},
+ remove=True,
+ )
+ except Exception as e:
+ logger.warning(f"Could not fix public volume ownership: {e}")
+
+ volumes[public_volume_name] = {'bind': db.get_public_mount_path(), 'mode': 'rw'}
+
# Get system-wide full_capabilities setting (not per-agent)
full_capabilities = get_agent_full_capabilities()
diff --git a/src/backend/services/agent_service/file_sharing.py b/src/backend/services/agent_service/file_sharing.py
new file mode 100644
index 000000000..cc4dd20f2
--- /dev/null
+++ b/src/backend/services/agent_service/file_sharing.py
@@ -0,0 +1,137 @@
+"""
+Agent file-sharing (outbound) configuration service.
+
+FILES-001 / amazing-file-outbound Step 2: per-agent opt-in for file sharing.
+
+The toggle flips `agent_ownership.file_sharing_enabled`. Volume mount/unmount
+happens on the next agent restart (the lifecycle flow calls
+`check_public_folder_mount_matches` and triggers container recreation when
+DB config and actual mounts disagree).
+
+Actual file writing/reading is Step 3 territory.
+"""
+
+import logging
+
+from fastapi import HTTPException
+
+from database import db
+from models import User
+from services.docker_service import get_agent_container
+
+logger = logging.getLogger(__name__)
+
+
+# Per-agent storage quota (Step 5 wire-up will enforce; here for status only).
+DEFAULT_QUOTA_BYTES = 500 * 1024 * 1024 # 500 MB
+
+
+def _mount_present(container, destination: str) -> bool:
+ """Whether the container has a mount with the given destination path."""
+ for mount in container.attrs.get("Mounts", []):
+ if mount.get("Destination") == destination:
+ return True
+ return False
+
+
+def check_public_folder_mount_matches(container, agent_name: str) -> bool:
+ """
+ Check if the container's /home/developer/public mount matches the
+ agent's file_sharing_enabled flag.
+
+ Returns True when mounts are consistent with DB, False if recreation
+ is needed. Called from the agent-start lifecycle alongside the other
+ mount-match checks (shared-folders, resources, etc.).
+ """
+ enabled = db.get_file_sharing_enabled(agent_name)
+ mount_path = db.get_public_mount_path()
+ mount_present = _mount_present(container, mount_path)
+
+ if enabled and not mount_present:
+ return False
+ if not enabled and mount_present:
+ return False
+ return True
+
+
+async def get_file_sharing_status_logic(agent_name: str, current_user: User) -> dict:
+ """Return the current file-sharing status for the agent."""
+ if not db.can_user_access_agent(current_user.username, agent_name):
+ raise HTTPException(status_code=403, detail="You don't have permission to access this agent")
+
+ container = get_agent_container(agent_name)
+ if not container:
+ raise HTTPException(status_code=404, detail="Agent not found")
+
+ enabled = db.get_file_sharing_enabled(agent_name)
+ volume_attached = _mount_present(container, db.get_public_mount_path())
+
+ # Restart is required when config and actual mounts disagree.
+ restart_required = enabled != volume_attached
+
+ # Step 1 Note: file_count / total_bytes are placeholders until the
+ # AgentSharedFilesOperations DB layer lands in Step 3. Returning zeros
+ # here lets the UI render the quota bar immediately.
+ file_count = 0
+ total_bytes = 0
+
+ return {
+ "agent_name": agent_name,
+ "enabled": enabled,
+ "volume_attached": volume_attached,
+ "restart_required": restart_required,
+ "file_count": file_count,
+ "total_bytes": total_bytes,
+ "quota_bytes": DEFAULT_QUOTA_BYTES,
+ "status": container.status,
+ }
+
+
+async def set_file_sharing_status_logic(
+ agent_name: str,
+ body: dict,
+ current_user: User,
+) -> dict:
+ """Enable/disable file sharing for an agent (owner-only)."""
+ if not db.can_user_share_agent(current_user.username, agent_name):
+ raise HTTPException(status_code=403, detail="Only the owner can modify file-sharing settings")
+
+ container = get_agent_container(agent_name)
+ if not container:
+ raise HTTPException(status_code=404, detail="Agent not found")
+
+ if db.is_system_agent(agent_name):
+ raise HTTPException(status_code=403, detail="Cannot modify file sharing for the system agent")
+
+ if "enabled" not in body:
+ raise HTTPException(status_code=400, detail="enabled is required")
+ enabled = bool(body["enabled"])
+
+ previous = db.get_file_sharing_enabled(agent_name)
+ db.set_file_sharing_enabled(agent_name, enabled)
+
+ changed = previous != enabled
+ if changed:
+ logger.info(
+ "File sharing %s for agent %s by %s",
+ "enabled" if enabled else "disabled",
+ agent_name,
+ current_user.username,
+ )
+
+ volume_attached = _mount_present(container, db.get_public_mount_path())
+ restart_required = enabled != volume_attached
+
+ return {
+ "status": "updated",
+ "agent_name": agent_name,
+ "enabled": enabled,
+ "restart_required": restart_required,
+ "message": (
+ "File sharing enabled. Restart the agent to mount the volume."
+ if enabled and restart_required
+ else "File sharing disabled. Restart the agent to detach the volume."
+ if not enabled and restart_required
+ else "File sharing updated."
+ ),
+ }
diff --git a/src/backend/services/agent_service/files.py b/src/backend/services/agent_service/files.py
index 5aa7c9544..84abed2ce 100644
--- a/src/backend/services/agent_service/files.py
+++ b/src/backend/services/agent_service/files.py
@@ -3,7 +3,9 @@
Handles file listing, download, preview, and delete for agent workspaces.
"""
+import fnmatch
import logging
+import posixpath
import httpx
from fastapi import HTTPException, Request
@@ -18,6 +20,75 @@
logger = logging.getLogger(__name__)
+# AISEC-C2 / #590 — backend-side deny list for PUT /api/agents/{name}/files.
+# Mirrors the `path_deny` list in docker/base-image/hooks/guardrails-baseline.json
+# so an authenticated owner cannot bypass the agent-server's EDIT_PROTECTED_PATHS
+# check by exploiting any future router/proxy gap. Defense in depth — the
+# agent-server still re-validates server-side. KEEP IN SYNC with both:
+# - docker/base-image/hooks/guardrails-baseline.json::path_deny
+# - docker/base-image/agent_server/routers/files.py::EDIT_PROTECTED_PATHS
+_FILE_WRITE_DENY_PATTERNS = (
+ ".env",
+ ".env.*",
+ ".mcp.json",
+ ".mcp.json.template",
+ ".credentials.enc",
+ ".ssh/*",
+ ".aws/*",
+ ".gcp/*",
+ ".claude/settings.json",
+ ".claude/settings.local.json",
+ ".trinity/*",
+ ".git/*",
+ ".gitignore",
+ "/opt/trinity/*",
+ "/etc/claude-code/*",
+ "/etc/*",
+ "/proc/*",
+ "/sys/*",
+)
+
+
+def _normalize_user_path(raw: str) -> str:
+ """Reduce a user-supplied path to a stable absolute form for deny matching.
+
+ Resolves `..`/`.` segments lexically (no FS access — mirrors the agent-server
+ Path.resolve() guard). Non-absolute paths are anchored at /home/developer to
+ match how the agent-server interprets them (see agent_server/routers/files.py).
+ """
+ if not raw:
+ return ""
+ # posixpath.normpath collapses `..` and `.` lexically; fine for matching.
+ if raw.startswith("/"):
+ return posixpath.normpath(raw)
+ return posixpath.normpath(posixpath.join("/home/developer", raw))
+
+
+def _is_user_writable_path(path: str) -> bool:
+ """Reject writes to credential / runtime-config / Trinity-managed paths.
+
+ Match strategy (mirrors docker/base-image/hooks/file-guardrail.py):
+ - basename match against any pattern (handles `.env`, `.mcp.json` etc.)
+ - full-path glob match (handles `.ssh/*`, `/opt/trinity/*` etc.)
+ - relative-form glob match against /home/developer-relative path
+ """
+ normalized = _normalize_user_path(path)
+ if not normalized:
+ return False
+ basename = posixpath.basename(normalized)
+ rel_to_home = ""
+ if normalized.startswith("/home/developer/"):
+ rel_to_home = normalized[len("/home/developer/"):]
+ for pattern in _FILE_WRITE_DENY_PATTERNS:
+ if fnmatch.fnmatch(basename, pattern):
+ return False
+ if fnmatch.fnmatch(normalized, pattern):
+ return False
+ if rel_to_home and fnmatch.fnmatch(rel_to_home, pattern):
+ return False
+ return True
+
+
async def list_agent_files_logic(
agent_name: str,
path: str,
@@ -267,6 +338,19 @@ async def update_agent_file_logic(
if not db.can_user_access_agent(current_user.username, agent_name):
raise HTTPException(status_code=403, detail="You don't have permission to access this agent")
+ # AISEC-C2 / #590: backend-side deny check before proxying to the agent.
+ # Stops the .mcp.json RCE escalation at the platform boundary; the
+ # agent-server still re-validates as defense in depth.
+ if not _is_user_writable_path(path):
+ logger.warning(
+ "File write blocked at backend deny-list: agent=%s path=%s user=%s",
+ agent_name, path, current_user.username,
+ )
+ raise HTTPException(
+ status_code=403,
+ detail=f"Cannot edit protected path: {path}"
+ )
+
container = get_agent_container(agent_name)
if not container:
raise HTTPException(status_code=404, detail="Agent not found")
diff --git a/src/backend/services/agent_service/lifecycle.py b/src/backend/services/agent_service/lifecycle.py
index ed65ba9cf..4893087a9 100644
--- a/src/backend/services/agent_service/lifecycle.py
+++ b/src/backend/services/agent_service/lifecycle.py
@@ -26,6 +26,7 @@
from services.settings_service import get_anthropic_api_key, get_github_pat, get_agent_full_capabilities
from services.skill_service import skill_service
from .helpers import check_shared_folder_mounts_match, check_api_key_env_matches, check_github_pat_env_matches, check_resource_limits_match, check_full_capabilities_match, check_guardrails_env_matches
+from .file_sharing import check_public_folder_mount_matches
from .read_only import inject_read_only_hooks
logger = logging.getLogger(__name__)
@@ -247,6 +248,7 @@ async def start_agent_internal(agent_name: str) -> dict:
shared_folder_match = await check_shared_folder_mounts_match(container, agent_name)
needs_recreation = (
not shared_folder_match or
+ not check_public_folder_mount_matches(container, agent_name) or
not check_api_key_env_matches(container, agent_name) or
not check_github_pat_env_matches(container, agent_name) or
not check_resource_limits_match(container, agent_name) or
@@ -419,6 +421,9 @@ async def recreate_container_with_updated_config(agent_name: str, old_container,
# Skip shared folder mounts - we'll add the correct ones
if dest == "/home/developer/shared-out" or dest.startswith("/home/developer/shared-in/"):
continue
+ # Skip public mount — re-added below based on current file_sharing_enabled flag.
+ if dest == db.get_public_mount_path():
+ continue
# Keep other mounts
if m.get("Type") == "bind":
volumes[m.get("Source")] = {"bind": dest, "mode": "rw" if m.get("RW", True) else "ro"}
@@ -470,6 +475,36 @@ async def recreate_container_with_updated_config(agent_name: str, old_container,
except docker.errors.NotFound:
pass
+ # Add public folder mount based on current file_sharing_enabled flag
+ # (FILES-001 Step 2). Mirrors the shared-folders expose pattern.
+ if db.get_file_sharing_enabled(agent_name):
+ public_volume_name = db.get_public_volume_name(agent_name)
+ public_volume_created = False
+ try:
+ await volume_get(public_volume_name)
+ except docker.errors.NotFound:
+ await volume_create(
+ name=public_volume_name,
+ labels={
+ 'trinity.platform': 'agent-public',
+ 'trinity.agent-name': agent_name,
+ },
+ )
+ public_volume_created = True
+
+ if public_volume_created:
+ try:
+ await containers_run(
+ 'alpine',
+ command='chown 1000:1000 /public',
+ volumes={public_volume_name: {'bind': '/public', 'mode': 'rw'}},
+ remove=True,
+ )
+ except Exception as e:
+ logger.warning(f"Could not fix public volume ownership: {e}")
+
+ volumes[public_volume_name] = {'bind': db.get_public_mount_path(), 'mode': 'rw'}
+
# Create new container with security settings
# Security principle: ALWAYS apply baseline security, even in full_capabilities mode
# - Always drop ALL caps, then add back only what's needed
diff --git a/src/backend/services/agent_service/queue.py b/src/backend/services/agent_service/queue.py
index 2ec4579ea..2d9623d8e 100644
--- a/src/backend/services/agent_service/queue.py
+++ b/src/backend/services/agent_service/queue.py
@@ -1,16 +1,16 @@
"""
Agent Service Queue - Execution queue operations.
-Handles agent execution queue management.
+Handles agent execution queue management. Funnels all capacity operations
+through the unified CapacityManager (#428).
"""
import logging
from fastapi import HTTPException
from models import User
+from services.capacity_manager import get_capacity_manager
from services.docker_service import get_agent_container
-from services.execution_queue import get_execution_queue
-from services.slot_service import get_slot_service
logger = logging.getLogger(__name__)
@@ -36,8 +36,10 @@ async def get_agent_queue_status_logic(
raise HTTPException(status_code=404, detail="Agent not found")
try:
- queue = get_execution_queue()
- status = await queue.get_status(agent_name)
+ capacity = get_capacity_manager()
+ # /chat path is serial (max_concurrent=1); status endpoint historically
+ # reflected /chat queue state.
+ status = await capacity.get_status(agent_name, max_concurrent=1)
return status.model_dump()
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to get queue status: {str(e)}")
@@ -60,8 +62,8 @@ async def clear_agent_queue_logic(
raise HTTPException(status_code=404, detail="Agent not found")
try:
- queue = get_execution_queue()
- cleared_count = await queue.clear_queue(agent_name)
+ capacity = get_capacity_manager()
+ cleared_count = await capacity.clear_in_memory_queue(agent_name)
return {
"status": "cleared",
@@ -90,22 +92,14 @@ async def force_release_agent_logic(
raise HTTPException(status_code=404, detail="Agent not found")
try:
- queue = get_execution_queue()
- was_running = await queue.force_release(agent_name)
-
- # Issue #98: Also clear all capacity slots since we're force-releasing
- slots_cleared = 0
- try:
- slot_service = get_slot_service()
- slots_cleared = await slot_service.force_clear_slots(agent_name)
- except Exception as e:
- logger.warning(f"[Queue] Failed to clear slots during force release of '{agent_name}': {e}")
+ capacity = get_capacity_manager()
+ result = await capacity.force_release(agent_name)
return {
"status": "released",
"agent": agent_name,
- "was_running": was_running,
- "slots_cleared": slots_cleared,
+ "was_running": result.was_running,
+ "slots_cleared": result.slots_cleared,
"warning": "Agent queue state and capacity slots have been reset. Any in-progress execution may still be running."
}
except Exception as e:
diff --git a/src/backend/services/agent_shared_files_service.py b/src/backend/services/agent_shared_files_service.py
new file mode 100644
index 000000000..a7d9a7dba
--- /dev/null
+++ b/src/backend/services/agent_shared_files_service.py
@@ -0,0 +1,385 @@
+"""
+Share-creation service for outbound file sharing (amazing-file-outbound, FILES-001 Step 3).
+
+Responsibilities:
+- Validate the filename the agent names via MCP (reject absolute, `..`, etc.)
+- Extract the single file via Docker SDK `get_archive`
+- Enforce size cap + per-agent quota
+- Detect MIME with python-magic and reject executables
+- Persist to /data/agent-files/{file_id} (under the existing trinity-data mount)
+- Insert into agent_shared_files
+- Return {file_id, url, expires_at, size_bytes, mime_type, one_time}
+"""
+
+from __future__ import annotations
+
+import io
+import logging
+import os
+import re
+import secrets
+import shutil
+import tarfile
+import uuid
+from datetime import datetime, timedelta, timezone
+from pathlib import PurePosixPath
+from typing import Optional
+
+import docker.errors
+from fastapi import HTTPException
+
+from database import db
+from services.docker_service import get_agent_container
+from services.docker_utils import container_get_archive
+from services.settings_service import get_public_chat_url
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Constants — MVP hardcoded; can migrate to settings later
+# ---------------------------------------------------------------------------
+
+MAX_FILE_SIZE_BYTES = 50 * 1024 * 1024 # 50 MB per file
+MAX_AGENT_QUOTA_BYTES = 500 * 1024 * 1024 # 500 MB per agent
+MIN_EXPIRES_IN = 60 # 1 minute
+MAX_EXPIRES_IN = 7 * 24 * 60 * 60 # 7 days
+DEFAULT_EXPIRES_IN = MAX_EXPIRES_IN
+
+# C3: refuse writes when /data has less than this much free space.
+# 500 MB × (typical concurrent shares + SQLite WAL + Vector logs) is a
+# reasonable floor before the shared `/data` mount starts causing
+# problems for the backend itself (DB writes, Vector, log archives).
+MIN_FREE_DISK_BYTES = 500 * 1024 * 1024
+
+PUBLISH_DIR = "/home/developer/public"
+STORAGE_ROOT = "/data/agent-files"
+
+# Magic-byte signatures for executables we reject outright.
+EXECUTABLE_SIGNATURES = (
+ b"MZ", # PE (Windows)
+ b"\x7fELF", # ELF (Linux)
+ b"\xca\xfe\xba\xbe", # Mach-O (32 & fat binaries)
+ b"\xcf\xfa\xed\xfe", # Mach-O (64 little-endian)
+ b"\xfe\xed\xfa\xce", # Mach-O (32 big-endian)
+ b"\xfe\xed\xfa\xcf", # Mach-O (64 big-endian)
+ b"#!", # Shell/interpreter scripts
+)
+
+# Optional python-magic; gracefully fall back to None on import failure
+# (docker/backend/Dockerfile installs libmagic1 + python-magic since #354).
+try: # pragma: no cover
+ import magic # type: ignore
+
+ _MAGIC_AVAILABLE = True
+except Exception: # pragma: no cover
+ _MAGIC_AVAILABLE = False
+ logger.warning("[shared-files] python-magic unavailable; MIME detection falls back to 'application/octet-stream'")
+
+
+# ---------------------------------------------------------------------------
+# Path validation
+# ---------------------------------------------------------------------------
+
+
+def validate_publish_path(filename: str) -> str:
+ """
+ Ensure the caller-provided filename is a safe relative path inside
+ PUBLISH_DIR. Returns the **container-absolute** path on success.
+
+ Raises HTTPException(400, 'PATH_TRAVERSAL') for any escape attempt.
+ """
+ if not filename:
+ raise HTTPException(status_code=400, detail="PATH_TRAVERSAL: filename required")
+
+ # Reject absolute paths — agent should only name things relative to the
+ # publish dir.
+ if filename.startswith("/"):
+ raise HTTPException(status_code=400, detail="PATH_TRAVERSAL: absolute paths rejected")
+
+ # PurePosixPath normalises `./a/b` → `a/b` and reveals escapes as `..`.
+ parts = PurePosixPath(filename).parts
+ if any(p in ("..", "") for p in parts):
+ raise HTTPException(status_code=400, detail="PATH_TRAVERSAL: '..' segments rejected")
+ if any(p.startswith("/") for p in parts):
+ raise HTTPException(status_code=400, detail="PATH_TRAVERSAL: absolute component rejected")
+
+ # Rebuild under the publish dir; guard against backslash tricks on
+ # Windows-style input (defense in depth even though containers are linux).
+ if "\\" in filename:
+ raise HTTPException(status_code=400, detail="PATH_TRAVERSAL: backslashes rejected")
+
+ resolved = str(PurePosixPath(PUBLISH_DIR, *parts))
+ return resolved
+
+
+# ---------------------------------------------------------------------------
+# MIME detection + blocklist
+# ---------------------------------------------------------------------------
+
+
+def detect_mime(data: bytes) -> str:
+ """Return the MIME type inferred from the first bytes of `data`."""
+ if _MAGIC_AVAILABLE:
+ try:
+ return magic.from_buffer(data[:4096], mime=True) or "application/octet-stream"
+ except Exception as e:
+ logger.warning(f"[shared-files] MIME detection failed: {e}")
+ return "application/octet-stream"
+
+
+def check_mime_blocklist(data: bytes, mime_type: str) -> None:
+ """Raise HTTPException(400) if `data`/`mime_type` looks like an executable."""
+ # Header-byte check — strongest signal
+ for sig in EXECUTABLE_SIGNATURES:
+ if data.startswith(sig):
+ raise HTTPException(status_code=400, detail=f"MIME_BLOCKED: executable content ({sig!r})")
+
+ # MIME-type check as backup
+ blocked_prefixes = ("application/x-executable", "application/x-dosexec", "application/x-mach-binary")
+ if any(mime_type.startswith(p) for p in blocked_prefixes):
+ raise HTTPException(status_code=400, detail=f"MIME_BLOCKED: {mime_type}")
+
+
+# ---------------------------------------------------------------------------
+# Quota + extraction
+# ---------------------------------------------------------------------------
+
+
+def enforce_quota(agent_name: str, new_bytes: int) -> None:
+ """Raise HTTPException(413) if the new file would exceed the agent's quota."""
+ current = db.total_shared_file_bytes_for_agent(agent_name)
+ if current + new_bytes > MAX_AGENT_QUOTA_BYTES:
+ raise HTTPException(
+ status_code=413,
+ detail=f"QUOTA_EXCEEDED: {current + new_bytes} bytes would exceed {MAX_AGENT_QUOTA_BYTES}",
+ )
+
+
+async def extract_from_agent(
+ agent_name: str, container_path: str
+) -> tuple[bytes, str]:
+ """
+ Pull the single file at `container_path` out of the agent container.
+
+ Returns (raw_bytes, basename). Raises:
+ - 404 FILE_NOT_FOUND if the path doesn't exist in the container
+ - 400 NOT_REGULAR_FILE if the entry is a dir, symlink, or device
+ - 413 SIZE_LIMIT_EXCEEDED if raw size > MAX_FILE_SIZE_BYTES
+ """
+ container = get_agent_container(agent_name)
+ if not container:
+ raise HTTPException(status_code=404, detail="Agent not found")
+
+ try:
+ stream, _stat = await container_get_archive(container, container_path)
+ except docker.errors.NotFound:
+ raise HTTPException(status_code=404, detail=f"FILE_NOT_FOUND: {container_path}")
+
+ # `stream` is a generator of tar bytes — join it, then untar in memory.
+ # Cap the buffer early at MAX_FILE_SIZE_BYTES + tar overhead (~1 KB) so a
+ # malicious agent can't OOM us by naming a huge file.
+ cap = MAX_FILE_SIZE_BYTES + 4096
+ buf = bytearray()
+ for chunk in stream:
+ buf.extend(chunk)
+ if len(buf) > cap:
+ raise HTTPException(status_code=413, detail="SIZE_LIMIT_EXCEEDED")
+
+ try:
+ with tarfile.open(fileobj=io.BytesIO(bytes(buf)), mode="r") as tar:
+ members = [m for m in tar.getmembers() if m.name]
+ if not members:
+ raise HTTPException(status_code=404, detail="FILE_NOT_FOUND: empty archive")
+ member = members[0]
+ if not member.isfile():
+ raise HTTPException(status_code=400, detail="NOT_REGULAR_FILE")
+ if member.size > MAX_FILE_SIZE_BYTES:
+ raise HTTPException(status_code=413, detail="SIZE_LIMIT_EXCEEDED")
+
+ extracted = tar.extractfile(member)
+ if extracted is None:
+ raise HTTPException(status_code=400, detail="NOT_REGULAR_FILE")
+ data = extracted.read()
+ return data, os.path.basename(member.name)
+ except tarfile.TarError as e:
+ raise HTTPException(status_code=500, detail=f"archive extraction failed: {e}")
+
+
+# ---------------------------------------------------------------------------
+# Filename sanitization for download headers (Content-Disposition)
+# ---------------------------------------------------------------------------
+
+# Conservative allowlist for on-disk + display filenames. Anything outside
+# becomes '_'. Prevents CRLF injection into Content-Disposition and keeps
+# filesystems happy.
+_SAFE_NAME_RE = re.compile(r"[^A-Za-z0-9._\- ]")
+
+
+def sanitize_display_name(name: str, fallback: str) -> str:
+ if not name:
+ return fallback
+ cleaned = _SAFE_NAME_RE.sub("_", name.strip())
+ cleaned = cleaned.strip(" .")
+ return cleaned or fallback
+
+
+# ---------------------------------------------------------------------------
+# URL building — matches public-chat convention
+# ---------------------------------------------------------------------------
+
+
+def build_download_url(file_id: str, download_token: str) -> str:
+ """
+ Build the external download URL.
+
+ Uses the `/api/files/{id}` path so requests traverse the Vite dev
+ proxy and the prod nginx config's existing `/api/*` rules — no
+ dedicated `/files/*` proxy needed anywhere.
+
+ The query parameter name is `sig` (signature) rather than
+ `download_token`. The backend's credential sanitizer
+ (`utils/credential_sanitizer.py`) redacts any `...TOKEN...=value`
+ pattern, which would strip our legitimate token out of agent
+ responses before they reach the user. `sig` avoids all the
+ sanitizer's sensitive-key patterns.
+
+ If `public_chat_url` is configured, the URL is absolute and
+ user-shareable; otherwise it's relative (frontend resolves vs.
+ window.origin on click).
+ """
+ base = (get_public_chat_url() or "").rstrip("/")
+ path = f"/api/files/{file_id}?sig={download_token}"
+ return f"{base}{path}" if base else path
+
+
+# ---------------------------------------------------------------------------
+# Orchestrator
+# ---------------------------------------------------------------------------
+
+
+def _ensure_storage_dir() -> str:
+ os.makedirs(STORAGE_ROOT, exist_ok=True)
+ return STORAGE_ROOT
+
+
+def check_disk_space(needed_bytes: int, *, root: str = "/data", min_free: int = MIN_FREE_DISK_BYTES) -> None:
+ """
+ Refuse to write if `root` doesn't have `needed_bytes + min_free` free.
+
+ Raises HTTPException(507 Insufficient Storage) on failure. This runs
+ before every share to protect the shared `/data` mount — the same
+ filesystem holds the SQLite DB, Vector logs, and log archives, so
+ letting an agent fill it causes platform-wide outage, not just a
+ failed share.
+ """
+ try:
+ usage = shutil.disk_usage(root)
+ except Exception as e:
+ logger.warning(f"[shared-files] disk_usage({root}) failed: {e} — skipping check")
+ return
+ required = needed_bytes + min_free
+ if usage.free < required:
+ logger.error(
+ "[shared-files] refusing write: /data has %d free bytes, "
+ "need %d (file=%d + floor=%d)",
+ usage.free, required, needed_bytes, min_free,
+ )
+ raise HTTPException(
+ status_code=507,
+ detail=(
+ f"INSUFFICIENT_STORAGE: platform disk is too full to accept "
+ f"a {needed_bytes}-byte share (need {min_free} bytes free after write, "
+ f"have {usage.free})."
+ ),
+ )
+
+
+async def create_share(
+ agent_name: str,
+ filename: str,
+ *,
+ display_name: Optional[str] = None,
+ expires_in: Optional[int] = None,
+ created_by: Optional[str] = None,
+) -> dict:
+ """
+ End-to-end: extract → validate → persist → DB insert → return URL payload.
+ """
+ # --- flag gate ---
+ if not db.get_file_sharing_enabled(agent_name):
+ raise HTTPException(status_code=403, detail="FEATURE_DISABLED: file sharing is not enabled for this agent")
+
+ # --- expiration ---
+ if expires_in is None:
+ expires_in = DEFAULT_EXPIRES_IN
+ if expires_in < MIN_EXPIRES_IN or expires_in > MAX_EXPIRES_IN:
+ raise HTTPException(
+ status_code=400,
+ detail=f"INVALID_EXPIRATION: expires_in must be between {MIN_EXPIRES_IN} and {MAX_EXPIRES_IN}",
+ )
+
+ # --- path ---
+ container_path = validate_publish_path(filename)
+
+ # --- extract ---
+ data, basename = await extract_from_agent(agent_name, container_path)
+ size_bytes = len(data)
+
+ # --- MIME ---
+ mime_type = detect_mime(data)
+ check_mime_blocklist(data, mime_type)
+
+ # --- quota ---
+ enforce_quota(agent_name, size_bytes)
+
+ # --- disk-space pre-check (C3) ---
+ _ensure_storage_dir()
+ check_disk_space(size_bytes)
+
+ # --- persist bytes on disk ---
+ file_id = str(uuid.uuid4())
+ stored_filename = file_id # UUID filename — name is opaque on disk
+ stored_path = os.path.join(STORAGE_ROOT, stored_filename)
+ with open(stored_path, "wb") as fh:
+ fh.write(data)
+
+ # --- DB row ---
+ display = sanitize_display_name(display_name or basename, fallback=f"file-{file_id}.bin")
+ download_token = secrets.token_urlsafe(32)
+ now = datetime.now(timezone.utc)
+ expires_at = now + timedelta(seconds=expires_in)
+
+ try:
+ db.create_agent_shared_file(
+ file_id=file_id,
+ agent_name=agent_name,
+ filename=display,
+ stored_filename=stored_filename,
+ size_bytes=size_bytes,
+ mime_type=mime_type,
+ download_token=download_token,
+ created_by=created_by or agent_name,
+ created_at=now.isoformat(),
+ expires_at=expires_at.isoformat(),
+ )
+ except Exception:
+ # Don't leak half-written files on DB failure
+ try:
+ os.unlink(stored_path)
+ except Exception:
+ pass
+ raise
+
+ url = build_download_url(file_id, download_token)
+ logger.info(
+ "[shared-files] agent=%s shared file_id=%s filename=%s size=%d mime=%s",
+ agent_name, file_id, display, size_bytes, mime_type,
+ )
+
+ return {
+ "file_id": file_id,
+ "url": url,
+ "expires_at": expires_at.isoformat(),
+ "size_bytes": size_bytes,
+ "mime_type": mime_type,
+ }
diff --git a/src/backend/services/audit_retention_service.py b/src/backend/services/audit_retention_service.py
new file mode 100644
index 000000000..de2823dc4
--- /dev/null
+++ b/src/backend/services/audit_retention_service.py
@@ -0,0 +1,107 @@
+"""
+Audit Log Retention Service.
+
+Daily APScheduler job that prunes ``audit_log`` rows past the retention
+window. Closes the gap left after #20 / Phase 4 of
+``docs/requirements/AUDIT_TRAIL_ARCHITECTURE.md``: the append-only
+contract is enforced by SQLite triggers, but nothing was deleting old
+entries.
+
+Configuration (env vars):
+
+- ``AUDIT_LOG_RETENTION_DAYS`` (default ``365``) — minimum age before a
+ row is eligible for deletion. Floored at 365 because the
+ ``audit_log_no_delete`` trigger refuses younger rows.
+- ``AUDIT_RETENTION_ENABLED`` (default ``true``) — set to ``false`` to
+ disable the daily prune.
+- ``AUDIT_RETENTION_HOUR`` (default ``4``) — UTC hour to run. Defaults
+ to one hour after log archival to spread the nightly DB writes.
+
+Hash chain note: pruning DELETEs entries, which breaks the SHA-256
+``previous_hash``/``entry_hash`` chain across the cutoff. Verification
+via ``POST /api/audit-log/verify`` should be scoped to ranges *within*
+the retention window. This is documented behavior — pruning ages out
+unverifiable history rather than maintaining a chain over deleted rows.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from typing import Any, Dict
+
+from apscheduler.schedulers.asyncio import AsyncIOScheduler
+from apscheduler.triggers.cron import CronTrigger
+
+from database import db
+from services.platform_audit_service import platform_audit_service
+
+logger = logging.getLogger(__name__)
+
+# Trigger floor — see ``audit_log_no_delete`` in db/schema.py
+_RETENTION_FLOOR_DAYS = 365
+
+AUDIT_LOG_RETENTION_DAYS = max(
+ int(os.getenv("AUDIT_LOG_RETENTION_DAYS", str(_RETENTION_FLOOR_DAYS))),
+ _RETENTION_FLOOR_DAYS,
+)
+AUDIT_RETENTION_ENABLED = os.getenv("AUDIT_RETENTION_ENABLED", "true").lower() == "true"
+AUDIT_RETENTION_HOUR = int(os.getenv("AUDIT_RETENTION_HOUR", "4"))
+
+
+class AuditRetentionService:
+ """Daily prune of expired audit_log rows."""
+
+ def __init__(self) -> None:
+ self.scheduler = AsyncIOScheduler()
+
+ def start(self) -> None:
+ if not AUDIT_RETENTION_ENABLED:
+ logger.info("Audit retention disabled (AUDIT_RETENTION_ENABLED=false)")
+ return
+
+ self.scheduler.add_job(
+ self.prune,
+ CronTrigger(hour=AUDIT_RETENTION_HOUR, minute=15),
+ id="audit_log_retention",
+ name="Daily audit_log retention prune",
+ replace_existing=True,
+ misfire_grace_time=3600,
+ )
+ self.scheduler.start()
+ logger.info(
+ "Audit retention scheduler started: daily at %02d:15 UTC (retention=%dd)",
+ AUDIT_RETENTION_HOUR,
+ AUDIT_LOG_RETENTION_DAYS,
+ )
+
+ def stop(self) -> None:
+ if self.scheduler.running:
+ self.scheduler.shutdown(wait=False)
+ logger.info("Audit retention scheduler stopped")
+
+ async def prune(self) -> Dict[str, Any]:
+ """Run a single prune cycle. Returns summary for tests/manual triggers."""
+ retention_days = AUDIT_LOG_RETENTION_DAYS
+ try:
+ removed = db.prune_audit_log(retention_days)
+ except Exception as exc:
+ logger.exception("audit_log prune failed: %s", exc)
+ return {"removed": 0, "retention_days": retention_days, "error": str(exc)}
+
+ if getattr(platform_audit_service, "_hash_chain_enabled", False) and removed:
+ logger.warning(
+ "audit_log prune removed %d rows while hash chain is enabled — "
+ "verification ranges spanning the cutoff will fail by design",
+ removed,
+ )
+
+ logger.info(
+ "audit_log prune complete: removed=%d retention_days=%d",
+ removed,
+ retention_days,
+ )
+ return {"removed": removed, "retention_days": retention_days}
+
+
+audit_retention_service = AuditRetentionService()
diff --git a/src/backend/services/capacity_manager.py b/src/backend/services/capacity_manager.py
new file mode 100644
index 000000000..37eab13e9
--- /dev/null
+++ b/src/backend/services/capacity_manager.py
@@ -0,0 +1,505 @@
+"""
+CapacityManager — unified per-agent execution capacity (#428).
+
+Replaces the three-class pyramid (`ExecutionQueue` + `SlotService` +
+`BacklogService`) with a single facade. The two surviving primitives are kept
+as internal collaborators because each has a distinct, well-tested job:
+
+ SlotService — atomic N-ary count gate (Redis ZSET).
+ BacklogService — persistent overflow store (SQL row + drain spawn).
+
+The third class (`ExecutionQueue`) is deleted; its responsibilities collapse
+into:
+ - the N=1 case of the count gate (handled by SlotService), and
+ - an in-memory FIFO overflow store (Redis LIST) implemented inline below.
+
+Public API (this is the *only* surface callers should reach for):
+
+ acquire(agent_name, execution_id, max_concurrent, *,
+ overflow_policy, overflow_payload=None, ...) -> AcquireResult
+ release(agent_name, execution_id) -> None
+ release_if_matches(agent_name, execution_id) -> bool
+ get_status(agent_name, max_concurrent) -> QueueStatus
+ get_all_states(agent_capacities) -> Dict[str, Dict[str, int]]
+ force_release(agent_name) -> ForceReleaseResult
+ reclaim_stale(agent_timeouts) -> Dict[str, List[str]]
+ cancel_all_overflow(agent_name, reason) -> int
+
+Overflow policies:
+ "reject" — no queue; raise CapacityFull when at capacity.
+ "queue_in_memory" — Redis LIST FIFO bounded by IN_MEMORY_DEPTH.
+ Used by /chat: position is observability only;
+ the agent serializes Claude subprocess execution.
+ "queue_persistent" — SQL backlog (BacklogService). Used by /task: caller
+ returns 202 Accepted and the drain reconstructs the
+ request later.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import uuid
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Any, Dict, List, Literal, Optional
+
+import redis
+
+from models import (
+ Execution,
+ ExecutionSource,
+ QueueItemStatus,
+ QueueStatus,
+)
+from services.backlog_service import BacklogService
+from services.slot_service import SlotService
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Public types
+# ---------------------------------------------------------------------------
+
+OverflowPolicy = Literal["reject", "queue_in_memory", "queue_persistent"]
+
+# In-memory overflow depth — preserved from the original ExecutionQueue
+# (MAX_QUEUE_SIZE=3). This is a rate-limit / observability bound, not a
+# serialization mechanism — the agent's Claude subprocess is the real serial
+# bottleneck.
+IN_MEMORY_DEPTH = 3
+
+
+@dataclass
+class AcquireResult:
+ """Outcome of `CapacityManager.acquire`.
+
+ state:
+ "admitted" — slot was acquired; caller proceeds.
+ "queued_in_memory" — overflow recorded in the in-memory FIFO; caller
+ still proceeds (the agent serializes execution).
+ `queue_position` is set for observability.
+ "queued_persistent" — overflow persisted to the SQL backlog; caller
+ should return 202 Accepted. The drain handles
+ actual execution later.
+ """
+ state: Literal["admitted", "queued_in_memory", "queued_persistent"]
+ execution_id: str
+ queue_position: Optional[int] = None
+
+
+@dataclass
+class ForceReleaseResult:
+ """Result of an emergency `force_release` — used by force-release endpoint."""
+ was_running: bool
+ slots_cleared: int
+
+
+@dataclass
+class PersistentTaskPayload:
+ """All fields needed to reconstruct a /task request when drained from the
+ persistent backlog. Mirrors the existing BacklogService.enqueue signature
+ so the wire format on the SQL row is unchanged."""
+ request: Any # ParallelTaskRequest — kept Any to avoid a router import here
+ effective_timeout: int
+ user_id: Optional[int]
+ user_email: Optional[str]
+ subscription_id: Optional[str]
+ x_source_agent: Optional[str]
+ x_mcp_key_id: Optional[str]
+ x_mcp_key_name: Optional[str]
+ triggered_by: str
+ collaboration_activity_id: Optional[str]
+ is_self_task: bool = False
+ self_task_activity_id: Optional[str] = None
+
+
+class CapacityFull(Exception):
+ """Raised when admit fails: capacity exhausted AND no overflow available.
+
+ `reason` distinguishes the cases for the caller's HTTP error message:
+ "rejected" — overflow_policy="reject" and at capacity.
+ "in_memory_full" — in-memory queue at IN_MEMORY_DEPTH.
+ "persistent_full" — persistent backlog at its configured depth.
+ """
+ def __init__(
+ self,
+ agent_name: str,
+ max_concurrent: int,
+ reason: Literal["rejected", "in_memory_full", "persistent_full"],
+ depth: Optional[int] = None,
+ ):
+ self.agent_name = agent_name
+ self.max_concurrent = max_concurrent
+ self.reason = reason
+ self.depth = depth
+ super().__init__(
+ f"Agent '{agent_name}' at capacity ({max_concurrent}); reason={reason}"
+ )
+
+
+# ---------------------------------------------------------------------------
+# CapacityManager
+# ---------------------------------------------------------------------------
+
+
+class CapacityManager:
+ """Unified capacity facade — composes SlotService + BacklogService and owns
+ the in-memory overflow store. See module docstring for usage."""
+
+ # In-memory overflow stores under a distinct key from the slot ZSET so the
+ # two never collide. Preserved from the original ExecutionQueue prefix
+ # naming so any existing dashboards / debug tools keep working.
+ _MEM_QUEUE_PREFIX = "agent:queue:"
+
+ def __init__(
+ self,
+ redis_url: Optional[str] = None,
+ slot_service: Optional[SlotService] = None,
+ backlog_service: Optional[BacklogService] = None,
+ ):
+ url = redis_url or os.getenv("REDIS_URL", "redis://redis:6379")
+ self._redis = redis.from_url(url, decode_responses=True)
+ self._slots = slot_service or SlotService(url)
+ self._backlog = backlog_service or BacklogService()
+ # The drain callback used to live in main.py wiring SlotService → Backlog;
+ # CapacityManager owns it now so callers don't have to know.
+ self._slots.register_on_release(self._on_slot_released)
+
+ # ------------------------------------------------------------------
+ # Acquire
+ # ------------------------------------------------------------------
+
+ async def acquire(
+ self,
+ *,
+ agent_name: str,
+ execution_id: str,
+ max_concurrent: int,
+ message_preview: str = "",
+ timeout_seconds: int = 900,
+ overflow_policy: OverflowPolicy = "reject",
+ overflow_payload: Optional[PersistentTaskPayload] = None,
+ # In-memory overflow needs source metadata for the status endpoint:
+ source: Optional[ExecutionSource] = None,
+ source_agent: Optional[str] = None,
+ source_user_id: Optional[str] = None,
+ source_user_email: Optional[str] = None,
+ message: Optional[str] = None,
+ ) -> AcquireResult:
+ """Try to acquire a slot. On overflow, dispatch to the chosen policy.
+
+ Raises:
+ CapacityFull: if at capacity AND overflow store is also full
+ (or overflow_policy="reject").
+ """
+ # First try to acquire a slot directly. SlotService already does the
+ # atomic ZADD + count check.
+ admitted = await self._slots.acquire_slot(
+ agent_name=agent_name,
+ execution_id=execution_id,
+ max_parallel_tasks=max_concurrent,
+ message_preview=message_preview,
+ timeout_seconds=timeout_seconds,
+ )
+ if admitted:
+ return AcquireResult(state="admitted", execution_id=execution_id)
+
+ # At capacity — apply overflow policy.
+ if overflow_policy == "reject":
+ raise CapacityFull(agent_name, max_concurrent, "rejected")
+
+ if overflow_policy == "queue_in_memory":
+ position = self._mem_enqueue(
+ agent_name=agent_name,
+ execution_id=execution_id,
+ source=source or ExecutionSource.USER,
+ source_agent=source_agent,
+ source_user_id=source_user_id,
+ source_user_email=source_user_email,
+ message=message or "",
+ )
+ return AcquireResult(
+ state="queued_in_memory",
+ execution_id=execution_id,
+ queue_position=position,
+ )
+
+ if overflow_policy == "queue_persistent":
+ if overflow_payload is None:
+ raise ValueError(
+ "queue_persistent overflow requires overflow_payload"
+ )
+ enqueued = await self._backlog.enqueue(
+ agent_name=agent_name,
+ execution_id=execution_id,
+ request=overflow_payload.request,
+ effective_timeout=overflow_payload.effective_timeout,
+ user_id=overflow_payload.user_id,
+ user_email=overflow_payload.user_email,
+ subscription_id=overflow_payload.subscription_id,
+ x_source_agent=overflow_payload.x_source_agent,
+ x_mcp_key_id=overflow_payload.x_mcp_key_id,
+ x_mcp_key_name=overflow_payload.x_mcp_key_name,
+ triggered_by=overflow_payload.triggered_by,
+ collaboration_activity_id=overflow_payload.collaboration_activity_id,
+ is_self_task=overflow_payload.is_self_task,
+ self_task_activity_id=overflow_payload.self_task_activity_id,
+ )
+ if not enqueued:
+ raise CapacityFull(agent_name, max_concurrent, "persistent_full")
+ return AcquireResult(
+ state="queued_persistent", execution_id=execution_id
+ )
+
+ raise ValueError(f"Unknown overflow_policy: {overflow_policy!r}")
+
+ # ------------------------------------------------------------------
+ # Release
+ # ------------------------------------------------------------------
+
+ async def release(self, agent_name: str, execution_id: str) -> None:
+ """Release the slot for `execution_id`. Drains overflow internally:
+ - In-memory queue: pops the next entry (bookkeeping only).
+ - Persistent backlog: SlotService.register_on_release fires
+ `_on_slot_released` which drains one row.
+ """
+ await self._slots.release_slot(agent_name, execution_id)
+ # In-memory overflow: pop next for status-endpoint bookkeeping. The
+ # persistent backlog is drained via the slot-release callback wired
+ # in __init__.
+ self._mem_pop(agent_name)
+
+ async def release_if_matches(
+ self, agent_name: str, execution_id: str
+ ) -> bool:
+ """Release the slot only if `execution_id` currently holds it.
+
+ Used by the watchdog (TOCTOU-safe): we only want to release when the
+ execution we're recovering is still the running one. SlotService's
+ ZSET model is naturally per-execution_id — release_slot is a no-op
+ for an execution_id that isn't there — so this is effectively the
+ same as `release`. Kept as a separate method to preserve the existing
+ watchdog call site's intent.
+ """
+ # Check membership first so we can report whether this was a match.
+ slots_key = f"{self._slots.slots_prefix}{agent_name}"
+ if self._redis.zscore(slots_key, execution_id) is None:
+ return False
+ await self._slots.release_slot(agent_name, execution_id)
+ self._mem_pop(agent_name)
+ return True
+
+ async def _on_slot_released(self, agent_name: str) -> None:
+ """SlotService callback. Drains one persistent backlog row if any."""
+ try:
+ await self._backlog.drain_next(agent_name)
+ except Exception as e: # pragma: no cover - defensive
+ logger.error(
+ f"[Capacity] backlog drain on release failed for "
+ f"'{agent_name}': {e}",
+ exc_info=True,
+ )
+
+ # ------------------------------------------------------------------
+ # Status / introspection
+ # ------------------------------------------------------------------
+
+ async def get_status(
+ self, agent_name: str, max_concurrent: int = 1
+ ) -> QueueStatus:
+ """Status for the queue endpoint — current execution + in-memory queue.
+
+ For backwards compatibility with the original /api/agents/{name}/queue
+ endpoint, this reports only the in-memory queue (the historical /chat
+ observability surface). Persistent backlog state is exposed via the
+ executions endpoints (status='queued' rows).
+ """
+ slot_state = await self._slots.get_slot_state(agent_name, max_concurrent)
+ is_busy = slot_state.active_slots > 0
+
+ # The original ExecutionQueue.get_status surfaced one "current" item;
+ # SlotService tracks N. For status-endpoint compatibility, surface
+ # the oldest active slot as `current_execution` when there is one.
+ current_execution: Optional[Execution] = None
+ if is_busy and slot_state.slots:
+ oldest = slot_state.slots[0]
+ current_execution = Execution(
+ id=oldest.execution_id,
+ agent_name=agent_name,
+ source=ExecutionSource.USER, # not tracked per-slot today
+ message=oldest.message_preview,
+ queued_at=datetime.utcnow(),
+ status=QueueItemStatus.RUNNING,
+ )
+
+ queued_executions = self._mem_list(agent_name)
+ return QueueStatus(
+ agent_name=agent_name,
+ is_busy=is_busy,
+ current_execution=current_execution,
+ queue_length=len(queued_executions),
+ queued_executions=queued_executions,
+ )
+
+ async def get_all_states(
+ self, agent_capacities: Dict[str, int]
+ ) -> Dict[str, Dict[str, int]]:
+ """Bulk capacity meter for the dashboard."""
+ return await self._slots.get_all_slot_states(agent_capacities)
+
+ async def get_slot_state(self, agent_name: str, max_concurrent: int):
+ """Detailed slot view for the per-agent capacity endpoint.
+
+ Returns the underlying SlotState (slot_number, started_at,
+ message_preview, duration_seconds per slot). Kept as a thin
+ pass-through so the agent_config router doesn't need to know about
+ the underlying primitive.
+ """
+ return await self._slots.get_slot_state(agent_name, max_concurrent)
+
+ # ------------------------------------------------------------------
+ # Cleanup / emergency
+ # ------------------------------------------------------------------
+
+ async def reclaim_stale(
+ self, agent_timeouts: Optional[Dict[str, int]] = None
+ ) -> Dict[str, List[str]]:
+ """Reclaim slots whose TTL has expired. Used by the cleanup watchdog.
+
+ `agent_timeouts` lets the watchdog supply per-agent execution timeouts
+ so each agent's slots are cleaned with the right TTL+buffer (#226).
+ """
+ return await self._slots.cleanup_stale_slots(agent_timeouts=agent_timeouts)
+
+ async def force_release(self, agent_name: str) -> ForceReleaseResult:
+ """Emergency: clear all running slots and the in-memory queue."""
+ slots_cleared = await self._slots.force_clear_slots(agent_name)
+ # Clear in-memory queue too.
+ mem_key = self._mem_queue_key(agent_name)
+ was_running_or_queued = (
+ slots_cleared > 0 or self._redis.exists(mem_key) > 0
+ )
+ if self._redis.exists(mem_key):
+ self._redis.delete(mem_key)
+ return ForceReleaseResult(
+ was_running=was_running_or_queued,
+ slots_cleared=slots_cleared,
+ )
+
+ async def clear_in_memory_queue(self, agent_name: str) -> int:
+ """Clear only the in-memory overflow queue (leaves running executions)."""
+ mem_key = self._mem_queue_key(agent_name)
+ count = self._redis.llen(mem_key)
+ if count > 0:
+ self._redis.delete(mem_key)
+ logger.info(
+ f"[Capacity] Cleared {count} in-memory queued items for '{agent_name}'"
+ )
+ return count
+
+ async def run_maintenance(self, max_age_hours: float = 24) -> None:
+ """Periodic maintenance for the persistent overflow store.
+
+ Expires queued rows older than `max_age_hours` to FAILED, then drains
+ any orphan backlog that didn't trigger a release callback (e.g. after
+ a backend restart between enqueue and drain). Called from main.py's
+ 60s loop.
+ """
+ await self._backlog.expire_stale(max_age_hours=max_age_hours)
+ await self._backlog.drain_orphans_all()
+
+ async def cancel_all_overflow(self, agent_name: str, reason: str) -> int:
+ """Cancel all queued work (in-memory + persistent) for an agent.
+
+ Used on agent deletion so dangling 'queued' rows don't point at a dead
+ agent. Returns the count of persistent rows cancelled (the in-memory
+ queue is best-effort).
+ """
+ # In-memory: best-effort delete.
+ await self.clear_in_memory_queue(agent_name)
+ # Persistent: real cancellation with audit reason.
+ return await self._backlog.cancel_all_backlog(agent_name, reason=reason)
+
+ # ------------------------------------------------------------------
+ # In-memory queue helpers (private)
+ # ------------------------------------------------------------------
+
+ def _mem_queue_key(self, agent_name: str) -> str:
+ return f"{self._MEM_QUEUE_PREFIX}{agent_name}"
+
+ def _mem_enqueue(
+ self,
+ *,
+ agent_name: str,
+ execution_id: str,
+ source: ExecutionSource,
+ source_agent: Optional[str],
+ source_user_id: Optional[str],
+ source_user_email: Optional[str],
+ message: str,
+ ) -> int:
+ """LPUSH an Execution into the in-memory queue. Returns 1-based position.
+
+ Bounded by IN_MEMORY_DEPTH so queue cannot grow unbounded; raises
+ CapacityFull when at the bound. (Bound preserved from the original
+ ExecutionQueue MAX_QUEUE_SIZE.)
+ """
+ queue_key = self._mem_queue_key(agent_name)
+ depth = self._redis.llen(queue_key)
+ if depth >= IN_MEMORY_DEPTH:
+ raise CapacityFull(
+ agent_name, IN_MEMORY_DEPTH, "in_memory_full", depth=depth
+ )
+ execution = Execution(
+ id=execution_id,
+ agent_name=agent_name,
+ source=source,
+ source_agent=source_agent,
+ source_user_id=source_user_id,
+ source_user_email=source_user_email,
+ message=message,
+ queued_at=datetime.utcnow(),
+ status=QueueItemStatus.QUEUED,
+ )
+ self._redis.lpush(queue_key, execution.model_dump_json())
+ # Position is 1-based and human-readable: oldest queued = position 1.
+ return depth + 1
+
+ def _mem_pop(self, agent_name: str) -> None:
+ """RPOP the oldest in-memory queued entry. Bookkeeping only — the
+ caller proceeded with the request when it was queued; this just
+ drops the record so the status endpoint stays accurate."""
+ queue_key = self._mem_queue_key(agent_name)
+ self._redis.rpop(queue_key)
+
+ def _mem_list(self, agent_name: str) -> List[Execution]:
+ """List in-memory queued executions, oldest first (FIFO order)."""
+ queue_key = self._mem_queue_key(agent_name)
+ items = self._redis.lrange(queue_key, 0, -1)
+ executions = [Execution.model_validate_json(item) for item in items]
+ # LPUSH/RPOP means LRANGE returns newest first; reverse for FIFO.
+ executions.reverse()
+ return executions
+
+
+# ---------------------------------------------------------------------------
+# Global singleton
+# ---------------------------------------------------------------------------
+
+_capacity_manager: Optional[CapacityManager] = None
+
+
+def get_capacity_manager() -> CapacityManager:
+ """Get the global CapacityManager instance."""
+ global _capacity_manager
+ if _capacity_manager is None:
+ _capacity_manager = CapacityManager()
+ return _capacity_manager
+
+
+def reset_capacity_manager() -> None:
+ """Reset the singleton — used by tests to inject mocked services."""
+ global _capacity_manager
+ _capacity_manager = None
diff --git a/src/backend/services/cleanup_service.py b/src/backend/services/cleanup_service.py
index d530d299d..5b375700a 100644
--- a/src/backend/services/cleanup_service.py
+++ b/src/backend/services/cleanup_service.py
@@ -22,8 +22,7 @@
from database import db
from models import TaskExecutionStatus
-from services.slot_service import get_slot_service
-from services.execution_queue import get_execution_queue
+from services.capacity_manager import get_capacity_manager
from utils.helpers import utc_now, utc_now_iso, parse_iso_timestamp
from utils.credential_sanitizer import sanitize_text
@@ -60,13 +59,14 @@ class CleanupReport:
stale_activities: int = 0
stale_slots: int = 0
stale_slot_executions: int = 0 # Issue #219: executions failed when their slot was reclaimed
+ shared_files_purged: int = 0 # C4 / FILES-001: expired or old-revoked file shares
@property
def total(self) -> int:
return (self.orphaned_executions + self.auto_terminated +
self.stale_executions + self.no_session_executions +
self.orphaned_skipped + self.stale_activities + self.stale_slots +
- self.stale_slot_executions)
+ self.stale_slot_executions + self.shared_files_purged)
def to_dict(self) -> Dict:
return {
@@ -78,6 +78,7 @@ def to_dict(self) -> Dict:
"stale_activities": self.stale_activities,
"stale_slots": self.stale_slots,
"stale_slot_executions": self.stale_slot_executions,
+ "shared_files_purged": self.shared_files_purged,
"total": self.total,
}
@@ -189,13 +190,13 @@ async def _run_cleanup_inner(self) -> CleanupReport:
# 3. Cleanup stale Redis slots and fail corresponding execution records
# (#219, #226, #378 — see _process_stale_slot_reclaims docstring).
try:
- slot_service = get_slot_service()
+ capacity = get_capacity_manager()
# #226: Query per-agent timeouts from DB so slot cleanup uses the
# correct TTL instead of a fixed 20-min default.
agent_timeouts = db.get_all_execution_timeouts()
- reclaimed = await slot_service.cleanup_stale_slots(
+ reclaimed = await capacity.reclaim_stale(
agent_timeouts=agent_timeouts
)
report.stale_slots = sum(len(ids) for ids in reclaimed.values())
@@ -216,6 +217,35 @@ async def _run_cleanup_inner(self) -> CleanupReport:
logger.info(f"[Cleanup] Pruned {pruned} rate-limit events (>24h old)")
except Exception as e:
logger.error(f"[Cleanup] Error pruning rate-limit events: {e}")
+
+ # 4b. Purge expired / old-revoked shared files (C4 / FILES-001).
+ # Every cycle — the set is usually small and both DB row + disk
+ # unlink are cheap. Grace period for revoked rows keeps them
+ # queryable for a day post-revoke (incident diagnosis).
+ try:
+ from pathlib import Path
+ stored_filenames = db.delete_expired_and_revoked_shared_files(
+ revoke_grace_hours=24
+ )
+ if stored_filenames:
+ storage_root = Path("/data/agent-files")
+ unlinked = 0
+ for sf in stored_filenames:
+ try:
+ p = storage_root / sf
+ if p.exists():
+ p.unlink()
+ unlinked += 1
+ except Exception as e:
+ logger.warning(f"[Cleanup] failed to unlink {sf}: {e}")
+ report.shared_files_purged = len(stored_filenames)
+ logger.info(
+ f"[Cleanup] Purged {len(stored_filenames)} shared-file "
+ f"rows ({unlinked} files unlinked from /data/agent-files/)"
+ )
+ except Exception as e:
+ logger.error(f"[Cleanup] Error purging shared files: {e}")
+
self._cycle_count += 1
self.last_run_at = utc_now_iso()
@@ -589,21 +619,16 @@ async def _recover_execution(
# Race condition: execution completed normally between check and update
return False
- # Release capacity slot (idempotent — no error if already released)
- try:
- slot_service = get_slot_service()
- await slot_service.release_slot(agent_name, execution_id)
- except Exception as e:
- logger.warning(f"[Watchdog] Error releasing slot for {execution_id}: {e}")
-
- # Atomically release execution queue only if THIS execution holds the slot.
- # Uses Lua script to prevent TOCTOU race where a new execution could start
- # between checking and releasing.
+ # Release capacity (idempotent — no error if already released).
+ # CAPACITY-CONSOLIDATE (#428): single CapacityManager.release_if_matches
+ # replaces the prior slot_service.release_slot + queue.force_release_if_matches
+ # pair. The match check preserves the TOCTOU-safety the original Lua
+ # script provided.
try:
- queue = get_execution_queue()
- await queue.force_release_if_matches(agent_name, execution_id)
+ capacity = get_capacity_manager()
+ await capacity.release_if_matches(agent_name, execution_id)
except Exception as e:
- logger.warning(f"[Watchdog] Error releasing queue for agent '{agent_name}': {e}")
+ logger.warning(f"[Watchdog] Error releasing capacity for {execution_id}: {e}")
# Broadcast WebSocket event with combined error (includes original context)
await self._broadcast_watchdog_event(action, agent_name, execution_id, combined_error)
@@ -713,7 +738,7 @@ async def recover_orphaned_executions() -> Dict:
if not running:
return {"recovered": 0, "still_running": 0, "errors": 0}
- slot_service = get_slot_service()
+ capacity = get_capacity_manager()
# Group by agent to minimize container/HTTP checks
by_agent: Dict[str, list] = {}
@@ -730,7 +755,7 @@ async def recover_orphaned_executions() -> Dict:
if not container or container.status != "running":
# Container down — all executions for this agent are orphaned
for execution in executions:
- if await _recover_execution(execution, agent_name, slot_service):
+ if await _recover_execution(execution, agent_name, capacity):
recovered += 1
else:
errors += 1
@@ -752,7 +777,7 @@ async def recover_orphaned_executions() -> Dict:
if execution["id"] in registry_ids:
still_running += 1
else:
- if await _recover_execution(execution, agent_name, slot_service):
+ if await _recover_execution(execution, agent_name, capacity):
recovered += 1
else:
errors += 1
@@ -764,15 +789,16 @@ async def recover_orphaned_executions() -> Dict:
return {"recovered": recovered, "still_running": still_running, "errors": errors}
-async def _recover_execution(execution: Dict, agent_name: str, slot_service) -> bool:
- """Mark a single execution as orphaned and release its slot. Returns True on success."""
+async def _recover_execution(execution: Dict, agent_name: str, capacity) -> bool:
+ """Mark a single execution as orphaned and release its capacity. Returns True on success."""
try:
- db.update_execution_status(
+ # Use the guarded writer so a real completion that arrived during restart
+ # is not overwritten (RELIABILITY-005).
+ db.mark_execution_failed_by_watchdog(
execution_id=execution["id"],
- status=TaskExecutionStatus.FAILED,
- error="Execution orphaned — recovered on backend restart",
+ error_message="Execution orphaned — recovered on backend restart",
)
- await slot_service.release_slot(agent_name, execution["id"])
+ await capacity.release(agent_name, execution["id"])
return True
except Exception as e:
logger.error(f"[Recovery] Error recovering execution {execution['id']}: {e}")
diff --git a/src/backend/services/docker_utils.py b/src/backend/services/docker_utils.py
index 489673df7..37cf75b31 100644
--- a/src/backend/services/docker_utils.py
+++ b/src/backend/services/docker_utils.py
@@ -225,6 +225,24 @@ def _exec():
return await loop.run_in_executor(_docker_executor, _exec)
+async def container_get_archive(container, path: str) -> tuple:
+ """Read a tar archive out of a container without blocking the event loop.
+
+ Args:
+ container: Docker container object
+ path: Source path inside the container
+
+ Returns:
+ (stream_generator, stat_dict) matching docker-py's return shape.
+ Raises docker.errors.NotFound when the path doesn't exist.
+ """
+ loop = asyncio.get_event_loop()
+ return await loop.run_in_executor(
+ _docker_executor,
+ lambda: container.get_archive(path),
+ )
+
+
async def container_put_archive(container, path: str, data: bytes) -> bool:
"""Write a tar archive into a container without blocking the event loop.
diff --git a/src/backend/services/execution_queue.py b/src/backend/services/execution_queue.py
deleted file mode 100644
index d4fbabac9..000000000
--- a/src/backend/services/execution_queue.py
+++ /dev/null
@@ -1,361 +0,0 @@
-"""
-Execution Queue Service for Trinity platform.
-
-Implements platform-level queueing to prevent parallel execution on the same agent.
-Only one execution can run per agent at a time; additional requests are queued.
-
-Redis-backed for:
-- Multi-worker support (uvicorn with multiple workers)
-- Persistence across backend restarts
-- Already deployed infrastructure (reuse existing Redis)
-
-Queue Rules:
-- One execution at a time per agent (enforced at platform level)
-- Queue max 3 waiting requests per agent
-- Reject (429) if queue full
-- Timeout queued requests after 120s of waiting
-- Track source (user/schedule/agent) for observability
-"""
-
-import json
-import logging
-from datetime import datetime
-from typing import Optional, List
-import redis
-import uuid
-
-from models import Execution, ExecutionSource, QueueItemStatus, QueueStatus
-
-logger = logging.getLogger(__name__)
-
-# Configuration
-MAX_QUEUE_SIZE = 3 # Max queued requests per agent
-EXECUTION_TTL = 600 # 10 minutes max execution time (Redis TTL)
-QUEUE_WAIT_TIMEOUT = 120 # 120 seconds max wait in queue
-
-
-class QueueFullError(Exception):
- """Raised when an agent's queue is full."""
- def __init__(self, agent_name: str, queue_length: int):
- self.agent_name = agent_name
- self.queue_length = queue_length
- super().__init__(f"Agent '{agent_name}' queue is full ({queue_length} waiting)")
-
-
-class AgentBusyError(Exception):
- """Raised when an agent is busy and caller doesn't want to wait."""
- def __init__(self, agent_name: str, current_execution: Optional[Execution] = None):
- self.agent_name = agent_name
- self.current_execution = current_execution
- super().__init__(f"Agent '{agent_name}' is currently executing")
-
-
-class ExecutionQueue:
- """
- Redis-backed execution queue for agents.
-
- Ensures only one execution runs per agent at a time.
- Additional requests are queued (up to MAX_QUEUE_SIZE) or rejected.
-
- Thread-safety: Uses atomic Redis operations to prevent race conditions:
- - submit(): Uses SET NX EX for atomic slot acquisition
- - complete(): Uses Lua script for atomic pop-and-set
- """
-
- # Lua script for atomic conditional release (Issue #129 review fix)
- # Only deletes the running key if it holds the expected execution ID
- CONDITIONAL_RELEASE_SCRIPT = """
-local running_key = KEYS[1]
-local expected_id = ARGV[1]
-local current = redis.call('GET', running_key)
-if current == false then
- return 0
-end
-local data = cjson.decode(current)
-if data['id'] == expected_id then
- redis.call('DEL', running_key)
- return 1
-end
-return 0
-"""
-
- # Lua script for atomic complete operation
- # Atomically pops next from queue and sets as running, or clears if empty
- COMPLETE_SCRIPT = """
-local running_key = KEYS[1]
-local queue_key = KEYS[2]
-local ttl = tonumber(ARGV[1])
-
--- Pop next from queue (RPOP for FIFO)
-local next_item = redis.call('RPOP', queue_key)
-
-if next_item then
- -- Atomically set as running with TTL
- redis.call('SET', running_key, next_item, 'EX', ttl)
- return next_item
-else
- -- No more items, clear running state
- redis.call('DEL', running_key)
- return nil
-end
-"""
-
- def __init__(self, redis_url: str = "redis://redis:6379"):
- self.redis = redis.from_url(redis_url, decode_responses=True)
- self.running_prefix = "agent:running:"
- self.queue_prefix = "agent:queue:"
- # Register Lua scripts for atomic operations
- self._complete_script = self.redis.register_script(self.COMPLETE_SCRIPT)
- self._conditional_release_script = self.redis.register_script(self.CONDITIONAL_RELEASE_SCRIPT)
-
- def _running_key(self, agent_name: str) -> str:
- """Redis key for currently running execution."""
- return f"{self.running_prefix}{agent_name}"
-
- def _queue_key(self, agent_name: str) -> str:
- """Redis key for waiting queue (list)."""
- return f"{self.queue_prefix}{agent_name}"
-
- def _serialize_execution(self, execution: Execution) -> str:
- """Serialize execution to JSON for Redis storage."""
- return execution.model_dump_json()
-
- def _deserialize_execution(self, data: str) -> Execution:
- """Deserialize execution from JSON."""
- return Execution.model_validate_json(data)
-
- def create_execution(
- self,
- agent_name: str,
- message: str,
- source: ExecutionSource,
- source_agent: Optional[str] = None,
- source_user_id: Optional[str] = None,
- source_user_email: Optional[str] = None
- ) -> Execution:
- """Create a new execution request (not yet submitted)."""
- return Execution(
- id=str(uuid.uuid4()),
- agent_name=agent_name,
- source=source,
- source_agent=source_agent,
- source_user_id=source_user_id,
- source_user_email=source_user_email,
- message=message,
- queued_at=datetime.utcnow(),
- status=QueueItemStatus.QUEUED
- )
-
- async def submit(
- self,
- execution: Execution,
- wait_if_busy: bool = True
- ) -> tuple[str, Execution]:
- """
- Submit execution request for an agent.
-
- Uses atomic SET NX EX to prevent race conditions where two concurrent
- requests could both acquire the execution slot.
-
- Args:
- execution: The execution request to submit
- wait_if_busy: If True, queue the request if busy. If False, raise AgentBusyError.
-
- Returns:
- Tuple of (status, execution):
- - ("running", execution) - Started immediately
- - ("queued:N", execution) - Queued at position N
-
- Raises:
- QueueFullError: If queue is at MAX_QUEUE_SIZE
- AgentBusyError: If wait_if_busy=False and agent is busy
- """
- running_key = self._running_key(execution.agent_name)
- queue_key = self._queue_key(execution.agent_name)
-
- # Prepare execution for running state
- execution.status = QueueItemStatus.RUNNING
- execution.started_at = datetime.utcnow()
- serialized = self._serialize_execution(execution)
-
- # Atomic acquire: SET key value NX EX ttl
- # Returns True if key was set (we acquired), False/None if key exists (busy)
- acquired = self.redis.set(
- running_key,
- serialized,
- nx=True, # Only set if Not eXists
- ex=EXECUTION_TTL
- )
-
- if acquired:
- logger.info(f"[Queue] Agent '{execution.agent_name}' execution started: {execution.id}")
- return ("running", execution)
-
- # Agent is busy - atomic acquire failed
- if not wait_if_busy:
- current = self.redis.get(running_key)
- current_exec = self._deserialize_execution(current) if current else None
- raise AgentBusyError(execution.agent_name, current_exec)
-
- # Reset execution state for queuing
- execution.status = QueueItemStatus.QUEUED
- execution.started_at = None
-
- # Check queue length and add (queue operations are a separate concern)
- queue_len = self.redis.llen(queue_key)
- if queue_len >= MAX_QUEUE_SIZE:
- logger.warning(f"[Queue] Agent '{execution.agent_name}' queue full ({queue_len} waiting)")
- raise QueueFullError(execution.agent_name, queue_len)
-
- # Add to queue (left push for FIFO - rpop will get oldest)
- self.redis.lpush(queue_key, self._serialize_execution(execution))
- position = queue_len + 1
- logger.info(f"[Queue] Agent '{execution.agent_name}' execution queued at position {position}: {execution.id}")
- return (f"queued:{position}", execution)
-
- async def complete(self, agent_name: str, success: bool = True) -> Optional[Execution]:
- """
- Mark current execution done and start next if queued.
-
- Uses a Lua script for atomic pop-and-set to prevent race conditions
- where queue entries could be lost or processed twice.
-
- Args:
- agent_name: The agent that completed execution
- success: Whether execution succeeded
-
- Returns:
- Next execution that was started (if any), or None
- """
- running_key = self._running_key(agent_name)
- queue_key = self._queue_key(agent_name)
-
- # Log completed execution (before atomic operation)
- completed = self.redis.get(running_key)
- if completed:
- completed_exec = self._deserialize_execution(completed)
- status_str = "completed" if success else "failed"
- logger.info(f"[Queue] Agent '{agent_name}' execution {status_str}: {completed_exec.id}")
-
- # Atomic: pop next and set as running (or clear if empty)
- # The Lua script handles both operations atomically
- next_item = self._complete_script(
- keys=[running_key, queue_key],
- args=[EXECUTION_TTL]
- )
-
- if next_item:
- next_exec = self._deserialize_execution(next_item)
- # Update status in Python object (Redis already has the data)
- next_exec.status = QueueItemStatus.RUNNING
- next_exec.started_at = datetime.utcnow()
- logger.info(f"[Queue] Agent '{agent_name}' starting next execution: {next_exec.id}")
- return next_exec
- else:
- logger.info(f"[Queue] Agent '{agent_name}' queue empty, now idle")
- return None
-
- async def get_status(self, agent_name: str) -> QueueStatus:
- """Get current queue status for an agent."""
- running_key = self._running_key(agent_name)
- queue_key = self._queue_key(agent_name)
-
- # Get current execution
- running = self.redis.get(running_key)
- current_execution = self._deserialize_execution(running) if running else None
-
- # Get queued executions
- queue_items = self.redis.lrange(queue_key, 0, -1)
- queued_executions = [self._deserialize_execution(item) for item in queue_items]
- # Reverse to show oldest first (FIFO order)
- queued_executions.reverse()
-
- return QueueStatus(
- agent_name=agent_name,
- is_busy=current_execution is not None,
- current_execution=current_execution,
- queue_length=len(queued_executions),
- queued_executions=queued_executions
- )
-
- async def is_busy(self, agent_name: str) -> bool:
- """Check if agent is currently executing."""
- running_key = self._running_key(agent_name)
- return self.redis.exists(running_key) > 0
-
- async def clear_queue(self, agent_name: str) -> int:
- """
- Clear all queued executions for an agent (not current execution).
-
- Returns the number of cleared items.
- """
- queue_key = self._queue_key(agent_name)
- count = self.redis.llen(queue_key)
- if count > 0:
- self.redis.delete(queue_key)
- logger.info(f"[Queue] Cleared {count} queued executions for agent '{agent_name}'")
- return count
-
- async def force_release(self, agent_name: str) -> bool:
- """
- Force release an agent (emergency use - clears running state).
-
- Use this if an execution is stuck or the agent died without completing.
- Returns True if there was a running execution.
- """
- running_key = self._running_key(agent_name)
- existed = self.redis.exists(running_key) > 0
- if existed:
- self.redis.delete(running_key)
- logger.warning(f"[Queue] Force released agent '{agent_name}' from running state")
- return existed
-
- async def force_release_if_matches(self, agent_name: str, execution_id: str) -> bool:
- """Atomically release running state only if the given execution holds it.
-
- Uses a Lua script to GET, compare execution ID, and conditional DELETE
- in a single atomic operation. Prevents TOCTOU race where a new execution
- could start between the check and the release.
-
- Returns True if the running state was released, False if it didn't match
- or no running execution existed.
- """
- running_key = self._running_key(agent_name)
- result = self._conditional_release_script(
- keys=[running_key], args=[execution_id]
- )
- if result:
- logger.warning(
- f"[Queue] Conditionally released agent '{agent_name}' "
- f"(execution {execution_id})"
- )
- return bool(result)
-
- async def get_all_busy_agents(self) -> List[str]:
- """Get list of all agents currently executing.
-
- Uses SCAN instead of KEYS to avoid blocking Redis on large datasets.
- """
- pattern = f"{self.running_prefix}*"
- agents = []
- cursor = 0
- while True:
- cursor, keys = self.redis.scan(cursor, match=pattern, count=100)
- agents.extend(key.replace(self.running_prefix, "") for key in keys)
- if cursor == 0:
- break
- return agents
-
-
-# Global instance (initialized on import if Redis is available)
-_execution_queue: Optional[ExecutionQueue] = None
-
-
-def get_execution_queue() -> ExecutionQueue:
- """Get the global execution queue instance."""
- global _execution_queue
- if _execution_queue is None:
- import os
- redis_url = os.getenv("REDIS_URL", "redis://redis:6379")
- _execution_queue = ExecutionQueue(redis_url)
- return _execution_queue
diff --git a/src/backend/services/gemini_voice.py b/src/backend/services/gemini_voice.py
index 3eac2fef9..43714c3da 100644
--- a/src/backend/services/gemini_voice.py
+++ b/src/backend/services/gemini_voice.py
@@ -6,6 +6,7 @@
Architecture:
Browser (mic) → WebSocket → Backend → Gemini Live API → Backend → WebSocket → Browser (speaker)
+ Gemini tool_call → Backend._execute_tool → Agent container → tool_response → Gemini
"""
import asyncio
@@ -25,6 +26,34 @@
INPUT_SAMPLE_RATE = 16000 # 16kHz PCM input to Gemini
OUTPUT_SAMPLE_RATE = 24000 # 24kHz PCM output from Gemini
+# Max chars for tool call prompts (prevent injection via very long args)
+_TOOL_PROMPT_MAX = 2000
+
+# Single tool declaration for all voice sessions
+_RUN_TASK_TOOL = genai_types.Tool(
+ function_declarations=[
+ genai_types.FunctionDeclaration(
+ name="run_task",
+ description=(
+ "Execute a task in the agent's workspace — look something up, "
+ "read a file, search for information, or perform an action. "
+ "Use this when you need live data or agent capabilities to answer accurately. "
+ "Returns a text response from the agent."
+ ),
+ parameters=genai_types.Schema(
+ type=genai_types.Type.OBJECT,
+ properties={
+ "prompt": genai_types.Schema(
+ type=genai_types.Type.STRING,
+ description="Clear description of what to look up or do",
+ )
+ },
+ required=["prompt"],
+ ),
+ )
+ ]
+)
+
@dataclass
class VoiceTranscriptEntry:
@@ -47,13 +76,17 @@ class VoiceSession:
_gemini_session: object = field(default=None, repr=False)
_send_task: object = field(default=None, repr=False)
_receive_task: object = field(default=None, repr=False)
+ _timeout_task: object = field(default=None, repr=False)
_audio_in_queue: asyncio.Queue = field(default_factory=asyncio.Queue)
+ _pending_tool_tasks: dict = field(default_factory=dict) # call_id → asyncio.Task
_active: bool = False
_duration_seconds: float = 0.0
# Callbacks
_on_audio_out: Optional[Callable] = field(default=None, repr=False)
_on_transcript: Optional[Callable] = field(default=None, repr=False)
_on_status: Optional[Callable] = field(default=None, repr=False)
+ _on_tool_call: Optional[Callable] = field(default=None, repr=False) # (name, args) → None
+ _on_tool_result: Optional[Callable] = field(default=None, repr=False) # (name, result) → None
class GeminiVoiceService:
@@ -104,13 +137,16 @@ async def connect_and_stream(
session_id: str,
on_audio_out: Callable[[bytes], Awaitable[None]],
on_transcript: Callable[[str, str], Awaitable[None]], # (role, text)
- on_status: Callable[[str], Awaitable[None]], # status string
+ on_status: Callable[[str], Awaitable[None]], # status string
+ on_tool_call: Optional[Callable] = None, # (name, args) → None
+ on_tool_result: Optional[Callable] = None, # (name, result) → None
):
"""
Connect to Gemini Live API and begin streaming.
This is the main loop that runs for the lifetime of the voice session.
It spawns send/receive tasks and waits until the session ends.
+ Tool calls are executed asynchronously against the agent container.
"""
session = self._sessions.get(session_id)
if not session:
@@ -119,6 +155,8 @@ async def connect_and_stream(
session._on_audio_out = on_audio_out
session._on_transcript = on_transcript
session._on_status = on_status
+ session._on_tool_call = on_tool_call
+ session._on_tool_result = on_tool_result
session._active = True
client = self._get_client()
@@ -133,6 +171,7 @@ async def connect_and_stream(
)
)
),
+ tools=[_RUN_TASK_TOOL],
)
try:
@@ -153,7 +192,7 @@ async def connect_and_stream(
session._receive_task = tg.create_task(
self._receive_audio_loop(session)
)
- session._send_task = tg.create_task(
+ session._timeout_task = tg.create_task(
self._timeout_watchdog(session)
)
@@ -187,8 +226,7 @@ async def _send_audio_loop(self, session: VoiceSession):
break
async def _receive_audio_loop(self, session: VoiceSession):
- """Receive audio and transcriptions from Gemini."""
- # Track partial transcriptions for assembling full utterances
+ """Receive audio, transcriptions, and tool calls from Gemini."""
current_user_text = ""
current_assistant_text = ""
@@ -199,6 +237,17 @@ async def _receive_audio_loop(self, session: VoiceSession):
if not session._active:
return
+ # Tool calls — spawn async task per call, keyed by call_id
+ if hasattr(response, 'tool_call') and response.tool_call:
+ fc_list = getattr(response.tool_call, 'function_calls', []) or []
+ for fc in fc_list:
+ call_id = getattr(fc, 'id', None) or secrets.token_hex(8)
+ task = asyncio.create_task(
+ self._execute_and_respond(session, call_id, fc)
+ )
+ session._pending_tool_tasks[call_id] = task
+ continue
+
content = response.server_content
if not content:
continue
@@ -233,7 +282,6 @@ async def _receive_audio_loop(self, session: VoiceSession):
if session._on_status:
await session._on_status("listening")
- # Save completed utterances to transcript
if current_user_text.strip():
session.transcript.append(
VoiceTranscriptEntry(role="user", text=current_user_text.strip())
@@ -262,6 +310,71 @@ async def _receive_audio_loop(self, session: VoiceSession):
VoiceTranscriptEntry(role="assistant", text=current_assistant_text.strip())
)
+ async def _execute_and_respond(self, session: VoiceSession, call_id: str, fc):
+ """Execute a Gemini tool call and send the response back. Runs as a background task."""
+ tool_name = getattr(fc, 'name', 'run_task')
+ args = dict(fc.args) if getattr(fc, 'args', None) else {}
+
+ try:
+ if session._on_tool_call:
+ await session._on_tool_call(tool_name, args)
+
+ result = await asyncio.wait_for(
+ self._execute_tool(session.agent_name, tool_name, args),
+ timeout=30.0,
+ )
+ except asyncio.TimeoutError:
+ result = "Tool execution timed out after 30 seconds."
+ logger.warning(f"Voice tool call timed out: {tool_name} session={session.session_id}")
+ except Exception as e:
+ result = f"Tool error: {str(e)[:200]}"
+ logger.error(f"Voice tool call error: {e}")
+ finally:
+ session._pending_tool_tasks.pop(call_id, None)
+
+ if session._on_tool_result:
+ try:
+ await session._on_tool_result(tool_name, result)
+ except Exception:
+ pass
+
+ if session._gemini_session and session._active:
+ try:
+ await session._gemini_session.send_tool_response(
+ function_responses=[
+ genai_types.FunctionResponse(
+ id=call_id,
+ name=tool_name,
+ response={"output": result},
+ )
+ ]
+ )
+ except Exception as e:
+ logger.error(f"Failed to send tool response for {call_id}: {e}")
+
+ async def _execute_tool(self, agent_name: str, tool_name: str, args: dict) -> str:
+ """Route a tool call to the agent container via the task endpoint."""
+ from services.agent_client import get_agent_client, AgentNotReachableError, AgentRequestError
+
+ prompt = str(args.get("prompt", "")).strip()
+ if not prompt:
+ return "No prompt provided."
+ if len(prompt) > _TOOL_PROMPT_MAX:
+ prompt = prompt[:_TOOL_PROMPT_MAX] + "..."
+
+ logger.info(f"Voice tool call: agent={agent_name} tool={tool_name} prompt={prompt[:80]!r}")
+ try:
+ client = get_agent_client(agent_name)
+ response = await client.task(prompt, timeout=28.0)
+ return response.response or "Task completed with no response."
+ except AgentNotReachableError:
+ return f"Agent {agent_name!r} is not currently running."
+ except AgentRequestError as e:
+ return f"Task error: {str(e)[:200]}"
+ except Exception as e:
+ logger.error(f"Voice tool execution error for {agent_name}: {e}")
+ return f"Execution error: {str(e)[:200]}"
+
async def _timeout_watchdog(self, session: VoiceSession):
"""Auto-end session after max duration."""
await asyncio.sleep(VOICE_MAX_DURATION)
@@ -286,8 +399,14 @@ async def end_session(self, session_id: str) -> Optional[VoiceSession]:
# Send poison pill to unblock send loop
await session._audio_in_queue.put(None)
- # Cancel tasks
- for task in [session._send_task, session._receive_task]:
+ # Cancel pending tool tasks
+ for task in list(session._pending_tool_tasks.values()):
+ if not task.done():
+ task.cancel()
+ session._pending_tool_tasks.clear()
+
+ # Cancel send/receive/timeout tasks
+ for task in [session._send_task, session._receive_task, session._timeout_task]:
if task and not task.done():
task.cancel()
diff --git a/src/backend/services/git_service.py b/src/backend/services/git_service.py
index 642a64789..191b17185 100644
--- a/src/backend/services/git_service.py
+++ b/src/backend/services/git_service.py
@@ -497,6 +497,12 @@ async def sync_to_github(
message="Agent must be running to sync"
)
+ # #462: bring the workspace `.gitignore` up to the current canonical list
+ # and untrack any files that NOW match a rule. Runs on every Push so
+ # existing agents migrate without re-init or container rebuild. Best
+ # effort — failures are logged inside the helper and Push proceeds.
+ await _migrate_workspace_gitignore(agent_name)
+
try:
# Call the agent's internal sync endpoint
async with httpx.AsyncClient(timeout=120.0) as client:
@@ -643,22 +649,53 @@ def delete_agent_git_config(agent_name: str) -> bool:
# Git Initialization in Container
# ============================================================================
-# Hardcoded patterns merged (append-if-missing) into the agent's `.gitignore`
-# at init time. Ordering is preserved in the file so operators reading it see
-# the legacy shell/cache entries first followed by the credential files that
-# `inject_credentials` writes (the trio the #458 bug report named).
+# Canonical exclusion list merged (append-if-missing) into every agent's
+# `.gitignore`. This is the single source of truth — the matching block in
+# `docs/TRINITY_COMPATIBLE_AGENT_GUIDE.md` mirrors it, and a unit test
+# (`test_doc_and_constant_in_sync`) keeps them aligned.
+#
+# Ordering is preserved in the file so operators reading it see entries
+# grouped by category. The list covers the runtime/instance noise the #462
+# bug report named (1,599 files leaked on a single Push) plus the credential
+# files `inject_credentials` writes (the #458 trio).
_GITIGNORE_PATTERNS: Tuple[str, ...] = (
+ # Shell init / history (instance-specific)
".bash_logout",
".bashrc",
".profile",
".bash_history",
+ ".sudo_as_admin_successful",
+ # Credentials — NEVER COMMIT
+ ".env",
+ ".env.*",
+ ".mcp.json",
+ "credentials.json",
+ "*.pem",
+ "*.key",
+ # Instance-specific directories
".cache/",
".local/",
".npm/",
".ssh/",
- ".env",
- ".env.*",
- ".mcp.json",
+ ".trinity/",
+ # Large generated content
+ "content/",
+ # Claude Code runtime — commit commands/skills/agents, exclude runtime data
+ ".claude.json",
+ ".claude.json.backup",
+ ".claude/projects/",
+ ".claude/statsig/",
+ ".claude/todos/",
+ ".claude/debug/",
+ ".claude/sessions/",
+ ".claude/shell-snapshots/",
+ # Temporary files
+ "*.log",
+ "*.tmp",
+ ".DS_Store",
+ # Local overrides
+ "*.local.md",
+ "*.local.json",
)
@@ -676,6 +713,93 @@ def _build_gitignore_merge_command(git_dir: str) -> str:
return f"bash -c {shlex.quote(script)}"
+def _build_rm_cached_ignored_command(git_dir: str) -> str:
+ """Build a bash command that ``git rm --cached``s any tracked files that
+ NOW match an ignore rule. Idempotent — `git ls-files -ci` returns the
+ empty set after the first successful run.
+
+ Two-pass: a non-NUL `git ls-files` to check emptiness via shell variable
+ (bash can't hold NUL bytes), then a NUL-delimited pipe to xargs so paths
+ with spaces or unicode survive the round-trip. Working-tree files are
+ left alone; only the index is touched.
+ """
+ script = (
+ f"cd {shlex.quote(git_dir)} && "
+ "ignored=$(git ls-files -ci --exclude-standard) && "
+ 'if [ -n "$ignored" ]; then '
+ "git ls-files -ci -z --exclude-standard | "
+ "xargs -0 git rm --cached --quiet -r --; "
+ "fi"
+ )
+ return f"bash -c {shlex.quote(script)}"
+
+
+async def _detect_git_dir(container_name: str) -> str:
+ """Pick the directory git operations should run in for an agent container.
+
+ Standard path is ``/home/developer``. Returns ``/home/developer/workspace``
+ only for legacy agents (created before 2026-02) that have content under
+ that subdirectory. Mirrors the detection ``initialize_git_in_container``
+ has always used so init and the post-init migration agree.
+ """
+ check_workspace = await execute_command_in_container(
+ container_name=container_name,
+ command=(
+ 'bash -c "[ -d /home/developer/workspace ] && '
+ 'find /home/developer/workspace -mindepth 1 -maxdepth 1 | '
+ 'head -1 | wc -l"'
+ ),
+ timeout=5,
+ )
+ workspace_has_content = (
+ check_workspace.get("exit_code") == 0
+ and "1" in check_workspace.get("output", "")
+ )
+ return "/home/developer/workspace" if workspace_has_content else "/home/developer"
+
+
+async def _migrate_workspace_gitignore(agent_name: str) -> None:
+ """Idempotently bring an existing agent's `.gitignore` up to the current
+ `_GITIGNORE_PATTERNS` and untrack any files that NOW match a rule.
+
+ Runs on every Push (#462) so existing agents adopt new patterns without
+ requiring a re-init or container rebuild. Errors are logged and swallowed
+ — a transient migration failure must not break an operator's Push.
+
+ No-op if the container has no `.git` directory (agent not initialized for
+ git sync).
+ """
+ container_name = f"agent-{agent_name}"
+ try:
+ git_dir = await _detect_git_dir(container_name)
+ # Bail if not git-initialized — the agent's /api/git/sync will
+ # return its own 400 in that case.
+ check_git = await execute_command_in_container(
+ container_name=container_name,
+ command=f'bash -c "[ -d {shlex.quote(git_dir)}/.git ]"',
+ timeout=5,
+ )
+ if check_git.get("exit_code") != 0:
+ return
+ # 1. Append missing patterns (idempotent).
+ await execute_command_in_container(
+ container_name=container_name,
+ command=_build_gitignore_merge_command(git_dir),
+ timeout=10,
+ )
+ # 2. Untrack any indexed files that now match an ignore rule.
+ await execute_command_in_container(
+ container_name=container_name,
+ command=_build_rm_cached_ignored_command(git_dir),
+ timeout=30,
+ )
+ except Exception as exc:
+ logger.warning(
+ f"_migrate_workspace_gitignore failed for {agent_name}: {exc}. "
+ "Push will proceed against the existing .gitignore."
+ )
+
+
@dataclass
class GitInitResult:
"""Result of git initialization in container."""
@@ -724,28 +848,13 @@ async def initialize_git_in_container(
"""
container_name = f"agent-{agent_name}"
- # Step 1: Determine git directory
- # NOTE: All agents use /home/developer as their home directory.
- # The /home/developer/workspace check is LEGACY support for agents created before 2026-02.
- # New agents should never have a workspace subdirectory.
- check_workspace = await execute_command_in_container(
- container_name=container_name,
- command='bash -c "[ -d /home/developer/workspace ] && find /home/developer/workspace -mindepth 1 -maxdepth 1 | head -1 | wc -l"',
- timeout=5
- )
-
- workspace_has_content = (
- check_workspace.get("exit_code") == 0 and
- "1" in check_workspace.get("output", "")
- )
-
- if workspace_has_content:
- # Legacy agent with workspace subdirectory
- git_dir = "/home/developer/workspace"
+ # Step 1: Determine git directory (workspace for legacy agents, else home).
+ # Detection logic is shared with `_migrate_workspace_gitignore` so the
+ # post-init Push migration targets the same path.
+ git_dir = await _detect_git_dir(container_name)
+ if git_dir == "/home/developer/workspace":
logger.info(f"[LEGACY] Using workspace directory with existing content: {git_dir}")
else:
- # Standard path for all current agents
- git_dir = "/home/developer"
logger.info(f"Using home directory: {git_dir}")
# Step 2: Append any missing `_GITIGNORE_PATTERNS` entries to the
diff --git a/src/backend/services/mcp_validator.py b/src/backend/services/mcp_validator.py
new file mode 100644
index 000000000..1e9041bbe
--- /dev/null
+++ b/src/backend/services/mcp_validator.py
@@ -0,0 +1,573 @@
+"""
+MCP server config validator (#598, Layer 2 of AISEC-C2 closure).
+
+Re-allows `.mcp.json` content through `POST /api/agents/{name}/credentials/inject`
+ONLY when the structure passes strict validation. Layer 1 (#590) closed the
+RCE-by-config bypass by removing `.mcp.json` from the inject allowlist; this
+module restores the legitimate use case (owners adding/editing MCP servers
+post-deploy) while keeping the attack surface closed.
+
+Public API:
+ validate_mcp_config(content: str) -> None
+ Raises McpValidationError on any rejection.
+
+ class McpValidationError(ValueError):
+ Distinct exception for the router to surface as 400 Bad Request.
+
+Threat model:
+ - Attacker is an authenticated agent OWNER (already has the JWT).
+ - Goal: defense in depth against shell-injection patterns and the
+ AISEC-C2 exact reproduction. Does NOT prevent owners from running
+ malicious code via approved runtimes (npx ) — that's
+ Layer 3 (sandbox MCP execution).
+
+Architecture (SOLID at appropriate scale — single file, internal classes):
+ validate_mcp_config()
+ └─ _validate_servers_dict() schema + per-entry dispatch
+ └─ _ENTRY_VALIDATORS_BY_TRANSPORT[transport].validate(name, server)
+ ├─ _StdioValidator command + args + env
+ ├─ _HttpValidator url + headers + env (+ SSRF)
+ └─ _SseValidator subclass of _HttpValidator (semantically
+ distinct, same rules)
+"""
+from __future__ import annotations
+
+import ipaddress
+import json
+import re
+import socket
+from typing import Mapping
+from urllib.parse import urlparse
+
+
+# ---------------------------------------------------------------------------
+# Public exception
+# ---------------------------------------------------------------------------
+
+
+class McpValidationError(ValueError):
+ """Raised when an MCP server config fails validation.
+
+ Routers translate this to HTTP 400. The message is included verbatim in
+ the response, so it must be safe to surface to the caller (no internal
+ paths or stack traces).
+ """
+
+
+# ---------------------------------------------------------------------------
+# Constants — tunable in one place
+# ---------------------------------------------------------------------------
+
+# Maximum size of the rendered `.mcp.json` content. 64KB is ~10x what a
+# realistic config needs and keeps validation O(n) bounded.
+MAX_CONTENT_BYTES = 64 * 1024
+
+# Maximum number of mcpServers entries. Real configs have <10; cap defends
+# against pathological JSON.
+MAX_SERVER_COUNT = 32
+
+# Server name (the dict key under mcpServers): conservative ASCII rule. Long
+# enough for realistic identifiers, short enough to be UI-safe.
+_SERVER_NAME_RE = re.compile(r"^[a-zA-Z0-9_-]{1,64}$")
+
+# Reserved server names — owners cannot overwrite Trinity's auto-injected
+# entry, which would break agent-to-agent collaboration.
+RESERVED_SERVER_NAMES = frozenset({"trinity"})
+
+# Allowed values for the `transport` field. `stdio` is implicit when only
+# `command` is present (we set transport explicitly during validation).
+ALLOWED_TRANSPORTS = frozenset({"stdio", "http", "sse"})
+
+# Stdio runtime allowlist. Each entry is the EXACT command name the user can
+# specify; absolute paths and alternates are rejected. `python3` listed
+# separately from `python` because both are real on different distros.
+COMMAND_ALLOWLIST = frozenset({
+ "npx", "uvx", "python", "python3", "node", "bun", "deno", "docker",
+})
+
+# Per-runtime "execution flags" that turn the runtime into a shell. We block
+# these as the FIRST positional arg (which would replace the script/package
+# arg with inline code). Owners with a real need (`-c "import …"`) should
+# package as a script and reference it instead.
+_INLINE_EXEC_FLAGS_BY_COMMAND: Mapping[str, frozenset[str]] = {
+ "python": frozenset({"-c", "--command"}),
+ "python3": frozenset({"-c", "--command"}),
+ "node": frozenset({"-e", "--eval", "-p", "--print"}),
+ "bun": frozenset({"-e", "--eval"}),
+ "deno": frozenset({"eval"}),
+ # npx / uvx don't have an inline-exec flag; the first positional IS the
+ # package name. Likewise docker.
+}
+
+# Shell metacharacters that have no business in any arg passed to a runtime
+# allowlisted above. Each runtime invokes its target via execve, not a shell,
+# so these characters never need to appear unescaped in real configs.
+_SHELL_METACHARS_RE = re.compile(r"[;&|<>`$\n\r\x00]")
+
+# Substring patterns that indicate command substitution. `re.search` finds
+# them anywhere in the arg.
+_COMMAND_SUBSTITUTION_RE = re.compile(r"\$\([^)]+\)|`[^`]+`")
+
+# Env var names allowed as `${VAR}` references in args/env values. ASCII
+# uppercase + digits + underscore, must start with a letter — standard
+# POSIX shape; rejects `${PATH}` at a separate gate.
+_ENV_VAR_REF_RE = re.compile(r"^[A-Z][A-Z0-9_]*$")
+
+# Env vars the user must NOT reference — overriding any of these from an
+# attacker-controlled `.mcp.json` could change library/binary loading,
+# attach an interpreter, or hijack subsequent process launches.
+RESERVED_ENV_REFS = frozenset({
+ "PATH", "HOME", "USER", "SHELL",
+ "LD_PRELOAD", "LD_LIBRARY_PATH", "DYLD_INSERT_LIBRARIES",
+ "PYTHONPATH", "PYTHONSTARTUP", "PYTHONHOME",
+ "NODE_OPTIONS", "NODE_PATH",
+ # Trinity-internal — the agent server reads these on startup
+ "TRINITY_MCP_API_KEY", "TRINITY_MCP_URL", "ADMIN_PASSWORD",
+ "SECRET_KEY", "INTERNAL_API_SECRET", "CREDENTIAL_ENCRYPTION_KEY",
+ "CLAUDE_CODE_OAUTH_TOKEN", "ANTHROPIC_API_KEY",
+})
+
+# Patterns that look like raw secrets accidentally pasted into a config —
+# reject with a clear message rather than silently accepting them. Mirrors
+# `credential_patterns` in docker/base-image/hooks/guardrails-baseline.json.
+_LITERAL_SECRET_PATTERNS = (
+ re.compile(r"sk-ant-[a-zA-Z0-9_-]{20,}"),
+ re.compile(r"sk-ant-oat01-[a-zA-Z0-9_-]{20,}"),
+ re.compile(r"sk-(?:proj-)?[a-zA-Z0-9]{32,}"),
+ re.compile(r"ghp_[a-zA-Z0-9]{30,}"),
+ re.compile(r"github_pat_[a-zA-Z0-9_]{40,}"),
+ re.compile(r"AKIA[0-9A-Z]{16}"),
+ re.compile(r"xox[baprs]-[0-9a-zA-Z-]{20,}"),
+ re.compile(r"AIza[0-9A-Za-z_-]{35}"),
+)
+
+# Header names allowed in http/sse server entries. Limit to the small set
+# real servers actually use to avoid weird header smuggling vectors.
+_ALLOWED_HEADER_NAMES = frozenset({
+ "authorization", "x-api-key", "user-agent", "accept", "content-type",
+})
+
+
+# ---------------------------------------------------------------------------
+# Helpers (private)
+# ---------------------------------------------------------------------------
+
+
+def _is_printable_ascii(s: str) -> bool:
+ """Strict ASCII printable check (defeats Unicode lookalikes + null bytes)."""
+ return all(0x20 <= ord(c) <= 0x7E for c in s)
+
+
+# `${VAR}` substring used to extract refs from values like `Bearer ${TOKEN}`.
+# Any reference found must satisfy `_ENV_VAR_REF_RE` shape AND not be in
+# `RESERVED_ENV_REFS`. The remaining literal portion (with refs stripped)
+# is then checked for shell metacharacters.
+_ENV_VAR_SUBSTRING_RE = re.compile(r"\$\{([^}]*)\}")
+
+
+def _validate_env_value(server_name: str, key: str, value: object) -> None:
+ """Validate one env value.
+
+ Allowed shapes (covers real-world cases like `Bearer ${API_TOKEN}`,
+ `${OPENAI_BASE_URL}/v1`, plain literal URLs, plain `${VAR}` refs):
+ - Any number of `${VAR}` substring references, each with a valid
+ var name that is NOT in RESERVED_ENV_REFS
+ - The remaining literal portion (refs stripped) is checked for shell
+ metacharacters and command substitution
+
+ Reject:
+ - non-string values, oversize values
+ - command substitution patterns (`$(…)` or backticks)
+ - shell metacharacters in the literal portion
+ - literal secrets (defense against accidental paste)
+ - malformed or reserved `${VAR}` references
+ """
+ if not isinstance(value, str):
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] must be a string"
+ )
+ if len(value) > 4096:
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] exceeds 4096 chars"
+ )
+
+ # Pull out every `${VAR}` reference; each must be safe.
+ refs = _ENV_VAR_SUBSTRING_RE.findall(value)
+ for var_name in refs:
+ if not _ENV_VAR_REF_RE.match(var_name):
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] references malformed "
+ f"variable name '{var_name}'"
+ )
+ if var_name in RESERVED_ENV_REFS:
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] references reserved "
+ f"variable '{var_name}'"
+ )
+
+ # Strip refs to get the literal portion, then apply shell-safety checks
+ # to it. This way `Bearer ${TOKEN}` validates as `Bearer ` (safe).
+ literal = _ENV_VAR_SUBSTRING_RE.sub("", value)
+
+ if _COMMAND_SUBSTITUTION_RE.search(literal):
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] contains command "
+ f"substitution"
+ )
+ if _SHELL_METACHARS_RE.search(literal):
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] contains shell "
+ f"metacharacters"
+ )
+ for pat in _LITERAL_SECRET_PATTERNS:
+ if pat.search(literal):
+ raise McpValidationError(
+ f"Server '{server_name}': env['{key}'] looks like a literal "
+ f"secret — store it in .env and reference as ${{VAR}}"
+ )
+
+
+def _resolves_to_private_ip(hostname: str) -> bool:
+ """Best-effort DNS check (mirrors SEC-179 / #179).
+
+ Returns True if the hostname resolves to ANY private/loopback/link-local/
+ multicast IP. On DNS failure: True (fail closed). Used to block IMDS,
+ localhost, RFC 1918, and similar SSRF targets.
+ """
+ try:
+ # getaddrinfo returns a list of (family, type, proto, canonname, sockaddr).
+ # We only need the IP from sockaddr.
+ infos = socket.getaddrinfo(hostname, None)
+ except socket.gaierror:
+ return True # fail closed
+ for info in infos:
+ sockaddr = info[4]
+ try:
+ ip = ipaddress.ip_address(sockaddr[0])
+ except (ValueError, IndexError):
+ continue
+ if (
+ ip.is_private
+ or ip.is_loopback
+ or ip.is_link_local
+ or ip.is_multicast
+ or ip.is_reserved
+ or ip.is_unspecified
+ ):
+ return True
+ return False
+
+
+# ---------------------------------------------------------------------------
+# Transport validators
+# ---------------------------------------------------------------------------
+
+
+class _StdioValidator:
+ """Stdio-transport server: command + args + env."""
+
+ @staticmethod
+ def validate(server_name: str, server: dict) -> None:
+ if "command" not in server:
+ raise McpValidationError(
+ f"Server '{server_name}' (stdio): missing required field 'command'"
+ )
+ command = server["command"]
+ if not isinstance(command, str) or not command:
+ raise McpValidationError(
+ f"Server '{server_name}': command must be a non-empty string"
+ )
+ if "/" in command or "\\" in command:
+ raise McpValidationError(
+ f"Server '{server_name}': command must be a name, not a path "
+ f"(got '{command}')"
+ )
+ if not _is_printable_ascii(command):
+ raise McpValidationError(
+ f"Server '{server_name}': command contains non-ASCII or "
+ f"control characters"
+ )
+ if command not in COMMAND_ALLOWLIST:
+ raise McpValidationError(
+ f"Server '{server_name}': command '{command}' not in allowlist. "
+ f"Allowed: {sorted(COMMAND_ALLOWLIST)}"
+ )
+
+ args = server.get("args", [])
+ if not isinstance(args, list):
+ raise McpValidationError(
+ f"Server '{server_name}': args must be a list"
+ )
+ if len(args) > 64:
+ raise McpValidationError(
+ f"Server '{server_name}': args list too long (max 64)"
+ )
+
+ # Block the inline-exec flag as the FIRST positional. Later positions
+ # would be bare strings the runtime treats as data.
+ inline_flags = _INLINE_EXEC_FLAGS_BY_COMMAND.get(command, frozenset())
+ for i, arg in enumerate(args):
+ if not isinstance(arg, str):
+ raise McpValidationError(
+ f"Server '{server_name}': args[{i}] must be a string"
+ )
+ if len(arg) > 1024:
+ raise McpValidationError(
+ f"Server '{server_name}': args[{i}] exceeds 1024 chars"
+ )
+ if "\x00" in arg:
+ raise McpValidationError(
+ f"Server '{server_name}': args[{i}] contains null byte"
+ )
+ if _COMMAND_SUBSTITUTION_RE.search(arg):
+ raise McpValidationError(
+ f"Server '{server_name}': args[{i}] contains command "
+ f"substitution"
+ )
+ if _SHELL_METACHARS_RE.search(arg):
+ raise McpValidationError(
+ f"Server '{server_name}': args[{i}] contains shell "
+ f"metacharacters"
+ )
+ if i == 0 and arg in inline_flags:
+ raise McpValidationError(
+ f"Server '{server_name}': inline-exec flag '{arg}' not "
+ f"allowed; package the code as a script and reference its path"
+ )
+
+ env = server.get("env", {})
+ if not isinstance(env, dict):
+ raise McpValidationError(
+ f"Server '{server_name}': env must be an object"
+ )
+ if len(env) > 64:
+ raise McpValidationError(
+ f"Server '{server_name}': env has too many entries (max 64)"
+ )
+ for key, value in env.items():
+ if not isinstance(key, str) or not _ENV_VAR_REF_RE.match(key):
+ raise McpValidationError(
+ f"Server '{server_name}': env key '{key}' must match "
+ f"^[A-Z][A-Z0-9_]*$"
+ )
+ _validate_env_value(server_name, key, value)
+
+
+class _HttpValidator:
+ """HTTP-transport server: url + headers + env."""
+
+ transport_label = "http"
+
+ @classmethod
+ def validate(cls, server_name: str, server: dict) -> None:
+ if "url" not in server:
+ raise McpValidationError(
+ f"Server '{server_name}' ({cls.transport_label}): "
+ f"missing required field 'url'"
+ )
+ url = server["url"]
+ if not isinstance(url, str) or len(url) > 2048:
+ raise McpValidationError(
+ f"Server '{server_name}': url must be a string < 2048 chars"
+ )
+
+ try:
+ parsed = urlparse(url)
+ except ValueError as e:
+ raise McpValidationError(
+ f"Server '{server_name}': invalid url ({e})"
+ )
+
+ if parsed.scheme != "https":
+ raise McpValidationError(
+ f"Server '{server_name}': url must use https (got '{parsed.scheme}')"
+ )
+ # Reject userinfo (`https://user:pass@evil.com/...`) which can confuse
+ # display-only auditors and is never needed for MCP.
+ if "@" in (parsed.netloc or ""):
+ raise McpValidationError(
+ f"Server '{server_name}': url must not contain userinfo (@)"
+ )
+ hostname = (parsed.hostname or "").lower()
+ if not hostname:
+ raise McpValidationError(
+ f"Server '{server_name}': url missing hostname"
+ )
+ if not _is_printable_ascii(hostname):
+ raise McpValidationError(
+ f"Server '{server_name}': url hostname contains non-ASCII "
+ f"(possible homograph)"
+ )
+ if _resolves_to_private_ip(hostname):
+ raise McpValidationError(
+ f"Server '{server_name}': url hostname '{hostname}' resolves "
+ f"to a private/loopback/link-local address (SSRF guard)"
+ )
+
+ headers = server.get("headers", {})
+ if not isinstance(headers, dict):
+ raise McpValidationError(
+ f"Server '{server_name}': headers must be an object"
+ )
+ if len(headers) > 16:
+ raise McpValidationError(
+ f"Server '{server_name}': headers has too many entries (max 16)"
+ )
+ for key, value in headers.items():
+ if not isinstance(key, str):
+ raise McpValidationError(
+ f"Server '{server_name}': header name must be a string"
+ )
+ if key.lower() not in _ALLOWED_HEADER_NAMES:
+ raise McpValidationError(
+ f"Server '{server_name}': header '{key}' not in allowlist. "
+ f"Allowed: {sorted(_ALLOWED_HEADER_NAMES)}"
+ )
+ # Header values reuse the env-value rules: ${VAR} or safe literal.
+ _validate_env_value(server_name, f"headers.{key}", value)
+
+ # http/sse can also carry env (rare but allowed by the MCP spec).
+ env = server.get("env", {})
+ if not isinstance(env, dict):
+ raise McpValidationError(
+ f"Server '{server_name}': env must be an object"
+ )
+ for key, value in env.items():
+ if not isinstance(key, str) or not _ENV_VAR_REF_RE.match(key):
+ raise McpValidationError(
+ f"Server '{server_name}': env key '{key}' must match "
+ f"^[A-Z][A-Z0-9_]*$"
+ )
+ _validate_env_value(server_name, key, value)
+
+
+class _SseValidator(_HttpValidator):
+ """SSE-transport server. Identical rules to HTTP; separate class for
+ diagnostic clarity in error messages.
+ """
+ transport_label = "sse"
+
+
+# Dispatch table — Open-Closed: add a new transport by adding a class and
+# one entry here, no edits elsewhere.
+_ENTRY_VALIDATORS_BY_TRANSPORT = {
+ "stdio": _StdioValidator,
+ "http": _HttpValidator,
+ "sse": _SseValidator,
+}
+
+
+# ---------------------------------------------------------------------------
+# Entry / config orchestration
+# ---------------------------------------------------------------------------
+
+
+def _resolve_transport(server_name: str, server: dict) -> str:
+ """Determine the transport for an entry.
+
+ The MCP config spec lets transport be implicit:
+ - stdio: presence of `command`
+ - http/sse: presence of `url` + explicit `type` field
+ We require the `type` field for http/sse to avoid ambiguity, and accept
+ `command` as an implicit stdio signal.
+ """
+ explicit = server.get("type")
+ if explicit is not None:
+ if not isinstance(explicit, str) or explicit not in ALLOWED_TRANSPORTS:
+ raise McpValidationError(
+ f"Server '{server_name}': type must be one of "
+ f"{sorted(ALLOWED_TRANSPORTS)}"
+ )
+ return explicit
+ # No explicit type → infer
+ if "command" in server:
+ return "stdio"
+ if "url" in server:
+ raise McpValidationError(
+ f"Server '{server_name}': url provided without 'type' field; "
+ f"set type to 'http' or 'sse'"
+ )
+ raise McpValidationError(
+ f"Server '{server_name}': cannot determine transport (no command, "
+ f"no url, no type)"
+ )
+
+
+def _validate_entry(server_name: str, server: object) -> None:
+ """Validate a single MCP server entry."""
+ if not isinstance(server_name, str):
+ raise McpValidationError("MCP server name must be a string")
+ if not _SERVER_NAME_RE.match(server_name):
+ raise McpValidationError(
+ f"MCP server name '{server_name}' invalid; must match "
+ f"^[a-zA-Z0-9_-]{{1,64}}$"
+ )
+ if server_name in RESERVED_SERVER_NAMES:
+ raise McpValidationError(
+ f"MCP server name '{server_name}' is reserved by Trinity"
+ )
+ if not isinstance(server, dict):
+ raise McpValidationError(
+ f"MCP server '{server_name}' must be a JSON object"
+ )
+
+ transport = _resolve_transport(server_name, server)
+ validator = _ENTRY_VALIDATORS_BY_TRANSPORT[transport]
+ validator.validate(server_name, server)
+
+ # Reject any unknown top-level fields. Closed schema = no surprise fields
+ # that future MCP versions might interpret in unexpected ways.
+ allowed_keys = {"command", "args", "env", "url", "headers", "type"}
+ extra = set(server.keys()) - allowed_keys
+ if extra:
+ raise McpValidationError(
+ f"Server '{server_name}': unknown field(s) {sorted(extra)}; "
+ f"allowed: {sorted(allowed_keys)}"
+ )
+
+
+def _validate_servers_dict(servers: dict) -> None:
+ """Validate the top-level mcpServers dict."""
+ if len(servers) > MAX_SERVER_COUNT:
+ raise McpValidationError(
+ f"Too many MCP servers ({len(servers)}); max {MAX_SERVER_COUNT}"
+ )
+ for name, entry in servers.items():
+ _validate_entry(name, entry)
+
+
+def validate_mcp_config(content: str) -> None:
+ """Validate the rendered `.mcp.json` content string.
+
+ Raises McpValidationError with a single human-readable message on the
+ first failure. Routers should surface the message in the HTTP 400 body.
+ """
+ if not isinstance(content, str):
+ raise McpValidationError(".mcp.json content must be a string")
+ if len(content.encode("utf-8")) > MAX_CONTENT_BYTES:
+ raise McpValidationError(
+ f".mcp.json content exceeds {MAX_CONTENT_BYTES} bytes"
+ )
+
+ try:
+ config = json.loads(content)
+ except json.JSONDecodeError as e:
+ raise McpValidationError(f".mcp.json is not valid JSON: {e.msg}")
+
+ if not isinstance(config, dict):
+ raise McpValidationError(".mcp.json root must be a JSON object")
+
+ # Allow only `mcpServers` at the root. Other top-level keys (e.g. legacy
+ # `inputs`, future spec fields) are rejected to keep the schema closed.
+ extra_root = set(config.keys()) - {"mcpServers"}
+ if extra_root:
+ raise McpValidationError(
+ f".mcp.json has unknown top-level field(s) {sorted(extra_root)}; "
+ f"only 'mcpServers' is allowed"
+ )
+
+ servers = config.get("mcpServers", {})
+ if not isinstance(servers, dict):
+ raise McpValidationError(".mcp.json mcpServers must be an object")
+
+ _validate_servers_dict(servers)
diff --git a/src/backend/services/platform_prompt_service.py b/src/backend/services/platform_prompt_service.py
index 375fcefa0..10fd653b0 100644
--- a/src/backend/services/platform_prompt_service.py
+++ b/src/backend/services/platform_prompt_service.py
@@ -40,6 +40,16 @@
**Note**: You can only communicate with agents you have been granted permission to access.
Use `list_agents` to discover your available collaborators.
+### Sharing Files with Users
+
+When the user asks for a file (CSV, PDF, report, image, exported data, etc.) or when your answer is best delivered as a file instead of inline text:
+
+1. Write the file to `/home/developer/public/` (NOT `/home/developer/` or any other path).
+2. Call the `mcp__trinity__share_file` MCP tool with the relative filename.
+3. Include the returned `url` in your reply as-is.
+
+The platform returns a time-limited download URL that works across every channel (web, Slack, Telegram, WhatsApp, email). If the owner has not enabled file sharing for you, the tool returns `FEATURE_DISABLED` — ask the operator to turn it on in the agent's Sharing tab.
+
### Operator Communication
You can communicate with your human operator through a file-based queue protocol. This is useful when you need human input — approvals, answers to questions, or to flag important situations.
diff --git a/src/backend/services/pre_check_service.py b/src/backend/services/pre_check_service.py
new file mode 100644
index 000000000..d18738894
--- /dev/null
+++ b/src/backend/services/pre_check_service.py
@@ -0,0 +1,94 @@
+"""
+Pre-Check Service for Trinity platform (SCHED-COND-001 / #454).
+
+Runs the agent's optional template-supplied pre-check hook
+``~/.trinity/pre-check`` via ``docker exec`` and returns a normalized
+contract dict. The hook is language-agnostic — interpreter is selected
+by the file's shebang, not by Trinity.
+
+Called by ``routers/internal.py`` on behalf of the dedicated scheduler
+before each cron-triggered fire. Reuses ``execute_command_in_container``
+(the same primitive as ``git_service``, ``ssh_service``,
+``agent_service/terminal``, Slack ingest, etc.) — no new HTTP edge from
+backend to agent-server, no new long-lived process.
+"""
+from __future__ import annotations
+
+import logging
+from typing import Dict
+
+from services.docker_service import (
+ execute_command_in_container,
+ get_agent_container,
+)
+
+logger = logging.getLogger(__name__)
+
+
+# Convention: language-agnostic executable shipped by the template.
+# Interpreter is selected by the shebang line; the file must be marked +x.
+HOOK_PATH = "/home/developer/.trinity/pre-check"
+
+# Stdout becomes the chat prompt — 32 KB is plenty even for verbose scan output.
+STDOUT_CAP = 32_000
+STDERR_CAP = 4_000
+
+EXISTENCE_TIMEOUT_S = 5
+EXEC_TIMEOUT_S = 60
+
+
+class AgentNotFound(Exception):
+ """Raised when the target agent has no running container."""
+
+
+async def run_pre_check(agent_name: str) -> Dict:
+ """Run the agent's optional pre-check hook.
+
+ Two-step exec:
+ 1. ``test -f`` (5s) — file presence check. Note ``-f``, not ``-x``:
+ a present-but-non-executable file surfaces as exec failure
+ (exit 126) so the operator gets a signal, instead of silently
+ falling through to the backward-compat "no hook" path.
+ 2. Run the hook directly (60s). Trinity does not prefix ``python3``
+ — the shebang determines interpreter.
+
+ Returns one of:
+ ``{"hook_present": False}``
+ Template ships no hook. Caller should fire as usual.
+
+ ``{"hook_present": True, "exit_code": int, "stdout": str, "stderr": str}``
+ Hook ran. Caller translates per the SCHED-COND-001 contract:
+ - exit != 0 → fail-open + log (broken hook must not suppress work)
+ - exit 0, empty stdout → record skip
+ - exit 0, non-empty stdout → fire with stdout as override message
+
+ Raises:
+ AgentNotFound: if no running container for ``agent_name``.
+ """
+ if not get_agent_container(agent_name):
+ raise AgentNotFound(agent_name)
+
+ container_name = f"agent-{agent_name}"
+
+ exists = await execute_command_in_container(
+ container_name=container_name,
+ command=f"test -f {HOOK_PATH}",
+ timeout=EXISTENCE_TIMEOUT_S,
+ )
+ if exists.get("exit_code") != 0:
+ return {"hook_present": False}
+
+ result = await execute_command_in_container(
+ container_name=container_name,
+ command=HOOK_PATH,
+ timeout=EXEC_TIMEOUT_S,
+ )
+ # `output` from container_exec_run is the combined stream — keep
+ # `stdout`/`stderr` as separate fields for forward-compat.
+ output = (result.get("output") or "")[: STDOUT_CAP + STDERR_CAP]
+ return {
+ "hook_present": True,
+ "exit_code": int(result.get("exit_code", 1)),
+ "stdout": output[:STDOUT_CAP],
+ "stderr": "",
+ }
diff --git a/src/backend/services/subscription_auto_switch.py b/src/backend/services/subscription_auto_switch.py
index 40db4d188..ad833970e 100644
--- a/src/backend/services/subscription_auto_switch.py
+++ b/src/backend/services/subscription_auto_switch.py
@@ -1,18 +1,27 @@
"""
Subscription Auto-Switch Service (SUB-003).
-Automatically switches an agent to a different subscription when it
-encounters repeated rate-limit errors (2+ consecutive).
+Automatically switches an agent to a different subscription on the first
+subscription failure — either a rate-limit (429) or an auth-class error
+(401/403/credit balance/expired token, etc.).
Preconditions (all must be true):
-1. Setting "auto_switch_subscriptions" is enabled
+1. Setting "auto_switch_subscriptions" is enabled (default: on, opt-out)
2. Agent has a subscription assigned (not API key)
-3. 2+ consecutive rate-limit errors on this (agent, subscription)
+3. At least one rate-limit / auth event recorded for this (agent, subscription)
4. At least one alternative subscription is available and not rate-limited
+
+Threshold note (#441): pre-#441 we required 2+ consecutive 429s before
+switching. That guaranteed at least one user-visible failure on long-running
+schedules and never fired on auth-class breakage at all. The 2h skip-list on
+alternative selection (`select_best_alternative_subscription` +
+`is_subscription_rate_limited`) is what prevents thrashing — see
+`tests/unit/test_subscription_auto_switch_pingpong.py` for the regression
+tests pinning that contract.
"""
import logging
-from typing import Optional, Tuple
+from typing import Optional
from database import db
from db_models import NotificationCreate
@@ -20,20 +29,53 @@
logger = logging.getLogger(__name__)
-async def handle_rate_limit_error(
+# Substrings that indicate an auth-class subscription failure. Mirrors the
+# scheduler's classification at `src/scheduler/service.py` (which now imports
+# this same list to keep the two surfaces from drifting).
+AUTH_INDICATORS = [
+ "credit balance",
+ "unauthorized",
+ "authentication",
+ "credentials",
+ "forbidden",
+ "401",
+ "403",
+ "oauth",
+ "token expired",
+ "not authenticated",
+]
+
+
+def is_auth_failure(error_message: str) -> bool:
+ """Return True if `error_message` contains any AUTH_INDICATORS substring."""
+ if not error_message:
+ return False
+ error_lower = error_message.lower()
+ return any(ind in error_lower for ind in AUTH_INDICATORS)
+
+
+async def handle_subscription_failure(
agent_name: str,
error_message: str = "",
+ failure_kind: str = "rate_limit",
) -> Optional[dict]:
"""
- Called when a 429 rate-limit error is received from an agent.
+ Called when a subscription-backed agent fails with either a rate-limit (429)
+ or an auth-class error.
- Records the event and triggers auto-switch if all preconditions are met.
+ Records the event and triggers auto-switch on the first occurrence (subject
+ to the alternative being viable per the 2h skip-list).
+
+ Args:
+ agent_name: name of the agent that failed
+ error_message: server-side error string for audit + notification text
+ failure_kind: "rate_limit" (429) or "auth" (401/403/credit/etc.)
Returns:
dict with switch details if auto-switch occurred, None otherwise.
"""
- # 1. Check if auto-switch is enabled
- enabled = db.get_setting_value("auto_switch_subscriptions", default="false") == "true"
+ # 1. Check if auto-switch is enabled (default: on, #441)
+ enabled = db.get_setting_value("auto_switch_subscriptions", default="true") == "true"
if not enabled:
return None
@@ -42,30 +84,28 @@ async def handle_rate_limit_error(
if not current_sub_id:
return None
- # 3. Record the rate-limit event and get consecutive count
+ # 3. Record the failure event in the rate-limit table. Auth-class events
+ # share the same table — the table tracks "subscription failure events"
+ # generically; `is_subscription_rate_limited` treats any event in the 2h
+ # window as a reason to skip the subscription as a candidate, which is the
+ # behavior we want for both kinds of failure.
consecutive_count = db.record_rate_limit_event(
agent_name=agent_name,
subscription_id=current_sub_id,
- error_message=error_message
+ error_message=error_message,
)
- if consecutive_count < 2:
- logger.info(
- f"[SUB-003] Rate-limit event #{consecutive_count} for agent '{agent_name}' "
- f"on subscription {current_sub_id} — waiting for 2+ before auto-switch"
- )
- return None
-
# 4. Find a viable alternative subscription
alternative = db.select_best_alternative_subscription(current_sub_id)
if not alternative:
logger.warning(
- f"[SUB-003] Agent '{agent_name}' has {consecutive_count} consecutive rate-limit errors "
- f"but no viable alternative subscription found"
+ f"[SUB-003] Agent '{agent_name}' hit a {failure_kind} failure on "
+ f"subscription {current_sub_id} (event #{consecutive_count}) "
+ f"but no viable alternative subscription is available"
)
return None
- # Get current subscription name for logging
+ # Get current subscription name for logging / notification
current_sub = db.get_subscription(current_sub_id)
old_name = current_sub.name if current_sub else current_sub_id
@@ -75,23 +115,48 @@ async def handle_rate_limit_error(
old_subscription_id=current_sub_id,
old_subscription_name=old_name,
new_subscription=alternative,
- consecutive_count=consecutive_count,
+ failure_kind=failure_kind,
+ event_count=consecutive_count,
)
+async def handle_rate_limit_error(
+ agent_name: str,
+ error_message: str = "",
+) -> Optional[dict]:
+ """Backward-compatible shim — delegates to `handle_subscription_failure`
+ with `failure_kind="rate_limit"`. Existing 429 callers don't need to
+ migrate atomically.
+ """
+ return await handle_subscription_failure(
+ agent_name=agent_name,
+ error_message=error_message,
+ failure_kind="rate_limit",
+ )
+
+
+def _failure_phrase(failure_kind: str) -> str:
+ """Notification + log wording per failure kind."""
+ if failure_kind == "auth":
+ return "an authentication failure"
+ return "a rate-limit error"
+
+
async def _perform_auto_switch(
agent_name: str,
old_subscription_id: str,
old_subscription_name: str,
new_subscription,
- consecutive_count: int,
+ failure_kind: str,
+ event_count: int,
) -> dict:
"""
Execute the subscription switch: DB update, container restart, log, notify.
"""
+ phrase = _failure_phrase(failure_kind)
logger.info(
f"[SUB-003] Auto-switching agent '{agent_name}' from '{old_subscription_name}' "
- f"to '{new_subscription.name}' after {consecutive_count} consecutive rate-limit errors"
+ f"to '{new_subscription.name}' after {phrase}"
)
# Switch subscription in DB
@@ -122,14 +187,15 @@ async def _perform_auto_switch(
"action": "subscription_auto_switch",
"old_subscription": old_subscription_name,
"new_subscription": new_subscription.name,
- "consecutive_errors": consecutive_count,
+ "failure_kind": failure_kind,
+ "event_count": event_count,
"restart_result": restart_result,
- }
+ },
)
await activity_service.complete_activity(
activity_id=activity_id,
status=ActivityState.COMPLETED,
- details={"message": f"Auto-switched from '{old_subscription_name}' to '{new_subscription.name}'"}
+ details={"message": f"Auto-switched from '{old_subscription_name}' to '{new_subscription.name}'"},
)
# Send notification to agent owner
@@ -141,16 +207,16 @@ async def _perform_auto_switch(
title=f"Subscription auto-switched to '{new_subscription.name}'",
message=(
f"Agent '{agent_name}' was automatically switched from subscription "
- f"'{old_subscription_name}' to '{new_subscription.name}' after "
- f"{consecutive_count} consecutive rate-limit errors."
+ f"'{old_subscription_name}' to '{new_subscription.name}' after {phrase}."
),
priority="high",
category="subscription",
metadata={
"old_subscription": old_subscription_name,
"new_subscription": new_subscription.name,
- "consecutive_errors": consecutive_count,
- }
+ "failure_kind": failure_kind,
+ "event_count": event_count,
+ },
)
)
except Exception as e:
@@ -161,7 +227,8 @@ async def _perform_auto_switch(
"agent_name": agent_name,
"old_subscription": old_subscription_name,
"new_subscription": new_subscription.name,
- "consecutive_errors": consecutive_count,
+ "failure_kind": failure_kind,
+ "event_count": event_count,
"restart_result": restart_result,
}
diff --git a/src/backend/services/task_execution_service.py b/src/backend/services/task_execution_service.py
index 6d582a09c..073d9573b 100644
--- a/src/backend/services/task_execution_service.py
+++ b/src/backend/services/task_execution_service.py
@@ -29,7 +29,7 @@
from database import db
from models import ActivityState, ActivityType, TaskExecutionStatus
from services.activity_service import activity_service
-from services.slot_service import get_slot_service
+from services.capacity_manager import CapacityFull, get_capacity_manager
from utils.credential_sanitizer import sanitize_execution_log, sanitize_response
from services.platform_prompt_service import (
ExecutionContext,
@@ -246,6 +246,7 @@ async def execute_task(
slot_already_held: bool = False,
schedule_context: Optional[dict] = None,
attempt: Optional[int] = None,
+ images: Optional[list] = None,
) -> TaskExecutionResult:
"""
Execute a task on an agent container with full lifecycle management.
@@ -266,7 +267,7 @@ async def execute_task(
callers are responsible for translating ``result.status == "failed"``
into the appropriate HTTP response.
"""
- slot_service = get_slot_service()
+ capacity = get_capacity_manager()
activity_id: Optional[str] = None
# If caller already acquired the slot (async /task path preserves 429-upfront
# contract by pre-flighting capacity), we still own releasing it in finally.
@@ -307,18 +308,29 @@ async def execute_task(
# stuck in 'running' status with NULL session_id and duration_ms.
try:
# ---- 2. Acquire capacity slot ------------------------------------
+ # CAPACITY-CONSOLIDATE (#428): policy=reject preserves prior
+ # behaviour — TaskExecutionService is invoked when the caller
+ # already decided this execution is admitted (router pre-acquires)
+ # OR is invoked from internal contexts where overflow isn't wanted
+ # (scheduler, fan-out). In both cases we want a hard rejection on
+ # capacity, not a backlog spill.
if not slot_already_held:
max_parallel_tasks = db.get_max_parallel_tasks(agent_name)
- slot_acquired = await slot_service.acquire_slot(
- agent_name=agent_name,
- execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
- max_parallel_tasks=max_parallel_tasks,
- message_preview=message[:100] if message else "",
- timeout_seconds=timeout_seconds, # TIMEOUT-001: Pass for dynamic slot TTL
- )
-
- if not slot_acquired:
- error_msg = f"Agent at capacity ({max_parallel_tasks}/{max_parallel_tasks} parallel tasks running)"
+ try:
+ cap_result = await capacity.acquire(
+ agent_name=agent_name,
+ execution_id=execution_id or f"temp-{datetime.utcnow().timestamp()}",
+ max_concurrent=max_parallel_tasks,
+ message_preview=message[:100] if message else "",
+ timeout_seconds=timeout_seconds,
+ overflow_policy="reject",
+ )
+ slot_acquired = cap_result.state == "admitted"
+ except CapacityFull:
+ error_msg = (
+ f"Agent at capacity ({max_parallel_tasks}/{max_parallel_tasks} "
+ f"parallel tasks running)"
+ )
if execution_id:
db.update_execution_status(
execution_id=execution_id,
@@ -404,6 +416,7 @@ async def execute_task(
"timeout_seconds": timeout_seconds,
"execution_id": execution_id,
"resume_session_id": resume_session_id,
+ "images": images or None,
}
effective_timeout = float(timeout_seconds or 600) + 10
@@ -468,6 +481,8 @@ async def execute_task(
"cost_usd": metadata.get("cost_usd"),
"execution_time_ms": execution_time_ms,
"tool_count": len(response_data.get("execution_log", [])),
+ # #514: short preview surfaced on dashboard timeline hover
+ "response_preview": (sanitized_resp or "")[:200],
},
)
@@ -527,14 +542,30 @@ async def execute_task(
error_msg = e.response.text[:500]
logger.error(f"[TaskExecService] Failed to execute task on {agent_name}: {error_msg}")
- # SUB-003: Auto-switch subscription on rate-limit errors
+ # SUB-003 (#441): Auto-switch on rate-limit (429) OR auth-class
+ # failures (503 from agent server, or auth indicators in the error
+ # text). Fire-and-forget under broad exception handling so a switch
+ # error never masks the underlying execution failure.
agent_status_code = getattr(getattr(e, "response", None), "status_code", None)
- if agent_status_code == 429:
- try:
- from services.subscription_auto_switch import handle_rate_limit_error
- await handle_rate_limit_error(agent_name=agent_name, error_message=error_msg)
- except Exception as switch_err:
- logger.error(f"[SUB-003] Auto-switch check failed for '{agent_name}': {switch_err}")
+ try:
+ from services.subscription_auto_switch import (
+ handle_subscription_failure,
+ is_auth_failure,
+ )
+ if agent_status_code == 429:
+ await handle_subscription_failure(
+ agent_name=agent_name,
+ error_message=error_msg,
+ failure_kind="rate_limit",
+ )
+ elif agent_status_code == 503 or is_auth_failure(error_msg):
+ await handle_subscription_failure(
+ agent_name=agent_name,
+ error_message=error_msg,
+ failure_kind="auth",
+ )
+ except Exception as switch_err:
+ logger.error(f"[SUB-003] Auto-switch check failed for '{agent_name}': {switch_err}")
# Issue #285: Detect auth failures (HTTP 503 from agent server)
# Return structured error code so callers can handle appropriately
@@ -592,7 +623,7 @@ async def execute_task(
finally:
# ---- 8. Release slot (only if acquired) ----------------------
if slot_acquired:
- await slot_service.release_slot(
+ await capacity.release(
agent_name,
execution_id or f"temp-{datetime.utcnow().timestamp()}",
)
diff --git a/src/backend/services/upload_service.py b/src/backend/services/upload_service.py
new file mode 100644
index 000000000..76d170c8c
--- /dev/null
+++ b/src/backend/services/upload_service.py
@@ -0,0 +1,365 @@
+"""
+Shared file upload processing for web chat and channel adapters.
+
+Extracted from adapters/message_router.py so that both the channel adapter path
+(Telegram/Slack/WhatsApp) and the web chat path (/task, /api/public/chat) can
+share the same validation, MIME-checking, and container-write logic.
+"""
+
+import base64
+import io
+import logging
+import os
+import re
+import tarfile
+import unicodedata
+from typing import List, Optional, Tuple
+
+from services.docker_utils import container_put_archive, container_exec_run
+from services.platform_audit_service import platform_audit_service, AuditEventType
+
+logger = logging.getLogger(__name__)
+
+# Try to import python-magic for MIME validation; graceful fallback if unavailable
+try:
+ import magic
+ _MAGIC_AVAILABLE = True
+except ImportError:
+ _MAGIC_AVAILABLE = False
+ logger.warning("[UPLOAD] python-magic not installed; MIME validation will trust declared MIME")
+
+# ---------------------------------------------------------------------------
+# Limits — channel adapter uses the larger set; web uses WEB_* constants
+# ---------------------------------------------------------------------------
+
+CHANNEL_MAX_FILE_SIZE = 10 * 1024 * 1024 # 10 MB per non-image file
+CHANNEL_MAX_IMAGE_SIZE = 5 * 1024 * 1024 # 5 MB per image
+CHANNEL_MAX_TOTAL_IMAGE_SIZE = 10 * 1024 * 1024 # 10 MB total images
+CHANNEL_MAX_FILES = 10
+
+WEB_MAX_FILE_SIZE = 5 * 1024 * 1024 # 5 MB per non-image file
+WEB_MAX_IMAGE_SIZE = 5 * 1024 * 1024 # 5 MB per image
+WEB_MAX_TOTAL_IMAGE_SIZE = 10 * 1024 * 1024 # 10 MB total images
+WEB_MAX_FILES = 3
+
+UNSUPPORTED_MIMES = {
+ "application/pdf", "application/zip", "application/x-tar",
+ "application/gzip", "application/x-rar-compressed",
+ "video/", "audio/",
+}
+
+UPLOAD_BASE = "/home/developer/uploads"
+
+# Filename sanitization constants (same rules as the original message_router)
+_FILENAME_MAX_LENGTH = 200
+_FILENAME_SAFE_CHARS_RE = re.compile(r'[^\w.\-()]')
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def sanitize_filename(name: str, file_id: str, used_names: set) -> str:
+ """
+ Sanitize a user-supplied filename for safe placement in the agent workspace.
+
+ Steps: NFKC normalize → basename → strip unsafe chars → truncate → dedup.
+ Rejects hidden filenames (.env, .gitignore, …) to preserve security posture.
+ """
+ normalized = unicodedata.normalize("NFKC", name or "")
+ base = os.path.basename(normalized)
+ safe = _FILENAME_SAFE_CHARS_RE.sub('_', base)
+
+ stripped = safe.strip('._')
+ if not stripped or safe.startswith('.'):
+ safe = f"file_{file_id}"
+
+ if len(safe) > _FILENAME_MAX_LENGTH:
+ stem, dot, ext = safe.rpartition('.')
+ if dot and len(ext) <= 16:
+ keep = _FILENAME_MAX_LENGTH - len(ext) - 1
+ safe = f"{stem[:keep]}.{ext}"
+ else:
+ safe = safe[:_FILENAME_MAX_LENGTH]
+
+ if safe in used_names:
+ stem, dot, ext = safe.rpartition('.')
+ if not dot:
+ stem, ext = safe, ""
+ suffix_n = 1
+ while True:
+ suffix = f"-{suffix_n}"
+ candidate_stem = stem
+ max_stem = _FILENAME_MAX_LENGTH - len(suffix) - (len(ext) + 1 if ext else 0)
+ if len(candidate_stem) > max_stem:
+ candidate_stem = candidate_stem[:max_stem]
+ candidate = f"{candidate_stem}{suffix}.{ext}" if ext else f"{candidate_stem}{suffix}"
+ if candidate not in used_names:
+ safe = candidate
+ break
+ suffix_n += 1
+
+ return safe
+
+
+def format_file_size(size_bytes: int) -> str:
+ """Format bytes as human-readable string."""
+ if size_bytes < 1024:
+ return f"{size_bytes} B"
+ elif size_bytes < 1024 * 1024:
+ return f"{size_bytes / 1024:.0f} KB"
+ else:
+ return f"{size_bytes / (1024 * 1024):.1f} MB"
+
+
+# ---------------------------------------------------------------------------
+# Core upload processor
+# ---------------------------------------------------------------------------
+
+async def process_file_uploads(
+ raw_files: List[dict],
+ agent_name: str,
+ container,
+ session_id: str,
+ uploader: str,
+ source: str = "web",
+ sender_id: str = "",
+ channel_id: str = "",
+ max_files: int = CHANNEL_MAX_FILES,
+ max_file_size: int = CHANNEL_MAX_FILE_SIZE,
+ max_image_size: int = CHANNEL_MAX_IMAGE_SIZE,
+ max_total_image_size: int = CHANNEL_MAX_TOTAL_IMAGE_SIZE,
+) -> Tuple[List[str], Optional[str], bool, List[dict]]:
+ """
+ Validate and store uploaded files for an agent.
+
+ Each entry in raw_files must have:
+ - name: str
+ - mimetype: str (declared; magic-byte validated if python-magic available)
+ - size: int (declared; actual size validated from data)
+ - data: Optional[bytes] — None means download failed (channel adapter path)
+ - id: str (used for fallback filename generation)
+
+ Returns (descriptions, upload_dir, all_writes_failed, image_data):
+ - descriptions: list of context strings injected into the prompt
+ - upload_dir: container path to clean up after execution, or None
+ - all_writes_failed: True iff at least one write was attempted but all failed
+ - image_data: list of {"media_type": str, "data": base64_str} for vision blocks
+ """
+ safe_session_id = re.sub(r"[^a-zA-Z0-9_-]", "_", session_id)
+ upload_dir = f"{UPLOAD_BASE}/{safe_session_id}"
+ descriptions: List[str] = []
+ image_data: List[dict] = []
+ dir_created = False
+ total_image_bytes = 0
+ used_names: set = set()
+ write_attempted = 0
+ write_succeeded = 0
+
+ for f in raw_files[:max_files]:
+ name = f.get("name", "")
+ mimetype = f.get("mimetype", "application/octet-stream")
+ data: Optional[bytes] = f.get("data")
+ file_id = f.get("id", f"file_{len(used_names)}")
+
+ # Handle download failures (channel adapter path)
+ if data is None:
+ safe_name = sanitize_filename(name, file_id, used_names)
+ used_names.add(safe_name)
+ descriptions.append(f"{safe_name} — download failed")
+ continue
+
+ is_image = mimetype.startswith("image/")
+
+ # Reject unsupported binary formats
+ if any(
+ mimetype.startswith(m) if m.endswith("/") else mimetype == m
+ for m in UNSUPPORTED_MIMES
+ ):
+ safe_name = sanitize_filename(name, file_id, used_names)
+ used_names.add(safe_name)
+ descriptions.append(
+ f"{safe_name} — unsupported format ({mimetype}). "
+ f"Text, CSV, JSON, and image files are supported."
+ )
+ continue
+
+ safe_name = sanitize_filename(name, file_id, used_names)
+ used_names.add(safe_name)
+
+ # Actual size validation (TOCTOU defense — declared size is advisory only)
+ actual_size = len(data)
+ size_limit = max_image_size if is_image else max_file_size
+ if actual_size > size_limit:
+ logger.warning(f"[UPLOAD] Rejecting {safe_name}: {actual_size} bytes > {size_limit}")
+ descriptions.append(
+ f"{safe_name} — rejected (exceeds {format_file_size(size_limit)} limit)"
+ )
+ continue
+
+ # Magic-byte MIME validation
+ actual_mime = mimetype
+ if _MAGIC_AVAILABLE:
+ try:
+ detected_mime = magic.from_buffer(data, mime=True)
+ declared_is_image = mimetype.startswith("image/")
+ detected_is_image = detected_mime.startswith("image/")
+
+ if detected_mime != mimetype:
+ if declared_is_image and detected_is_image:
+ # JPEG vs PNG mislabel — both images, accept with detected MIME
+ logger.debug(
+ f"[UPLOAD] Image MIME mismatch {safe_name}: "
+ f"declared={mimetype}, detected={detected_mime} (allowing)"
+ )
+ actual_mime = detected_mime
+ is_image = True
+ elif mimetype.startswith("text/") and detected_mime.startswith("text/"):
+ # text/plain vs text/csv — both text, accept
+ actual_mime = detected_mime
+ else:
+ logger.warning(
+ f"[UPLOAD] MIME mismatch for {safe_name}: "
+ f"declared={mimetype}, detected={detected_mime}"
+ )
+ descriptions.append(f"{safe_name} — rejected (file type mismatch)")
+ continue
+ except Exception as e:
+ logger.warning(f"[UPLOAD] MIME detection failed for {safe_name}: {e}")
+
+ size_str = format_file_size(actual_size)
+
+ if is_image:
+ if total_image_bytes + actual_size > max_total_image_size:
+ descriptions.append(
+ f"{safe_name} ({size_str}) — skipped (total image size limit reached)"
+ )
+ continue
+
+ write_attempted += 1
+ total_image_bytes += actual_size
+ b64 = base64.b64encode(data).decode()
+ image_data.append({"media_type": actual_mime, "data": b64})
+ descriptions.append(
+ f"[File uploaded by {uploader}]: {safe_name} ({size_str}) — image provided for visual analysis"
+ )
+ write_succeeded += 1
+ logger.info(f"[UPLOAD] Queued {safe_name} ({size_str}) as vision block for {agent_name}")
+
+ await platform_audit_service.log(
+ event_type=AuditEventType.EXECUTION,
+ event_action="file_upload",
+ source=source,
+ target_type="agent",
+ target_id=agent_name,
+ details={
+ "filename": safe_name,
+ "size_bytes": actual_size,
+ "mime_type": actual_mime,
+ "storage": "stream_json_vision",
+ "sender_id": sender_id,
+ "channel_id": channel_id,
+ "uploader": uploader,
+ },
+ )
+
+ else:
+ # Non-image: write to per-session upload directory in the container
+ if not dir_created:
+ write_attempted += 1
+ try:
+ await container_exec_run(
+ container, f"mkdir -p {upload_dir}", user="developer"
+ )
+ dir_created = True
+ except Exception as e:
+ logger.error(
+ f"[UPLOAD] Failed to create {upload_dir} in {agent_name}: {e}"
+ )
+ descriptions.append(
+ f"[File upload failed]: {safe_name} — could not create workspace upload directory"
+ )
+ continue
+ else:
+ write_attempted += 1
+
+ try:
+ tar_buf = io.BytesIO()
+ with tarfile.open(fileobj=tar_buf, mode="w") as tar:
+ info = tarfile.TarInfo(name=safe_name)
+ info.size = actual_size
+ info.uid = 1000 # developer user
+ info.gid = 1000
+ info.mode = 0o644
+ tar.addfile(info, io.BytesIO(data))
+ tar_buf.seek(0)
+
+ success = await container_put_archive(container, upload_dir, tar_buf.read())
+ if not success:
+ logger.error(f"[UPLOAD] Failed to copy {safe_name} into {agent_name}")
+ descriptions.append(
+ f"[File upload failed]: {safe_name} — could not save to agent workspace"
+ )
+ continue
+
+ dest_path = f"{upload_dir}/{safe_name}"
+ descriptions.append(
+ f"[File uploaded by {uploader}]: {safe_name} ({size_str}) saved to {dest_path}"
+ )
+ write_succeeded += 1
+ logger.info(f"[UPLOAD] Copied {safe_name} ({size_str}) to {agent_name}:{dest_path}")
+
+ await platform_audit_service.log(
+ event_type=AuditEventType.EXECUTION,
+ event_action="file_upload",
+ source=source,
+ target_type="agent",
+ target_id=agent_name,
+ details={
+ "filename": safe_name,
+ "size_bytes": actual_size,
+ "mime_type": actual_mime,
+ "storage": "container_file",
+ "dest_path": dest_path,
+ "sender_id": sender_id,
+ "channel_id": channel_id,
+ "uploader": uploader,
+ },
+ )
+
+ except Exception as e:
+ logger.error(f"[UPLOAD] Error copying {safe_name} to {agent_name}: {e}")
+ descriptions.append(
+ f"[File upload failed]: {safe_name} — workspace write error"
+ )
+
+ if len(raw_files) > max_files:
+ descriptions.append(
+ f"({len(raw_files) - max_files} more file(s) skipped — max {max_files} per message)"
+ )
+
+ all_writes_failed = write_attempted > 0 and write_succeeded == 0
+ return descriptions, upload_dir if dir_created else None, all_writes_failed, image_data
+
+
+def decode_web_file(f: dict) -> Optional[bytes]:
+ """
+ Decode a WebFileUpload's base64 data to bytes.
+
+ Handles both raw base64 and data: URI format (data:mime;base64,PAYLOAD)
+ emitted by browser FileReader.readAsDataURL().
+
+ Returns None on decode failure.
+ """
+ raw = f.get("data_base64", "")
+ if not raw:
+ return None
+ try:
+ # Strip data: URI prefix if present
+ if raw.startswith("data:"):
+ comma = raw.index(",")
+ raw = raw[comma + 1:]
+ return base64.b64decode(raw)
+ except Exception as e:
+ logger.warning(f"[UPLOAD] base64 decode failed for {f.get('name', '?')}: {e}")
+ return None
diff --git a/src/backend/services/ws_ticket_service.py b/src/backend/services/ws_ticket_service.py
new file mode 100644
index 000000000..d1965c6f3
--- /dev/null
+++ b/src/backend/services/ws_ticket_service.py
@@ -0,0 +1,112 @@
+"""
+WebSocket auth tickets (#550).
+
+Closes the JWT-in-URL leak on ``/ws`` by swapping the long-lived bearer
+token for a short-lived opaque ticket. Browser flow:
+
+ 1. Client calls ``POST /api/ws/ticket`` (JWT-authed, normal Bearer).
+ 2. Backend mints a 32-byte urlsafe ticket, stores it in Redis with
+ a 30-second TTL, and returns it.
+ 3. Client connects to ``/ws?ticket=``. Backend atomically
+ GETDELs the Redis key — single-use — and resolves it to the
+ authenticated subject before accepting the WebSocket.
+
+Why this matters:
+
+- The JWT no longer appears in nginx access logs, browser history,
+ or upstream proxy logs.
+- A leaked ticket is single-use and expires in 30s, so replay is
+ effectively impossible.
+- CSWSH is mitigated: a malicious page can't mint a ticket on behalf
+ of the victim because ``POST /api/ws/ticket`` requires the JWT in
+ an ``Authorization`` header (cross-origin requests would fail
+ CORS preflight or lack the header entirely).
+
+Redis is required. If Redis is unavailable ``mint_ticket`` raises;
+``consume_ticket`` returns ``None`` and the WebSocket connection is
+rejected. Failing closed is the right call for an auth path.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import secrets
+from typing import Optional
+
+from routers.auth import get_redis_client
+
+logger = logging.getLogger(__name__)
+
+_TICKET_TTL_SECONDS = 30
+_TICKET_KEY_PREFIX = "ws_ticket:"
+
+
+def _key(ticket: str) -> str:
+ return f"{_TICKET_KEY_PREFIX}{ticket}"
+
+
+def mint_ticket(subject: str, *, scope: str = "user") -> str:
+ """Mint a single-use opaque WebSocket ticket.
+
+ Args:
+ subject: The authenticated principal (username for JWT users).
+ scope: Identifier for the auth surface — currently always
+ ``"user"`` for the browser ``/ws`` flow. Reserved for a
+ future ticket variant on ``/ws/events`` (MCP scope).
+
+ Returns:
+ The opaque ticket string. Hand to client; expect them to
+ present it back on the WebSocket URL within 30 seconds.
+
+ Raises:
+ RuntimeError: Redis unavailable. Auth must fail closed.
+ """
+ r = get_redis_client()
+ if r is None:
+ raise RuntimeError("Redis unavailable — cannot mint WebSocket ticket")
+
+ ticket = secrets.token_urlsafe(32)
+ payload = json.dumps({"sub": subject, "scope": scope})
+ r.setex(_key(ticket), _TICKET_TTL_SECONDS, payload)
+ return ticket
+
+
+def consume_ticket(ticket: str) -> Optional[dict]:
+ """Atomically exchange a ticket for its principal.
+
+ Uses Redis ``GETDEL`` (Redis 6.2+) so the ticket is single-use:
+ the same ticket can never authenticate two connections, even
+ under a race. Returns ``None`` if the ticket is missing,
+ expired, or already consumed.
+
+ Args:
+ ticket: The opaque ticket the client presented.
+
+ Returns:
+ ``{"sub": "", "scope": ""}`` on success;
+ ``None`` on any failure mode (missing, expired, used,
+ Redis down, malformed).
+ """
+ if not ticket:
+ return None
+
+ r = get_redis_client()
+ if r is None:
+ logger.warning("Redis unavailable — rejecting WebSocket ticket exchange")
+ return None
+
+ try:
+ raw = r.getdel(_key(ticket))
+ except Exception as exc:
+ logger.warning("WebSocket ticket exchange failed: %s", exc)
+ return None
+
+ if not raw:
+ return None
+
+ try:
+ return json.loads(raw)
+ except (TypeError, ValueError):
+ logger.warning("WebSocket ticket payload was not valid JSON")
+ return None
diff --git a/src/frontend/.gitignore b/src/frontend/.gitignore
new file mode 100644
index 000000000..890e30b78
--- /dev/null
+++ b/src/frontend/.gitignore
@@ -0,0 +1,12 @@
+# Vite/Node
+node_modules
+dist
+.env
+.env.local
+
+# Playwright e2e
+/e2e/.auth/admin.json
+/e2e/test-results/
+/e2e/playwright-report/
+/e2e/blob-report/
+/playwright/.cache/
diff --git a/src/frontend/e2e/README.md b/src/frontend/e2e/README.md
new file mode 100644
index 000000000..cd3cd0136
--- /dev/null
+++ b/src/frontend/e2e/README.md
@@ -0,0 +1,81 @@
+# Frontend E2E Tests
+
+Playwright-based end-to-end tests for the Trinity frontend (#556).
+
+## Run locally
+
+```bash
+# 1. Start a Trinity stack (the tests don't spin one up themselves)
+./scripts/deploy/start.sh
+
+# 2. Run the tests
+cd src/frontend
+ADMIN_PASSWORD= npm run test:e2e
+```
+
+`ADMIN_PASSWORD` is required — `e2e/auth.setup.js` uses it to log in once and
+caches the session in `e2e/.auth/admin.json` (gitignored).
+
+## Useful flags
+
+```bash
+npm run test:e2e:smoke # only @smoke-tagged tests (CI parity)
+npm run test:e2e:headed # run with visible browser
+npm run test:e2e:ui # interactive Playwright UI
+npm run test:e2e:update # update visual regression snapshots
+```
+
+After a run, the HTML report is at `e2e/playwright-report/index.html`.
+
+## Spec tags
+
+Each test gets a tag in its name to control where it runs:
+
+| Tag | Runs in CI? | Purpose |
+|---|---|---|
+| `@smoke` | ✅ always | Cross-page health checks. Fast (~5s), zero flakiness. Must always pass. |
+| `@visual` | ❌ local only | Visual regression / screenshot baselines. Deferred until cross-platform baseline capture is sorted (issue #596). |
+| `@interactive` | ❌ local only | Forms, modals, multi-step flows. Local-only until stabilised. |
+
+CI runs `npm run test:e2e:smoke` (filters by `@smoke`). To promote a spec to CI, simply rename it to include `@smoke` — no workflow changes needed.
+
+```js
+// CI + local
+test('@smoke dashboard renders', async ({ page }) => { ... })
+
+// Local only (until visual regression infra lands — #596)
+test('@visual /monitoring summary cards', async ({ page }) => { ... })
+
+// Local only (interactive flow)
+test('@interactive create agent end-to-end', async ({ page }) => { ... })
+```
+
+## CI
+
+CI runs e2e only on PRs **labeled `ui`** — add the label to any frontend PR
+that should be exercised end-to-end. The workflow lives at
+`.github/workflows/frontend-e2e.yml` and stands up the full Trinity stack
+before running tests (~5 min total).
+
+To make e2e a required check on a PR, add the `ui` label and wait for the
+workflow to complete.
+
+## Adding tests
+
+- Smoke tests live in `e2e/smoke.spec.js` — the lightweight cross-page checks
+- New flows go in their own `*.spec.js` next to the smoke file
+- Visual regression: use `await expect(page).toHaveScreenshot()`. Snapshots
+ are committed in `e2e/.spec.js-snapshots/`. Run
+ `npm run test:e2e:update` after intentional UI changes, then commit the
+ updated PNGs.
+
+## Why this layer exists
+
+The frontend has no other automated test coverage today. E2E tests catch:
+- Login regressions
+- Top-level routing breakage
+- Auth boundary violations exposed via the UI
+- Color drift on the design system (with visual regression)
+
+Cheaper layers (Vitest unit tests, type checking) are tracked in #556
+Phase 1 / Phase 3 — separate follow-ups.
diff --git a/src/frontend/e2e/auth.setup.js b/src/frontend/e2e/auth.setup.js
new file mode 100644
index 000000000..af4fb40b5
--- /dev/null
+++ b/src/frontend/e2e/auth.setup.js
@@ -0,0 +1,37 @@
+import { test as setup, expect } from '@playwright/test'
+
+const ADMIN_PASSWORD = process.env.ADMIN_PASSWORD
+if (!ADMIN_PASSWORD) {
+ throw new Error('ADMIN_PASSWORD env var must be set for e2e tests')
+}
+
+setup('authenticate as admin', async ({ page }) => {
+ await page.goto('/')
+
+ // Default landing form is email-auth (when EMAIL_AUTH_ENABLED is true). The
+ // admin form is reached via a toggle button labelled "🔐 Admin Login" — the
+ // emoji prefix breaks Playwright's role-name normalization, so match by text.
+ // When EMAIL_AUTH_ENABLED is false the admin form shows immediately and the
+ // toggle isn't rendered.
+ const passwordInput = page.locator('#password')
+ if (!(await passwordInput.isVisible({ timeout: 3000 }).catch(() => false))) {
+ await page.locator('button:has-text("Admin Login")').click()
+ }
+
+ // Wait for the password input to be ready, then fill + submit. Using the
+ // form's submit button (rather than a name regex) keeps this resilient to
+ // copy changes.
+ await passwordInput.waitFor({ state: 'visible', timeout: 10000 })
+ await passwordInput.fill(ADMIN_PASSWORD)
+ await page.locator('form button[type="submit"]').click()
+
+ // Wait for the JWT to land in localStorage. Trinity's auth store persists
+ // the token here after a successful login (see stores/auth.js). Saving
+ // storageState before this completes produces an empty state and every
+ // downstream test lands on /login.
+ await expect
+ .poll(() => page.evaluate(() => localStorage.getItem('token')), { timeout: 10000 })
+ .not.toBeNull()
+ await expect(passwordInput).toBeHidden({ timeout: 5000 })
+ await page.context().storageState({ path: 'e2e/.auth/admin.json' })
+})
diff --git a/src/frontend/e2e/smoke.spec.js b/src/frontend/e2e/smoke.spec.js
new file mode 100644
index 000000000..0c8a775ec
--- /dev/null
+++ b/src/frontend/e2e/smoke.spec.js
@@ -0,0 +1,54 @@
+import { test, expect } from '@playwright/test'
+
+// Spec tag convention (#556 follow-up):
+// @smoke — must always pass; runs in CI on every `ui`-labelled PR.
+// @visual — visual regression / screenshot baselines; CI runs only
+// once cross-platform baselines exist (#596).
+// @interactive — exercises forms, modals, multi-step flows; expensive,
+// usually local-only until the test is stabilised.
+//
+// CI runs `npm run test:e2e:smoke` (filters by @smoke). Locally,
+// `npm run test:e2e` runs everything.
+test.describe('smoke', () => {
+ test('@smoke dashboard renders for authenticated admin', async ({ page }) => {
+ await page.goto('/')
+ // Top nav has Dashboard, Agents, Templates, Health, Ops, Keys, Settings.
+ await expect(page.getByRole('link', { name: 'Dashboard', exact: true })).toBeVisible({ timeout: 10000 })
+ await expect(page.getByRole('link', { name: 'Agents', exact: true })).toBeVisible()
+ await expect(page.getByRole('link', { name: 'Settings', exact: true })).toBeVisible()
+ })
+
+ test('@smoke agents page loads', async ({ page }) => {
+ await page.goto('/agents')
+ await expect(page.getByText(/agent|create/i).first()).toBeVisible({ timeout: 10000 })
+ })
+
+ test('@smoke operating room page loads', async ({ page }) => {
+ await page.goto('/operating-room')
+ // Either a queue list, filters, an empty state, or the title.
+ await expect(
+ page.getByText(/operating|queue|priority|all types|no items/i).first()
+ ).toBeVisible({ timeout: 10000 })
+ })
+
+ test('@smoke templates page loads', async ({ page }) => {
+ await page.goto('/templates')
+ await expect(page.getByText(/template/i).first()).toBeVisible({ timeout: 10000 })
+ })
+
+ test('@smoke monitoring page loads', async ({ page }) => {
+ await page.goto('/monitoring')
+ // Header, summary cards, or empty state — any of these confirms the route mounted.
+ await expect(
+ page.getByText(/monitoring|fleet|healthy|degraded|no agents/i).first()
+ ).toBeVisible({ timeout: 10000 })
+ })
+
+ test('@smoke api keys page loads', async ({ page }) => {
+ await page.goto('/api-keys')
+ // Header, info banner, list, or empty state — any confirms the route mounted.
+ await expect(
+ page.getByText(/mcp api keys|connect to mcp|no api keys|create api key/i).first()
+ ).toBeVisible({ timeout: 10000 })
+ })
+})
diff --git a/src/frontend/nginx.conf b/src/frontend/nginx.conf
index 18d2a81af..c8d88c4e5 100644
--- a/src/frontend/nginx.conf
+++ b/src/frontend/nginx.conf
@@ -4,6 +4,9 @@ server {
root /usr/share/nginx/html;
index index.html;
+ # File upload support (#364): 3 files × 5 MB base64-encoded ≈ ~21 MB JSON
+ client_max_body_size 25m;
+
# Strip server version from responses
server_tokens off;
diff --git a/src/frontend/package-lock.json b/src/frontend/package-lock.json
index a1e05b6d8..c315b4299 100644
--- a/src/frontend/package-lock.json
+++ b/src/frontend/package-lock.json
@@ -31,6 +31,7 @@
"vue-router": "^4.5.0"
},
"devDependencies": {
+ "@playwright/test": "^1.59.1",
"@tailwindcss/typography": "^0.5.15",
"@types/js-yaml": "^4.0.9",
"@vitejs/plugin-vue": "^5.2.1",
@@ -181,6 +182,21 @@
"node": ">= 8"
}
},
+ "node_modules/@playwright/test": {
+ "version": "1.59.1",
+ "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.59.1.tgz",
+ "integrity": "sha512-PG6q63nQg5c9rIi4/Z5lR5IVF7yU5MqmKaPOe0HSc0O2cX1fPi96sUQu5j7eo4gKCkB2AnNGoWt7y4/Xx3Kcqg==",
+ "dev": true,
+ "dependencies": {
+ "playwright": "1.59.1"
+ },
+ "bin": {
+ "playwright": "cli.js"
+ },
+ "engines": {
+ "node": ">=18"
+ }
+ },
"node_modules/@rollup/rollup-android-arm-eabi": {
"version": "4.60.1",
"resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.60.1.tgz",
@@ -1722,6 +1738,50 @@
"node": ">= 6"
}
},
+ "node_modules/playwright": {
+ "version": "1.59.1",
+ "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.59.1.tgz",
+ "integrity": "sha512-C8oWjPR3F81yljW9o5OxcWzfh6avkVwDD2VYdwIGqTkl+OGFISgypqzfu7dOe4QNLL2aqcWBmI3PMtLIK233lw==",
+ "dev": true,
+ "dependencies": {
+ "playwright-core": "1.59.1"
+ },
+ "bin": {
+ "playwright": "cli.js"
+ },
+ "engines": {
+ "node": ">=18"
+ },
+ "optionalDependencies": {
+ "fsevents": "2.3.2"
+ }
+ },
+ "node_modules/playwright-core": {
+ "version": "1.59.1",
+ "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.59.1.tgz",
+ "integrity": "sha512-HBV/RJg81z5BiiZ9yPzIiClYV/QMsDCKUyogwH9p3MCP6IYjUFu/MActgYAvK0oWyV9NlwM3GLBjADyWgydVyg==",
+ "dev": true,
+ "bin": {
+ "playwright-core": "cli.js"
+ },
+ "engines": {
+ "node": ">=18"
+ }
+ },
+ "node_modules/playwright/node_modules/fsevents": {
+ "version": "2.3.2",
+ "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
+ "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
+ "dev": true,
+ "hasInstallScript": true,
+ "optional": true,
+ "os": [
+ "darwin"
+ ],
+ "engines": {
+ "node": "^8.16.0 || ^10.6.0 || >=11.0.0"
+ }
+ },
"node_modules/postcss": {
"version": "8.5.6",
"funding": [
diff --git a/src/frontend/package.json b/src/frontend/package.json
index 2ed7a6204..9f01e4a63 100644
--- a/src/frontend/package.json
+++ b/src/frontend/package.json
@@ -4,7 +4,13 @@
"scripts": {
"dev": "vite",
"build": "vite build",
- "preview": "vite preview"
+ "preview": "vite preview",
+ "check:tokens": "node scripts/check-design-tokens.mjs",
+ "test:e2e": "playwright test",
+ "test:e2e:smoke": "playwright test --grep @smoke",
+ "test:e2e:ui": "playwright test --ui",
+ "test:e2e:headed": "playwright test --headed",
+ "test:e2e:update": "playwright test --update-snapshots"
},
"dependencies": {
"@heroicons/vue": "^2.2.0",
@@ -30,6 +36,7 @@
"vue-router": "^4.5.0"
},
"devDependencies": {
+ "@playwright/test": "^1.59.1",
"@tailwindcss/typography": "^0.5.15",
"@types/js-yaml": "^4.0.9",
"@vitejs/plugin-vue": "^5.2.1",
diff --git a/src/frontend/playwright.config.js b/src/frontend/playwright.config.js
new file mode 100644
index 000000000..2958e5b4e
--- /dev/null
+++ b/src/frontend/playwright.config.js
@@ -0,0 +1,33 @@
+// Playwright config for Trinity frontend e2e tests (#556).
+// Tests run against a live Trinity stack — set E2E_BASE_URL to override the
+// default `http://localhost`. Locally, run `./scripts/deploy/start.sh` first.
+
+import { defineConfig, devices } from '@playwright/test'
+
+const STORAGE_STATE = 'e2e/.auth/admin.json'
+
+export default defineConfig({
+ testDir: './e2e',
+ fullyParallel: true,
+ forbidOnly: !!process.env.CI,
+ retries: process.env.CI ? 2 : 0,
+ workers: process.env.CI ? 1 : undefined,
+ reporter: process.env.CI
+ ? [['github'], ['html', { outputFolder: 'e2e/playwright-report', open: 'never' }]]
+ : [['list'], ['html', { outputFolder: 'e2e/playwright-report', open: 'never' }]],
+ outputDir: 'e2e/test-results',
+ use: {
+ baseURL: process.env.E2E_BASE_URL || 'http://localhost',
+ trace: 'on-first-retry',
+ screenshot: 'only-on-failure',
+ video: 'retain-on-failure',
+ },
+ projects: [
+ { name: 'setup', testMatch: /.*\.setup\.js/ },
+ {
+ name: 'chromium',
+ use: { ...devices['Desktop Chrome'], storageState: STORAGE_STATE },
+ dependencies: ['setup'],
+ },
+ ],
+})
diff --git a/src/frontend/postcss.config.js b/src/frontend/postcss.config.mjs
similarity index 100%
rename from src/frontend/postcss.config.js
rename to src/frontend/postcss.config.mjs
diff --git a/src/frontend/scripts/check-design-tokens.mjs b/src/frontend/scripts/check-design-tokens.mjs
new file mode 100644
index 000000000..f7df9bddf
--- /dev/null
+++ b/src/frontend/scripts/check-design-tokens.mjs
@@ -0,0 +1,103 @@
+#!/usr/bin/env node
+/**
+ * Verifies the design-system color tokens (#67):
+ * 1. Each `status-*` token in tailwind.config.js is a direct alias of the
+ * Tailwind palette it claims to (catches accidental palette swaps).
+ * 2. Every `bg-status-*`, `text-status-*`, `focus:ring-status-*`, or
+ * `dark:*-status-*` reference in the migrated source files uses one of
+ * the defined token names (catches typos that Tailwind would silently
+ * drop).
+ *
+ * Run via `npm run check:tokens` or directly: `node scripts/check-design-tokens.mjs`.
+ */
+
+import { readFileSync, readdirSync, statSync } from 'node:fs'
+import { dirname, join, resolve } from 'node:path'
+import { fileURLToPath } from 'node:url'
+
+const __dirname = dirname(fileURLToPath(import.meta.url))
+const FRONTEND_ROOT = resolve(__dirname, '..')
+
+// Tailwind config uses `export default` but the frontend package.json is not
+// "type": "module", so Node can't import it directly here. Tailwind has its
+// own loader. We read the config as text and assert each token aliases the
+// expected palette via a literal `colors.` reference — that's the only
+// invariant this PR commits to.
+const EXPECTED_ALIASES = {
+ 'status-success': 'green',
+ 'status-warning': 'yellow',
+ 'status-danger': 'red',
+ 'status-info': 'blue',
+ 'status-urgent': 'orange',
+ 'state-autonomous': 'amber',
+ 'state-locked': 'rose',
+ 'brand-claude': 'orange',
+ 'brand-gemini': 'blue',
+ 'accent-purple': 'purple',
+}
+
+// Known token families. Reference scanner uses this to flag references like
+// `bg-status-foo-500` where `foo` isn't a defined token in the family.
+const KNOWN_FAMILIES = {
+ status: new Set(['success', 'warning', 'danger', 'info', 'urgent']),
+ state: new Set(['autonomous', 'locked']),
+ brand: new Set(['claude', 'gemini']),
+ accent: new Set(['purple']),
+}
+
+function checkPaletteEquivalence() {
+ const failures = []
+ const configText = readFileSync(join(FRONTEND_ROOT, 'tailwind.config.js'), 'utf8')
+ for (const [tokenName, paletteName] of Object.entries(EXPECTED_ALIASES)) {
+ const aliasRe = new RegExp(`['"]${tokenName}['"]\\s*:\\s*colors\\.${paletteName}\\b`)
+ if (!aliasRe.test(configText)) {
+ failures.push(`${tokenName}: expected alias of colors.${paletteName} not found in tailwind.config.js`)
+ }
+ }
+ return failures
+}
+
+const FAMILY_RE = Object.keys(KNOWN_FAMILIES).join('|')
+const TOKEN_REFERENCE_RE = new RegExp(
+ `(?:bg|text|border|ring|fill|stroke|from|to|via|focus:ring|focus:bg|focus:text|focus:border|hover:bg|hover:text|hover:border|hover:ring|dark:bg|dark:text|dark:border|dark:ring|dark:hover:bg|dark:hover:text)-(${FAMILY_RE})-([a-z]+)-(?:50|100|200|300|400|500|600|700|800|900|950)\\b`,
+ 'g'
+)
+
+function* walkVueAndJs(dir) {
+ for (const entry of readdirSync(dir)) {
+ const full = join(dir, entry)
+ if (entry === 'node_modules' || entry === 'dist' || entry.startsWith('.')) continue
+ const stat = statSync(full)
+ if (stat.isDirectory()) yield* walkVueAndJs(full)
+ else if (/\.(vue|js|ts|jsx|tsx)$/.test(entry)) yield full
+ }
+}
+
+function checkTokenReferences() {
+ const failures = []
+ for (const file of walkVueAndJs(join(FRONTEND_ROOT, 'src'))) {
+ const content = readFileSync(file, 'utf8')
+ for (const match of content.matchAll(TOKEN_REFERENCE_RE)) {
+ const [whole, family, variant] = match
+ const variants = KNOWN_FAMILIES[family]
+ if (!variants?.has(variant)) {
+ const line = content.slice(0, match.index).split('\n').length
+ failures.push(`${file.replace(FRONTEND_ROOT + '/', '')}:${line}: unknown ${family}-* token "${variant}" in "${whole}"`)
+ }
+ }
+ }
+ return failures
+}
+
+const paletteFailures = checkPaletteEquivalence()
+const referenceFailures = checkTokenReferences()
+const allFailures = [...paletteFailures, ...referenceFailures]
+
+if (allFailures.length > 0) {
+ console.error('Design-token check FAILED:')
+ for (const f of allFailures) console.error(' ' + f)
+ process.exit(1)
+}
+
+const tokenCount = Object.keys(EXPECTED_ALIASES).length
+console.log(`Design-token check OK: ${tokenCount} tokens equivalent to source palettes; all references resolve`)
diff --git a/src/frontend/src/components/AutonomyToggle.vue b/src/frontend/src/components/AutonomyToggle.vue
index 72f93409c..93802a314 100644
--- a/src/frontend/src/components/AutonomyToggle.vue
+++ b/src/frontend/src/components/AutonomyToggle.vue
@@ -5,7 +5,7 @@
class="font-medium whitespace-nowrap min-w-[3rem] text-right"
:class="[
labelSizeClass,
- modelValue ? 'text-amber-600 dark:text-amber-400' : 'text-gray-500 dark:text-gray-400'
+ modelValue ? 'text-state-autonomous-600 dark:text-state-autonomous-400' : 'text-gray-500 dark:text-gray-400'
]"
>
{{ modelValue ? 'AUTO' : 'Manual' }}
@@ -20,7 +20,7 @@
class="relative inline-flex items-center rounded-full transition-colors focus:outline-none focus:ring-2 focus:ring-offset-2 dark:focus:ring-offset-gray-800"
:class="[
sizeClasses,
- modelValue ? 'bg-amber-500 focus:ring-amber-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
+ modelValue ? 'bg-state-autonomous-500 focus:ring-state-autonomous-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
(disabled || loading) ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer'
]"
>
diff --git a/src/frontend/src/components/CapacityMeter.vue b/src/frontend/src/components/CapacityMeter.vue
index 02c0de3eb..7a1e643f5 100644
--- a/src/frontend/src/components/CapacityMeter.vue
+++ b/src/frontend/src/components/CapacityMeter.vue
@@ -38,10 +38,10 @@ const atCapacity = computed(() => utilization.value >= 100)
const fillClass = computed(() => {
const u = utilization.value
if (u <= 0) return 'bg-gray-200 dark:bg-gray-700'
- if (u < 50) return 'bg-green-500'
- if (u < 80) return 'bg-yellow-500'
- if (u < 100) return 'bg-orange-500'
- return 'bg-red-500'
+ if (u < 50) return 'bg-status-success-500'
+ if (u < 80) return 'bg-status-warning-500'
+ if (u < 100) return 'bg-status-urgent-500'
+ return 'bg-status-danger-500'
})
diff --git a/src/frontend/src/components/ChatPanel.vue b/src/frontend/src/components/ChatPanel.vue
index bdd76c814..6ef7077ef 100644
--- a/src/frontend/src/components/ChatPanel.vue
+++ b/src/frontend/src/components/ChatPanel.vue
@@ -129,17 +129,11 @@
class="flex-1 px-6"
>
-
-
-
Start a Conversation
-
- Chat with your agent using a simple interface. All activity is tracked in the Dashboard timeline.
-
-
+
@@ -177,7 +171,7 @@
import { ref, computed, nextTick, onMounted, onUnmounted, watch } from 'vue'
import axios from 'axios'
import { useAuthStore } from '../stores/auth'
-import { ChatMessages, ChatInput } from './chat'
+import { ChatMessages, ChatInput, ChatEmptyState } from './chat'
import VoiceOverlay from './chat/VoiceOverlay.vue'
import ModelSelector from './ModelSelector.vue'
import { getStatusFromStreamEvent, MIN_LABEL_DISPLAY_MS, HEARTBEAT_TIMEOUT_MS } from '../utils/execution-status'
@@ -276,6 +270,29 @@ const focusChatInput = () => {
// Model selection
const selectedModel = ref(localStorage.getItem('trinity_chat_model') || '')
+// Playbooks (for empty-state quick actions)
+const playbooks = ref([])
+const loadPlaybooks = async () => {
+ if (!props.agentName || props.agentStatus !== 'running') return
+ try {
+ const response = await axios.get(
+ `/api/agents/${props.agentName}/playbooks`,
+ { headers: authStore.authHeader }
+ )
+ playbooks.value = response.data.skills || []
+ } catch {
+ playbooks.value = []
+ }
+}
+const onEmptyStateSelect = ({ text, sendImmediately }) => {
+ if (sendImmediately) {
+ sendMessage(text)
+ } else {
+ message.value = text
+ nextTick(() => chatInputRef.value?.focus())
+ }
+}
+
// SSE state (THINK-001)
let heartbeatTimer = null
let labelTimer = null
@@ -586,8 +603,8 @@ const pollExecution = async (executionId) => {
}
// Send message
-const sendMessage = async (userMessage) => {
- if (!userMessage || loading.value || props.agentStatus !== 'running') return
+const sendMessage = async (userMessage, files = []) => {
+ if ((!userMessage && files.length === 0) || loading.value || props.agentStatus !== 'running') return
error.value = null
@@ -616,7 +633,8 @@ const sendMessage = async (userMessage) => {
create_new_session: !currentSessionId.value,
chat_session_id: currentSessionId.value || undefined,
async_mode: true,
- model: selectedModel.value || undefined
+ model: selectedModel.value || undefined,
+ files: files.length > 0 ? files : undefined,
}
// EXEC-023: Include resume_session_id for ALL messages in resume mode
@@ -708,6 +726,7 @@ watch(() => props.agentStatus, (newStatus) => {
if (newStatus === 'running') {
loadSessions()
checkVoiceAvailability()
+ loadPlaybooks()
}
})
@@ -732,6 +751,7 @@ watch(() => props.agentName, () => {
closeSSE()
if (props.agentStatus === 'running') {
loadSessions()
+ loadPlaybooks()
}
})
@@ -741,6 +761,7 @@ onMounted(() => {
if (props.agentStatus === 'running') {
loadSessions()
checkVoiceAvailability()
+ loadPlaybooks()
}
})
diff --git a/src/frontend/src/components/CredentialsPanel.vue b/src/frontend/src/components/CredentialsPanel.vue
index 15b204d00..547312928 100644
--- a/src/frontend/src/components/CredentialsPanel.vue
+++ b/src/frontend/src/components/CredentialsPanel.vue
@@ -497,7 +497,12 @@ const closeEditor = () => {
const getPlaceholder = (filename) => {
const placeholders = {
'.env': 'OPENAI_API_KEY=sk-...\nANTHROPIC_API_KEY=sk-ant-...',
- '.mcp.json': '{\n "mcpServers": {\n "trinity": {\n "command": "npx",\n "args": ["-y", "@anthropic-ai/trinity-mcp-server"]\n }\n }\n}',
+ // `.mcp.json` content is structure-validated at save (#598).
+ // The `trinity` server name is reserved (auto-injected on agent start);
+ // the example shows a real MCP server (context7) with the validated
+ // shape: command from allowlist (npx/uvx/python/python3/node/bun/deno/docker),
+ // args without shell metacharacters, env values as ${VAR} or safe literals.
+ '.mcp.json': '{\n "mcpServers": {\n "context7": {\n "command": "npx",\n "args": ["-y", "@upstash/context7-mcp@latest"]\n }\n }\n}',
'.mcp.json.template': '{\n "mcpServers": {}\n}'
}
return placeholders[filename] || ''
diff --git a/src/frontend/src/components/DashboardPanel.vue b/src/frontend/src/components/DashboardPanel.vue
index 759ef2656..126b1332e 100644
--- a/src/frontend/src/components/DashboardPanel.vue
+++ b/src/frontend/src/components/DashboardPanel.vue
@@ -504,13 +504,13 @@ const getSparklineColor = (widget) => {
// Get status badge colors
const getStatusColors = (color) => {
const colorMap = {
- green: 'bg-green-100 text-green-800 dark:bg-green-900/50 dark:text-green-300',
- red: 'bg-red-100 text-red-800 dark:bg-red-900/50 dark:text-red-300',
- yellow: 'bg-yellow-100 text-yellow-800 dark:bg-yellow-900/50 dark:text-yellow-300',
+ green: 'bg-status-success-100 text-status-success-800 dark:bg-status-success-900/50 dark:text-status-success-300',
+ red: 'bg-status-danger-100 text-status-danger-800 dark:bg-status-danger-900/50 dark:text-status-danger-300',
+ yellow: 'bg-status-warning-100 text-status-warning-800 dark:bg-status-warning-900/50 dark:text-status-warning-300',
gray: 'bg-gray-100 text-gray-600 dark:bg-gray-700 dark:text-gray-300',
- blue: 'bg-blue-100 text-blue-800 dark:bg-blue-900/50 dark:text-blue-300',
- orange: 'bg-orange-100 text-orange-800 dark:bg-orange-900/50 dark:text-orange-300',
- purple: 'bg-purple-100 text-purple-800 dark:bg-purple-900/50 dark:text-purple-300'
+ blue: 'bg-status-info-100 text-status-info-800 dark:bg-status-info-900/50 dark:text-status-info-300',
+ orange: 'bg-status-urgent-100 text-status-urgent-800 dark:bg-status-urgent-900/50 dark:text-status-urgent-300',
+ purple: 'bg-accent-purple-100 text-accent-purple-800 dark:bg-accent-purple-900/50 dark:text-accent-purple-300'
}
return colorMap[color] || colorMap.gray
}
diff --git a/src/frontend/src/components/FileSharingPanel.vue b/src/frontend/src/components/FileSharingPanel.vue
new file mode 100644
index 000000000..91d1c8123
--- /dev/null
+++ b/src/frontend/src/components/FileSharingPanel.vue
@@ -0,0 +1,230 @@
+
+
+
File Sharing
+
+ Let the agent publish files from its
+ /home/developer/public/
+ directory via time-limited download URLs. Agents call the
+ share_file
+ MCP tool once this toggle is on.
+
+
+
+
+
+
+
+
+
+
+ {{ status.enabled ? 'Enabled' : 'Disabled' }}
+
+
+ Flipping this toggle requires restarting the agent before it takes effect.
+
+
+
+
+
+
+ Configuration changed — restart the agent to mount or detach
+ /home/developer/public/.
+
+
+
+
+ {{ files.length }}
+ active file{{ files.length === 1 ? '' : 's' }} ·
+ {{ formatBytes(totalBytes) }}
+ of {{ formatBytes(quotaBytes) }}
+ used
+
+
+
+
+
+
+
+ Filename
+ Size
+ Expires
+ Downloads
+ Actions
+
+
+
+
+
+ {{ file.filename }}
+
+ {{ formatBytes(file.size_bytes) }}
+
+ {{ relativeTime(file.expires_at) }}
+
+ {{ file.download_count }}
+
+
+ {{ copiedId === file.file_id ? 'Copied!' : 'Copy URL' }}
+
+
+ {{ revokingId === file.file_id ? 'Revoking…' : 'Revoke' }}
+
+
+
+
+
+
+
+
+
+ No active shares yet. The agent will publish files via the
+ share_file MCP tool.
+
+
+
+
+
diff --git a/src/frontend/src/components/MetricsPanel.vue b/src/frontend/src/components/MetricsPanel.vue
index 71d46e0d3..1016ae611 100644
--- a/src/frontend/src/components/MetricsPanel.vue
+++ b/src/frontend/src/components/MetricsPanel.vue
@@ -263,9 +263,9 @@ const getPercentageColor = (value, metric) => {
const critical = metric.critical_threshold ?? 0
const warning = metric.warning_threshold ?? 0
- if (critical > 0 && value < critical) return 'text-red-600'
- if (warning > 0 && value < warning) return 'text-yellow-600'
- return 'text-green-600'
+ if (critical > 0 && value < critical) return 'text-status-danger-600'
+ if (warning > 0 && value < warning) return 'text-status-warning-600'
+ return 'text-status-success-600'
}
// Get percentage bar color
@@ -274,9 +274,9 @@ const getPercentageBarColor = (value, metric) => {
const critical = metric.critical_threshold ?? 0
const warning = metric.warning_threshold ?? 0
- if (critical > 0 && value < critical) return 'bg-red-500'
- if (warning > 0 && value < warning) return 'bg-yellow-500'
- return 'bg-green-500'
+ if (critical > 0 && value < critical) return 'bg-status-danger-500'
+ if (warning > 0 && value < warning) return 'bg-status-warning-500'
+ return 'bg-status-success-500'
}
// Get status badge colors
@@ -287,12 +287,12 @@ const getStatusColors = (value, metric) => {
if (!statusDef) return 'bg-gray-100 text-gray-600'
const colorMap = {
- green: 'bg-green-100 text-green-800',
- red: 'bg-red-100 text-red-800',
- yellow: 'bg-yellow-100 text-yellow-800',
+ green: 'bg-status-success-100 text-status-success-800',
+ red: 'bg-status-danger-100 text-status-danger-800',
+ yellow: 'bg-status-warning-100 text-status-warning-800',
gray: 'bg-gray-100 text-gray-600',
- blue: 'bg-blue-100 text-blue-800',
- orange: 'bg-orange-100 text-orange-800'
+ blue: 'bg-status-info-100 text-status-info-800',
+ orange: 'bg-status-urgent-100 text-status-urgent-800'
}
return colorMap[statusDef.color] || 'bg-gray-100 text-gray-600'
diff --git a/src/frontend/src/components/ReadOnlyToggle.vue b/src/frontend/src/components/ReadOnlyToggle.vue
index 5d2a32b21..464d999df 100644
--- a/src/frontend/src/components/ReadOnlyToggle.vue
+++ b/src/frontend/src/components/ReadOnlyToggle.vue
@@ -5,7 +5,7 @@
class="font-medium whitespace-nowrap min-w-[4.25rem] text-right"
:class="[
labelSizeClass,
- modelValue ? 'text-rose-600 dark:text-rose-400' : 'text-gray-500 dark:text-gray-400'
+ modelValue ? 'text-state-locked-600 dark:text-state-locked-400' : 'text-gray-500 dark:text-gray-400'
]"
>
{{ modelValue ? 'Read-Only' : 'Editable' }}
@@ -20,7 +20,7 @@
class="relative inline-flex items-center rounded-full transition-colors focus:outline-none focus:ring-2 focus:ring-offset-2 dark:focus:ring-offset-gray-800"
:class="[
sizeClasses,
- modelValue ? 'bg-rose-500 focus:ring-rose-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
+ modelValue ? 'bg-state-locked-500 focus:ring-state-locked-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
(disabled || loading) ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer'
]"
>
diff --git a/src/frontend/src/components/ReplayTimeline.vue b/src/frontend/src/components/ReplayTimeline.vue
index 51684631c..51df70932 100644
--- a/src/frontend/src/components/ReplayTimeline.vue
+++ b/src/frontend/src/components/ReplayTimeline.vue
@@ -326,7 +326,8 @@
class="transition-all duration-300 cursor-pointer hover:opacity-90 hover:stroke-white hover:stroke-2"
@click="navigateToExecution(activity)"
>
- {{ getBarTooltip(activity) }} (Click to open in new tab)
+ {{ getBarTooltip(activity) }}
+(Click to open in new tab)
@@ -665,7 +666,11 @@ const agentRows = computed(() => {
scheduleName: event.schedule_name,
// For navigation to execution details
executionId: event.execution_id,
- agentName: event.source_agent
+ agentName: event.source_agent,
+ // #514: previews + error text for hover tooltip
+ commandPreview: event.command_preview || '',
+ responsePreview: event.response_preview || '',
+ errorText: event.error || ''
})
})
@@ -721,7 +726,11 @@ const agentRows = computed(() => {
scheduleName: act.scheduleName,
// For navigation to execution details
executionId: act.executionId,
- agentName: act.agentName
+ agentName: act.agentName,
+ // #514: previews carried through to getBarTooltip
+ commandPreview: act.commandPreview,
+ responsePreview: act.responsePreview,
+ errorText: act.errorText
}
})
@@ -1093,7 +1102,29 @@ function getBarTooltip(activity) {
? `~${formatDuration(activity.durationMs)}`
: formatDuration(activity.durationMs)
- return `${prefix} ${status} - ${duration}`.trim().replace(' ', ' ')
+ // Header line — same as before.
+ const header = `${prefix} ${status} - ${duration}`.trim().replace(' ', ' ')
+
+ // #514: optional command + response/error preview lines.
+ const lines = [header]
+ const cmd = previewLine(activity.commandPreview)
+ if (cmd) lines.push(`Cmd: ${cmd}`)
+ if (activity.hasError && activity.errorText) {
+ lines.push(`Error: ${previewLine(activity.errorText)}`)
+ } else {
+ const resp = previewLine(activity.responsePreview)
+ if (resp) lines.push(`Out: ${resp}`)
+ }
+ return lines.join('\n')
+}
+
+// #514: collapse newlines, trim, truncate to 80 chars with an ellipsis.
+// Empty/whitespace-only input returns empty so the tooltip line is skipped.
+function previewLine(text) {
+ if (!text) return ''
+ const collapsed = String(text).replace(/\s+/g, ' ').trim()
+ if (!collapsed) return ''
+ return collapsed.length > 80 ? collapsed.slice(0, 80) + '…' : collapsed
}
// Schedule marker helper functions
diff --git a/src/frontend/src/components/RunningStateToggle.vue b/src/frontend/src/components/RunningStateToggle.vue
index 51875ca1b..153416f6f 100644
--- a/src/frontend/src/components/RunningStateToggle.vue
+++ b/src/frontend/src/components/RunningStateToggle.vue
@@ -5,7 +5,7 @@
class="font-medium whitespace-nowrap min-w-[3.25rem] text-right"
:class="[
labelSizeClass,
- modelValue ? 'text-green-600 dark:text-green-400' : 'text-gray-500 dark:text-gray-400'
+ modelValue ? 'text-status-success-600 dark:text-status-success-400' : 'text-gray-500 dark:text-gray-400'
]"
>
{{ modelValue ? 'Running' : 'Stopped' }}
@@ -19,7 +19,7 @@
class="relative inline-flex items-center rounded-full transition-colors focus:outline-none focus:ring-2 focus:ring-offset-2 dark:focus:ring-offset-gray-800"
:class="[
sizeClasses,
- modelValue ? 'bg-green-500 focus:ring-green-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
+ modelValue ? 'bg-status-success-500 focus:ring-status-success-500' : 'bg-gray-300 dark:bg-gray-600 focus:ring-gray-400',
(disabled || loading) ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer'
]"
>
diff --git a/src/frontend/src/components/RuntimeBadge.vue b/src/frontend/src/components/RuntimeBadge.vue
index 3d75214ae..159390b61 100644
--- a/src/frontend/src/components/RuntimeBadge.vue
+++ b/src/frontend/src/components/RuntimeBadge.vue
@@ -100,14 +100,11 @@ const tooltipText = computed(() => {
const badgeClasses = computed(() => {
if (isClaudeRuntime.value) {
- // Anthropic brand colors - coral/terracotta
- return 'bg-orange-50 dark:bg-orange-950/50 text-orange-700 dark:text-orange-300 border border-orange-200 dark:border-orange-800'
+ return 'bg-brand-claude-50 dark:bg-brand-claude-950/50 text-brand-claude-700 dark:text-brand-claude-300 border border-brand-claude-200 dark:border-brand-claude-800'
}
if (isGeminiRuntime.value) {
- // Google Gemini brand colors - blue/purple gradient feel
- return 'bg-blue-50 dark:bg-blue-950/50 text-blue-700 dark:text-blue-300 border border-blue-200 dark:border-blue-800'
+ return 'bg-brand-gemini-50 dark:bg-brand-gemini-950/50 text-brand-gemini-700 dark:text-brand-gemini-300 border border-brand-gemini-200 dark:border-brand-gemini-800'
}
- // Fallback
return 'bg-gray-100 dark:bg-gray-700 text-gray-700 dark:text-gray-300 border border-gray-200 dark:border-gray-600'
})
diff --git a/src/frontend/src/components/SchedulesPanel.vue b/src/frontend/src/components/SchedulesPanel.vue
index cb60468f9..594c75e09 100644
--- a/src/frontend/src/components/SchedulesPanel.vue
+++ b/src/frontend/src/components/SchedulesPanel.vue
@@ -986,10 +986,10 @@ function formatContextPercent(used, max) {
function getContextBarColor(used, max) {
if (!used || !max) return 'bg-gray-400'
const percent = (used / max) * 100
- if (percent < 50) return 'bg-green-500'
- if (percent < 75) return 'bg-yellow-500'
- if (percent < 90) return 'bg-orange-500'
- return 'bg-red-500'
+ if (percent < 50) return 'bg-status-success-500'
+ if (percent < 75) return 'bg-status-warning-500'
+ if (percent < 90) return 'bg-status-urgent-500'
+ return 'bg-status-danger-500'
}
function viewExecutionDetail(exec) {
diff --git a/src/frontend/src/components/SharingPanel.vue b/src/frontend/src/components/SharingPanel.vue
index ce263a5b5..d951a176b 100644
--- a/src/frontend/src/components/SharingPanel.vue
+++ b/src/frontend/src/components/SharingPanel.vue
@@ -222,6 +222,12 @@
+
+
+
+
+
+
@@ -238,6 +244,7 @@ import PublicLinksPanel from './PublicLinksPanel.vue'
import SlackChannelPanel from './SlackChannelPanel.vue'
import TelegramChannelPanel from './TelegramChannelPanel.vue'
import WhatsAppChannelPanel from './WhatsAppChannelPanel.vue'
+import FileSharingPanel from './FileSharingPanel.vue'
const props = defineProps({
agentName: {
diff --git a/src/frontend/src/components/SlackChannelPanel.vue b/src/frontend/src/components/SlackChannelPanel.vue
index bfb6c9c36..2b315d7a3 100644
--- a/src/frontend/src/components/SlackChannelPanel.vue
+++ b/src/frontend/src/components/SlackChannelPanel.vue
@@ -30,17 +30,37 @@
{{ channel.workspace_name }}
- (DM default)
-
- {{ unbinding ? 'Removing...' : 'Unbind' }}
-
+
+
+
+
+ DM default
+
+
+ {{ makingDefault ? 'Setting...' : 'Make default' }}
+
+
+ {{ unbinding ? 'Removing...' : 'Unbind' }}
+
+
@@ -70,7 +90,7 @@
diff --git a/src/frontend/src/components/StatusIndicator.vue b/src/frontend/src/components/StatusIndicator.vue
index 67b0b86d6..c1e3c3682 100644
--- a/src/frontend/src/components/StatusIndicator.vue
+++ b/src/frontend/src/components/StatusIndicator.vue
@@ -23,11 +23,11 @@ const props = defineProps({
const statusClasses = computed(() => {
switch (props.status) {
case 'in_progress':
- return 'bg-blue-500'
+ return 'bg-status-info-500'
case 'completed':
- return 'bg-green-500'
+ return 'bg-status-success-500'
case 'error':
- return 'bg-red-500'
+ return 'bg-status-danger-500'
case 'pending':
default:
return 'bg-gray-300 border border-gray-400'
diff --git a/src/frontend/src/components/TasksPanel.vue b/src/frontend/src/components/TasksPanel.vue
index 46473b452..4b25f8e31 100644
--- a/src/frontend/src/components/TasksPanel.vue
+++ b/src/frontend/src/components/TasksPanel.vue
@@ -1108,10 +1108,10 @@ function formatDuration(ms) {
function getContextBarColor(used, max) {
if (!used || !max) return 'bg-gray-400'
const percent = (used / max) * 100
- if (percent < 50) return 'bg-green-500'
- if (percent < 75) return 'bg-yellow-500'
- if (percent < 90) return 'bg-orange-500'
- return 'bg-red-500'
+ if (percent < 50) return 'bg-status-success-500'
+ if (percent < 75) return 'bg-status-warning-500'
+ if (percent < 90) return 'bg-status-urgent-500'
+ return 'bg-status-danger-500'
}
// Start polling
diff --git a/src/frontend/src/components/chat/ChatBubble.vue b/src/frontend/src/components/chat/ChatBubble.vue
index e9946ebd3..43979fa1a 100644
--- a/src/frontend/src/components/chat/ChatBubble.vue
+++ b/src/frontend/src/components/chat/ChatBubble.vue
@@ -2,21 +2,21 @@
-
+
-
{{ content }}
+
{{ content }}
{{ formattedTime }}
-
+
@@ -64,7 +64,7 @@
-
+
diff --git a/src/frontend/src/components/chat/ChatEmptyState.vue b/src/frontend/src/components/chat/ChatEmptyState.vue
new file mode 100644
index 000000000..6a634c1b0
--- /dev/null
+++ b/src/frontend/src/components/chat/ChatEmptyState.vue
@@ -0,0 +1,84 @@
+
+
+
+
{{ heading }}
+
+ {{ subheading }}
+
+
+
+
+
+ {{ item.label }}
+
+
+ {{ item.hint }}
+
+
+
+
+
+
+
diff --git a/src/frontend/src/components/chat/ChatHistoryDropdown.vue b/src/frontend/src/components/chat/ChatHistoryDropdown.vue
new file mode 100644
index 000000000..ff9102d76
--- /dev/null
+++ b/src/frontend/src/components/chat/ChatHistoryDropdown.vue
@@ -0,0 +1,158 @@
+
+
+
+
+
+
+
+ History
+
+
+
+
+
+
+
+
+
+
+ Previous Sessions
+ Logged-in chats only
+
+
+
+
+
+
+
+ {{ error }}
+
+
+
+
+ No previous sessions found.
+
+
+
+
+
+
+
+
+
+
diff --git a/src/frontend/src/components/chat/ChatInput.vue b/src/frontend/src/components/chat/ChatInput.vue
index 4b9be6275..274c33df3 100644
--- a/src/frontend/src/components/chat/ChatInput.vue
+++ b/src/frontend/src/components/chat/ChatInput.vue
@@ -1,5 +1,10 @@
-
+
+
+
+
+
+
+
+
{{ f.name }}
+
+
+
+
+
+
+
+
+
+
+ Drop files here
+
+