feat(audit): platform audit trail Phase 1 + lifecycle smoke test (#20)

[dolho]

Summary

First slice of SEC-001 / Issue #20 — the cross-cutting platform audit log that complements the Process Engine's existing audit_entries table. Two commits:

• Phase 1 (d4e39be): infrastructure — schema, immutability triggers, service, DB ops, admin query API, 24 unit tests
• Phase 2a (845a1c2): agent lifecycle smoke test — 4 audit calls in routers/agents.py + 5 integration-shape tests, so the audit log actually produces rows on real user actions

Issue #20 stays open — Phase 2b (auth, sharing, settings, credentials, rename, request_id middleware), Phase 3 (MCP TypeScript audit), and Phase 4 (hash chain, export, retention, compliance guide) follow as separate PRs.

What ships

Database

• New audit_log table, 6 indexes matching the spec's documented query patterns, 2 immutability triggers
• Append-only enforcement at the SQLite layer:
  • UPDATE blocked unconditionally
  • DELETE blocked within the 365-day retention window
  • DELETE of older entries allowed so a future retention cleanup can run
• Migration audit_log_table (System Manifest UI #31) creates the table on existing installs, idempotent via PRAGMA table_info guard

Service & DB layer

• db/audit.py — PlatformAuditOperations (insert, query, count, single fetch, range, stats) following the class-per-domain pattern (invariant Feature/gemini runtime support #2)
• services/platform_audit_service.py — PlatformAuditService with global instance, AuditEventType / AuditActorType enums, actor resolution (user > agent > system > mcp_client), JSON details encoding
• Audit failures never raise. Wrapped in try/except, logged at ERROR, return None. Caller's primary operation is never affected. Verified by test_service_never_raises_on_db_failure.
• Hash chain code present but dormant behind _hash_chain_enabled = False for Phase 4
• DatabaseManager exposes 6 thin delegate methods

API

• routers/audit_log.py mounted at /api/audit-log to coexist with the existing /api/audit Process Engine router (no breaking change to existing consumers)
• Three admin-only endpoints:
  • GET /api/audit-log — list with filters and pagination
  • GET /api/audit-log/stats — aggregate counts by event_type and actor_type
  • GET /api/audit-log/{event_id} — single entry by UUID
• Every endpoint depends on require_admin
• Route ordering: /stats declared before /{event_id} so it isn't shadowed (invariant fix: add missing logging_config.py to backend Dockerfile #4)

Lifecycle integration (Phase 2a)

Four handlers in routers/agents.py emit audit rows after a successful state transition:

Handler	Event action	Details payload
create_agent_endpoint	create	{template, base_image, agent_type}
start_agent_endpoint	start	{credentials_injection}
stop_agent_endpoint	stop	null
delete_agent_endpoint	delete	null

All four use the same kwargs shape: actor_user=current_user, actor_ip from request.client, target_type=\"agent\", target_id=agent_name, endpoint=request.url.path. Log call lives in the router, not the service, so current_user stays in scope without threading through service signatures. Placement is always after the state transition has committed — if the action fails partway, no audit row appears.

Tests

• tests/test_audit_log_unit.py — 29 unit tests covering:
  • Schema, query filters, pagination, immutability triggers (UPDATE blocked, DELETE within retention blocked, DELETE >1yr allowed)
  • Service actor resolution for all four actor types (user / agent / mcp_client / system)
  • JSON details serialization, unique event_id generation, request-metadata persistence
  • The "audit failures must not raise" contract
  • Lifecycle integration shape — 5 tests asserting create/start/stop/delete produce correctly-shaped rows; one test verifying the full create→start→stop→delete flow produces 4 rows in temporal order
• All 29/29 passing in 1.2s

Docs

• docs/memory/feature-flows/audit-trail.md — new flow doc with strict-format sections (Overview, Problem, User Story, Entry Points, Frontend Layer, Backend Layer, Side Effects, Error Handling, Security Considerations, Phase 2a integration, Out of Scope, Testing, Related Flows)
• docs/memory/feature-flows.md — index entry
• docs/memory/requirements.md §20.1 — status flipped to 🚧 Phase 1 + Phase 2a implemented
• docs/memory/architecture.md — new API endpoints table + new audit_log schema block

Manual verification

End-to-end smoke test against the local backend after merge:

TOKEN=...  # admin JWT or trinity_mcp_* key

# 1. Baseline empty
curl -s -H \"Authorization: Bearer $TOKEN\" \"http://localhost:8000/api/audit-log?limit=5\" | jq

# 2. Trigger lifecycle
curl -X POST http://localhost:8000/api/agents -H \"Authorization: Bearer $TOKEN\" -H 'Content-Type: application/json' -d '{\"name\":\"audit-test\",\"template\":\"business-assistant\"}'
curl -X POST http://localhost:8000/api/agents/audit-test/start -H \"Authorization: Bearer $TOKEN\"
curl -X POST http://localhost:8000/api/agents/audit-test/stop -H \"Authorization: Bearer $TOKEN\"
curl -X DELETE http://localhost:8000/api/agents/audit-test -H \"Authorization: Bearer $TOKEN\"

# 3. Expect 4 rows newest-first: delete, stop, start, create
curl -s -H \"Authorization: Bearer $TOKEN\" \"http://localhost:8000/api/audit-log?target_id=audit-test\" | jq

# 4. Stats
curl -s -H \"Authorization: Bearer $TOKEN\" http://localhost:8000/api/audit-log/stats | jq

# 5. Immutability — these should fail
docker exec trinity-backend python3 -c \"import sqlite3; sqlite3.connect('/data/trinity.db').execute(\\\"UPDATE audit_log SET event_action='tampered' WHERE target_id='audit-test'\\\")\"
docker exec trinity-backend python3 -c \"import sqlite3; sqlite3.connect('/data/trinity.db').execute(\\\"DELETE FROM audit_log WHERE target_id='audit-test'\\\")\"

This was actually run against the local backend during PR development — all 12 verification steps passed (table created, lifecycle calls produce rows with correct actor / target / details / timestamps, stats endpoint aggregates correctly, single-entry lookup works, bad token returns 401, UPDATE and DELETE both blocked by triggers). The audit log is observably useful end-to-end.

Design decisions worth eyeballing

Decision	Rationale
/api/audit-log prefix, not /api/audit	Existing routers/audit.py exposes the Process Engine's separate audit system at /api/audit. Mounting platform audit at the same prefix would either break that or require renaming it (breaking change for any current consumer). Two distinct prefixes, both intentional per spec; can be unified later via a ?system= query param.
Class is PlatformAuditService, not AuditService	Process Engine already has a services/process_engine/services/audit.py::AuditService. Two classes with the same name in the same Python process is a foot-gun. Prefix avoids the collision.
Hash chain code dormant in Phase 1	~25 lines of unreachable code in PlatformAuditService for the Phase 4 hash chain. Kept because Phase 4 will turn it on; ripping it out now means a larger Phase 4 PR. Documented in the flow doc.
Log calls live in the router, not the service	Keeps current_user in scope without threading through service signatures. Each call is 10 lines in the router. Easier to test.
No frontend	Per spec — the audit data is for compliance pipelines and incident response (SQL queries, CSV export, external SIEM ingestion), not interactive browsing. Phase 4 ships export. The spec's success criteria are all backend / data quality, with zero UI requirements. If we want a Trinity-native admin UI, that's a Phase 5 spec amendment.

What's NOT in this PR (intentional Phase 2b/3/4 deferrals)

• Authentication audit (login success/failure)
• Authorization audit (share, permission grants)
• Settings change audit
• Credential operation audit
• Agent rename / container recreate audit
• MCP TypeScript tool-call audit (Phase 3)
• request_id middleware for cross-event correlation
• Hash chain verification, CSV/JSON export, retention automation (Phase 4)
• Frontend admin UI

Issue #20 stays open until all four phases ship.

Test plan

• Unit tests pass (pytest tests/test_audit_log_unit.py -v — 29/29 in 1.2s)
• Compile check on all modified files
• Schema + triggers verified end-to-end against an in-memory SQLite (insert OK, UPDATE blocked, DELETE within retention blocked, DELETE >1yr allowed)
• Manual end-to-end against local backend (12-step verification covering rebuild, migration, table creation, baseline empty, lifecycle calls, audit rows appear, stats, single-entry lookup, auth gate, immutability)
• Phase 2b integrations follow as separate PRs

Refs #20

🤖 Generated with Claude Code

[vybe]

LGTM - PR validation passed all checks. Clean three-layer architecture, comprehensive documentation, strong test coverage (29 tests), and security-conscious design.