Skip to content

feat(audit): daily retention prune for audit_log entries past 365d (#552)#576

Merged
vybe merged 1 commit into
devfrom
feature/552-audit-retention-prune
Apr 29, 2026
Merged

feat(audit): daily retention prune for audit_log entries past 365d (#552)#576
vybe merged 1 commit into
devfrom
feature/552-audit-retention-prune

Conversation

@dolho

@dolho dolho commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the gap left by SEC-001 Phase 4: the append-only audit log was protected by SQLite triggers but never pruned, so audit_log grew unbounded. Adds a daily APScheduler job that DELETEs entries older than AUDIT_LOG_RETENTION_DAYS (default 365, floored at 365 to respect the audit_log_no_delete trigger floor).

  • New service: src/backend/services/audit_retention_service.py — mirrors log_archive_service shape, runs at 04:15 UTC daily.
  • New DB op: PlatformAuditOperations.prune_audit_log() + db.prune_audit_log() facade.
  • Wired into main.py lifespan start/stop.
  • Architecture note added to the Background Services table.

Why the SQL deliberately uses datetime('now', ?)

Architectural invariant #16 prefers iso_cutoff() for rolling-window filters on ISO-Z TEXT columns. Here the audit_log_no_delete trigger's WHEN clause already uses datetime('now', '-365 days') — using the same form in our WHERE clause keeps the prune set and the trigger's protected set in sync, avoiding IntegrityError on day-of-cutoff boundary rows. The trigger has the underlying ISO-Z mismatch noted in invariant #16; fixing that is tracked separately to keep this PR scope-bounded.

Hash chain note

Pruning DELETEs rows, which breaks the previous_hash/entry_hash chain across the cutoff. Hash-chain verification ranges should stay inside the retention window by design. Documented in the service docstring; if hash chain is enabled and prune removes >0 rows, a structured warning is logged.

Configuration

Env var Default Notes
AUDIT_LOG_RETENTION_DAYS 365 Floored at 365 (trigger-bounded).
AUDIT_RETENTION_ENABLED true Set false to disable the scheduler.
AUDIT_RETENTION_HOUR 4 UTC hour to run; runs at :15 past.

Closes #552.

Test plan

  • Unit: `pytest unit/test_audit_retention_prune.py` — 4/4 pass
    • removes only old rows
    • empty-table returns 0
    • retention_days < 365 raises before touching DB
    • boundary-row stress (mix of 1d/100d/200d/360d/366d/380d/400d/720d) — proves WHERE + trigger don't drift
  • Manual: insert a synthetic 400d-old row, restart backend, wait for 04:15 UTC (or call `audit_retention_service.prune()` from a Python REPL inside the backend container), verify the row is gone and the structured log line appears.

🤖 Generated with Claude Code

)

Closes the gap left by SEC-001 Phase 4: the append-only contract was
enforced by SQLite triggers but nothing pruned aged-out rows, so
audit_log grew unbounded.

Adds a daily APScheduler job (`audit_retention_service.py`) that
DELETEs `audit_log` rows older than `AUDIT_LOG_RETENTION_DAYS`
(default 365). Floored at 365 because the `audit_log_no_delete`
trigger refuses younger rows. Disable with
`AUDIT_RETENTION_ENABLED=false`; runs at `AUDIT_RETENTION_HOUR:15`
(default 04:15 UTC, one hour after log archival to spread DB writes).

The prune SQL deliberately uses the same `datetime('now', ?)` form as
the trigger's WHEN clause — this trades architectural invariant #16
(prefer `iso_cutoff()`) for trigger consistency, avoiding
IntegrityError on day-of-cutoff boundary rows. Fixing the trigger to
use ISO-Z form is tracked separately.

Hash chain note: pruning DELETEs entries, breaking the SHA-256 chain
across the cutoff. Verification ranges should stay within retention
by design — documented in the service docstring.

Tests cover: old-only removal, empty-table return, sub-365 floor
rejection, and a boundary-row stress case that fails loudly if the
WHERE clause and trigger ever drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — well-patterned background service, invariant #16 deviation correctly justified, boundary-stress test covers the trigger/WHERE alignment. Approved.

@vybe
vybe merged commit 2d4589e into dev Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants