Skip to content

[daily-security-observability] Restore and persist 7-day cache snapshot with stale-age gating#40881

Merged
pelikhan merged 2 commits into
mainfrom
copilot/cache-strategy-fix-cache-miss
Jun 22, 2026
Merged

[daily-security-observability] Restore and persist 7-day cache snapshot with stale-age gating#40881
pelikhan merged 2 commits into
mainfrom
copilot/cache-strategy-fix-cache-miss

Conversation

Copilot AI commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

The Daily Security Observability workflow declared shared cache-memory but did not reliably reuse prior 7-day DIFC state, causing cold recomputation despite available history. This change adds an explicit warm-start + staleness gate and ensures the updated snapshot is written back on every run.

  • Cache warm-start (startup restore)

    • Added restore logic from /tmp/gh-aw/cache-memory/security-observability/filtered-logs.snapshot.json.
    • Loads cached runs into /tmp/gh-aw/agent/integrity/filtered-logs.json when cache metadata is present and valid.
  • Explicit stale-data detection

    • Added age validation using updated_at with a 7-day TTL.
    • Treats missing/expired metadata as stale and falls back to fresh collection.
  • Fresh + cached state merge

    • Downloads fresh --filtered-integrity logs into a separate file.
    • Merges cached and fresh runs, deduping by run_id with deterministic precedence.
  • Always persist updated snapshot

    • Writes the merged 7-day view back to cache-memory each run with:
      • updated_at
      • runs
      • summary.total_runs
  • Workflow guidance alignment

    • Updated Phase 3 instructions to reflect warm-start validation and stale-cache handling semantics.
CACHE_FILE=/tmp/gh-aw/cache-memory/security-observability/filtered-logs.snapshot.json
RUN_FILE=/tmp/gh-aw/agent/integrity/filtered-logs.json
FRESH_LOGS=/tmp/gh-aw/agent/integrity/filtered-logs.fresh.json
MAX_CACHE_AGE_SECONDS=$((7 * 24 * 60 * 60))

# restore if fresh; otherwise treat as stale
# collect fresh logs
# merge cached+fresh by run_id
# persist merged snapshot with updated_at

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix cache miss in Daily Security Observability Report [daily-security-observability] Restore and persist 7-day cache snapshot with stale-age gating Jun 22, 2026
Copilot AI requested a review from pelikhan June 22, 2026 22:58
@pelikhan pelikhan marked this pull request as ready for review June 22, 2026 23:12
Copilot AI review requested due to automatic review settings June 22, 2026 23:12
@pelikhan pelikhan merged commit faedd0e into main Jun 22, 2026
1 check passed
@pelikhan pelikhan deleted the copilot/cache-strategy-fix-cache-miss branch June 22, 2026 23:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Daily Security Observability workflow to reuse a cached 7‑day DIFC integrity-filtered log snapshot across runs, apply a staleness gate (7 days), merge cached + fresh results deterministically, and write the updated snapshot back each run. It also updates Phase 3 guidance to reflect the new warm-start semantics.

Changes:

  • Added warm-start restore from a cached snapshot file and a 7‑day updated_at TTL gate.
  • Downloaded fresh logs to a separate file, merged cached+fresh runs by run_id (fresh wins), and always persisted a merged snapshot back to cache-memory.
  • Updated Phase 3 instructions to describe warm-start validation and stale-cache behavior.
Show a summary per file
File Description
.github/workflows/daily-security-observability.md Adds warm-start/merge/persist logic for DIFC integrity-filtered logs and updates Phase 3 instructions accordingly.
.github/workflows/daily-security-observability.lock.yml Regenerates the compiled workflow to include the updated run script.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 4

Comment on lines +40 to +44
mkdir -p /tmp/gh-aw/cache-memory/security-observability

CACHE_FILE=/tmp/gh-aw/cache-memory/security-observability/filtered-logs.snapshot.json
RUN_FILE=/tmp/gh-aw/agent/integrity/filtered-logs.json
FRESH_LOGS=/tmp/gh-aw/agent/integrity/filtered-logs.fresh.json
|| echo 0
)
cache_age_seconds=$((NOW_EPOCH - cache_updated_epoch))
if [ "$cache_updated_epoch" -gt 0 ] && [ "$cache_age_seconds" -le "$MAX_CACHE_AGE_SECONDS" ]; then
MAX_CACHE_AGE_SECONDS=$((7 * 24 * 60 * 60))

# Warm start from cached 7-day snapshot when available and fresh.
if [ -f "$CACHE_FILE" ] && jq -e '.runs and .updated_at' "$CACHE_FILE" > /dev/null 2>&1; then
Comment on lines +95 to +96
' "$RUN_FILE" "$FRESH_LOGS" > "$RUN_FILE.merged"
mv "$RUN_FILE.merged" "$RUN_FILE"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants