fix(rate-limit): admins bypass storyboard eval rate limits#2729
Merged
Conversation
Admins hit the 5/hour storyboard evaluation cap (and 30/hour step cap) while curating and debugging — the limit is one shared bucket per user across all storyboards, so even browsing-heavy admin work exhausts it. Add a `skipForAdmins` helper mirroring `requireAdmin` semantics (user `isAdmin` flag → ADMIN_EMAILS → `isWebUserAAOAdmin`) and wire it into both `storyboardEvalRateLimiter` and `storyboardStepRateLimiter` via express-rate-limit's async `skip` option. Non-admins are unaffected.
Contributor
|
deeper question - is 5/hour reasonable? doesn't seem like it? |
bokelley
added
admin-tool
bokelley
force-pushed
the
EmmaLouise2018/admin-rate-limit
branch
from
436663e to
481a7ca
Compare
This was referenced Apr 26, 2026
bokelley
pushed a commit
that referenced
this pull request
Apr 26, 2026
…tests
The top-level `import { invalidateMemberContextCache }` on member-tools.ts
forced member-context.ts (and transitively auth.ts) to load at module init,
triggering the eager WorkOS constructor before vi.mock could intercept it.
Removed the static import; both call sites in the fire-and-forget .then()
callbacks now use dynamic `await import('../member-context.js')` — same
pattern as line 2195 and as the fix applied in #2729.
https://claude.ai/code/session_018QWw9FNTxNgKLejx54HYs2
bokelley
pushed a commit
that referenced
this pull request
Apr 26, 2026
…tests
The top-level `import { invalidateMemberContextCache }` on member-tools.ts
forced member-context.ts (and transitively auth.ts) to load at module init,
triggering the eager WorkOS constructor before vi.mock could intercept it.
Removed the static import; both call sites in the fire-and-forget .then()
callbacks now use dynamic `await import('../member-context.js')` — same
pattern as line 2195 and as the fix applied in #2729.
https://claude.ai/code/session_018QWw9FNTxNgKLejx54HYs2
bokelley
added a commit
that referenced
this pull request
Apr 27, 2026
…3258) * feat(addie): instrument agent test runs for staleness-aware prompts (#3254) Adds `agent_test_runs` table and hydrates `agent_testing` on `MemberContext` so #2299 Stage 2 can fire a "Time for a fresh agent test" suggested prompt to builder personas who haven't run a test recently. https://claude.ai/code/session_018QWw9FNTxNgKLejx54HYs2 * fix(addie): lazy-import invalidateMemberContextCache to unblock unit tests The top-level `import { invalidateMemberContextCache }` on member-tools.ts forced member-context.ts (and transitively auth.ts) to load at module init, triggering the eager WorkOS constructor before vi.mock could intercept it. Removed the static import; both call sites in the fire-and-forget .then() callbacks now use dynamic `await import('../member-context.js')` — same pattern as line 2195 and as the fix applied in #2729. https://claude.ai/code/session_018QWw9FNTxNgKLejx54HYs2 * fix(addie): reconcile agent_testing type and renumber migration 444 Two post-rebase conflicts against main: 1. Remove fetchAgentTesting stub (main's narrow type) — keep this PR's richer AgentTestingContext shape (total_tests_30d + 4-state outcome) which the staleness-prompt rule actually depends on. 2. Renumber 442_agent_test_runs.sql → 444_agent_test_runs.sql; main claimed 442 (addie_prompt_telemetry_clicks) and 443 (agent_test_history_user_id_index) while this branch was in review. https://claude.ai/code/session_01W4Djuhqn6SJJk5qBSPSdTt --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
server/src/middleware/rate-limit.ts:12-29), so a handful of clicks across different agents/storyboards exhausts it.skipForAdminshelper mirroringrequireAdminsemantics (pre-resolveduser.isAdmin→ADMIN_EMAILSenv var → asyncisWebUserAAOAdmin(userId)with its built-in cache) and wires it into bothstoryboardEvalRateLimiterandstoryboardStepRateLimitervia express-rate-limit's asyncskipoption.Test plan
/runstill returns 429 after 5 evaluations within an hour./runcalls within an hour all succeed (modulo normal failures).ADMIN_EMAILSenv var override (no working group row), confirm same bypass behavior./storyboard/:id/compareand/applicable-storyboardsalso bypass for admins.isAdmin: truebypasses without hitting the DB admin check.