Skip to content

fix(files): cache TUS upload access checks#2258

Closed
riderx wants to merge 1 commit into
mainfrom
codex/cache-tus-file-db-checks
Closed

fix(files): cache TUS upload access checks#2258
riderx wants to merge 1 commit into
mainfrom
codex/cache-tus-file-db-checks

Conversation

@riderx
Copy link
Copy Markdown
Member

@riderx riderx commented May 13, 2026

Summary (AI generated)

  • Added a tracked migration for the production app_versions.r2_path index that Supabase recommended for the TUS/files ready-bundle guard.
  • Cached successful and ready-bundle TUS attachment write-access checks for exact API key, org, app, and file path matches so repeated HEAD/PATCH calls do not repeat the same DB work during an active upload.
  • Fixed /stats fallback version lookup so a deleted unknown placeholder can be reused instead of repeatedly scheduling no-op placeholder inserts.
  • Added focused regressions for the TUS write-access cache and the deleted unknown stats fallback.

Motivation (AI generated)

Supabase reported heavy production volume from the ready-bundle app_versions.r2_path lookup. That lookup comes from checkWriteAppAccess in the files/TUS upload route. The repeated placeholder insert comes from /stats falling back to unknown with the wrong deleted filter, which can miss the placeholder and schedule ensurePlaceholderVersions again on every request.

Business Impact (AI generated)

This reduces unnecessary database pressure on hot upload and stats paths, keeps the production index tracked in migrations, and lowers the risk of DB saturation from resumable-upload loops or stats traffic. Upload integrity stays intact because TUS create requests still perform the full primary DB validation and only repeated chunk/progress checks reuse a short-lived exact-match cache.

Test Plan (AI generated)

  • bun lint
  • bun lint:backend
  • bun test tests/files-upload-access-cache.unit.test.ts
  • bun test tests/stats.test.ts --test-timeout 20000
  • Commit hook: bun run cli:build && vue-tsc --noEmit
  • bunx supabase migration list --local blocked because local Supabase Postgres is not running on 127.0.0.1:54322 in this worktree.

Generated with AI

Summary by CodeRabbit

  • Performance Improvements

    • Implemented caching mechanism for upload authorization decisions to minimize database queries during file transfer operations.
    • Added database index to optimize app version lookups.
  • Bug Fixes

    • Corrected fallback behavior for deleted app versions in the statistics endpoint.

Review Change Stack

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 13, 2026

📝 Walkthrough

Walkthrough

This PR adds a caching layer for TUS upload write-access decisions to avoid repeated authorization checks, updates the stats endpoint to unconditionally allow deleted app versions in fallback scenarios, optimizes a database lookup with a new index, and includes comprehensive test coverage for both features.

Changes

Attachment Write Access Caching

Layer / File(s) Summary
Cache infrastructure and types
supabase/functions/_backend/files/files.ts
Imports CacheHelper and defines cache configuration constants (path, TTL) plus AttachmentWriteAccessCachePayload type holding cached authorization status, identity markers, and expiry.
Caching helper functions
supabase/functions/_backend/files/files.ts
Implements method-gated cache operations: building cache requests via CacheHelper, validating entries by identity + expiry, backfilling cache asynchronously, persisting cached status values, and throwing standardized 409 conflicts for ready-bundle paths.
Authorization flow integration
supabase/functions/_backend/files/files.ts
Updates checkWriteAppAccess to consult cache first for HEAD/PATCH requests, short-circuiting on allowed status and rejecting on ready_bundle; caches both ready_bundle (before 409) and allowed (on grant) statuses.
Database index optimization
supabase/migrations/20260513093535_app_versions_r2_path_index.sql
Adds concurrent btree index on public.app_versions r2_path column to optimize ready-bundle path lookups.
Write-access cache unit tests
tests/files-upload-access-cache.unit.test.ts
New test file with in-memory cache mock, base64 metadata encoding, and mocked backend dependencies. Verifies POST initializes upload with single DB setup and subsequent HEAD requests reuse cached authorization without additional PG calls.

Stats Endpoint Deleted Version Handling

Layer / File(s) Summary
Unknown version fallback
supabase/functions/_backend/plugins/stats.ts
Changes getAppVersionPostgres fallback to unconditionally pass true for allowedDeleted instead of a conditional flag, ensuring consistent deleted-record handling.
Stats deleted unknown test
tests/stats.test.ts
New test verifying stats endpoint uses deleted unknown placeholder when app is marked deleted with version_build containing -missing. segment, including proper cleanup.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • Cap-go/capgo#2037: Both PRs modify checkWriteAppAccess logic to detect and block uploads to ready bundle paths, with this PR adding outcome caching during TUS authorization.
  • Cap-go/capgo#1964: Both PRs modify checkWriteAppAccess in the TUS attachment write authorization path, one adding caching and the other refactoring with stricter path/authz gating.

Poem

A rabbit hops through cache layers bright, 🐰✨
Where TUS uploads find their might,
Stats endpoints know when versions disappear,
Indexes flutter, optimization's near!
Deleted records sing in harmony,
One swift lookup beats a thousand spree.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The PR description includes a comprehensive summary of changes, motivation, business impact, and test plan, but the provided description template requires specific sections (Summary, Test plan, Screenshots, Checklist) with checkboxes that are not fully completed in the PR description format. Ensure the PR description follows the template structure more closely, particularly completing the Checklist section with checked/unchecked boxes and confirming all required sections are present and filled.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix(files): cache TUS upload access checks' clearly and specifically summarizes the main change—implementing a cache for TUS upload access validation checks in the files module.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/cache-tus-file-db-checks

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 SQLFluff (4.1.0)
supabase/migrations/20260513093535_app_versions_r2_path_index.sql

User Error: No dialect was specified. You must configure a dialect or specify one on the command line using --dialect after the command. Available dialects:
ansi, athena, bigquery, clickhouse, databricks, db2, doris, duckdb, exasol, flink, greenplum, hive, impala, mariadb, materialize, mysql, oracle, postgres, redshift, snowflake, soql, sparksql, sqlite, starrocks, teradata, trino, tsql, vertica


Comment @coderabbitai help to get the list of available commands and usage tips.

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented May 13, 2026

Merging this PR will not alter performance

✅ 43 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing codex/cache-tus-file-db-checks (e486364) with main (1a31fd5)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/files-upload-access-cache.unit.test.ts`:
- Around line 83-97: The beforeEach and afterEach hooks are out of conventional
order in the test file; move the beforeEach block (which clears mocks and sets
up getPgClientMock, getUserIdFromApikeyMock, getAppByAppIdPgMock,
getAppByIdPgMock, checkPermissionPgMock) so it appears before the afterEach
block that restores globalThis.caches, ensuring hooks read: beforeEach(...) then
afterEach(...); update only the ordering around the beforeEach and afterEach
symbols in tests/files-upload-access-cache.unit.test.ts.
- Around line 11-13: The test uses the global Buffer in encodeMetadataValue
which ESLint flags; fix it by adding an explicit import from the buffer module
(import { Buffer } from 'buffer') at the top of the file and keep using Buffer
inside encodeMetadataValue so the symbol is resolved from the import rather than
the global; update any other test helpers in the same file that reference Buffer
to use the imported symbol as well.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3ff99e34-9460-44cc-a7e6-88e522549155

📥 Commits

Reviewing files that changed from the base of the PR and between 1a31fd5 and e486364.

📒 Files selected for processing (5)
  • supabase/functions/_backend/files/files.ts
  • supabase/functions/_backend/plugins/stats.ts
  • supabase/migrations/20260513093535_app_versions_r2_path_index.sql
  • tests/files-upload-access-cache.unit.test.ts
  • tests/stats.test.ts

Comment on lines +11 to +13
function encodeMetadataValue(value: string) {
return Buffer.from(value).toString('base64')
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Import Buffer explicitly instead of using the global.

ESLint flags using the global Buffer variable. Import it explicitly for clarity and Node.js module compatibility.

Proposed fix
 import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { Buffer } from 'node:buffer'
 
 const fileId = 'orgs/test-org/apps/test-app/cache-test.zip'
🧰 Tools
🪛 ESLint

[error] 12-12: Unexpected use of the global variable 'Buffer'. Use 'require("buffer").Buffer' instead.

(node/prefer-global/buffer)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/files-upload-access-cache.unit.test.ts` around lines 11 - 13, The test
uses the global Buffer in encodeMetadataValue which ESLint flags; fix it by
adding an explicit import from the buffer module (import { Buffer } from
'buffer') at the top of the file and keep using Buffer inside
encodeMetadataValue so the symbol is resolved from the import rather than the
global; update any other test helpers in the same file that reference Buffer to
use the imported symbol as well.

Comment on lines +83 to +97
afterEach(() => {
globalThis.caches = originalCaches
})

beforeEach(() => {
vi.clearAllMocks()
getPgClientMock.mockReturnValue({
end: vi.fn(() => Promise.resolve()),
query: vi.fn(async () => ({ rows: [] })),
})
getUserIdFromApikeyMock.mockResolvedValue('00000000-0000-0000-0000-000000000001')
getAppByAppIdPgMock.mockResolvedValue({ app_id: 'test-app', owner_org: 'test-org' })
getAppByIdPgMock.mockResolvedValue({ plan_valid: true })
checkPermissionPgMock.mockResolvedValue(true)
})
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Place beforeEach before afterEach to follow testing conventions.

ESLint's test/prefer-hooks-in-order rule expects beforeEach hooks to appear before afterEach hooks for readability and conventional ordering.

Proposed fix: swap hook order
 describe('files upload access cache', () => {
-  afterEach(() => {
-    globalThis.caches = originalCaches
-  })
-
   beforeEach(() => {
     vi.clearAllMocks()
     getPgClientMock.mockReturnValue({
       end: vi.fn(() => Promise.resolve()),
       query: vi.fn(async () => ({ rows: [] })),
     })
     getUserIdFromApikeyMock.mockResolvedValue('00000000-0000-0000-0000-000000000001')
     getAppByAppIdPgMock.mockResolvedValue({ app_id: 'test-app', owner_org: 'test-org' })
     getAppByIdPgMock.mockResolvedValue({ plan_valid: true })
     checkPermissionPgMock.mockResolvedValue(true)
   })

+  afterEach(() => {
+    globalThis.caches = originalCaches
+  })
+
   it('reuses cached TUS write access for repeated HEAD checks', async () => {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
afterEach(() => {
globalThis.caches = originalCaches
})
beforeEach(() => {
vi.clearAllMocks()
getPgClientMock.mockReturnValue({
end: vi.fn(() => Promise.resolve()),
query: vi.fn(async () => ({ rows: [] })),
})
getUserIdFromApikeyMock.mockResolvedValue('00000000-0000-0000-0000-000000000001')
getAppByAppIdPgMock.mockResolvedValue({ app_id: 'test-app', owner_org: 'test-org' })
getAppByIdPgMock.mockResolvedValue({ plan_valid: true })
checkPermissionPgMock.mockResolvedValue(true)
})
beforeEach(() => {
vi.clearAllMocks()
getPgClientMock.mockReturnValue({
end: vi.fn(() => Promise.resolve()),
query: vi.fn(async () => ({ rows: [] })),
})
getUserIdFromApikeyMock.mockResolvedValue('00000000-0000-0000-0000-000000000001')
getAppByAppIdPgMock.mockResolvedValue({ app_id: 'test-app', owner_org: 'test-org' })
getAppByIdPgMock.mockResolvedValue({ plan_valid: true })
checkPermissionPgMock.mockResolvedValue(true)
})
afterEach(() => {
globalThis.caches = originalCaches
})
🧰 Tools
🪛 ESLint

[error] 87-97: beforeEach hooks should be before any afterEach hooks

(test/prefer-hooks-in-order)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/files-upload-access-cache.unit.test.ts` around lines 83 - 97, The
beforeEach and afterEach hooks are out of conventional order in the test file;
move the beforeEach block (which clears mocks and sets up getPgClientMock,
getUserIdFromApikeyMock, getAppByAppIdPgMock, getAppByIdPgMock,
checkPermissionPgMock) so it appears before the afterEach block that restores
globalThis.caches, ensuring hooks read: beforeEach(...) then afterEach(...);
update only the ordering around the beforeEach and afterEach symbols in
tests/files-upload-access-cache.unit.test.ts.

@sonarqubecloud
Copy link
Copy Markdown

@digzrow-coder
Copy link
Copy Markdown

The allowed-write cache looks like it can bypass the ready-bundle immutability guard during the 60s TTL.

POST /upload/attachments performs the primary DB checks and then stores an allowed cache entry for the exact file_id. Subsequent HEAD/PATCH requests hit getCachedAttachmentWriteAccess() before opening the primary connection, so an allowed cache hit skips the app_versions lookup that blocks finalized bundle paths. If the upload is finalized and an app_versions row is created for the same r2_path while the cache entry is still alive, a later resumable PATCH for that same path can continue through next() instead of returning bundle_already_ready.

That seems to weaken the new ready-bundle guard exactly when the state changes from upload-in-progress to finalized. I would avoid caching allowed across mutating PATCH requests, or include an upload/session state that is invalidated when the bundle becomes ready. A regression can reproduce this by doing the POST to seed allowed, changing the mocked ready-bundle query to return a row, then asserting a following PATCH does not reuse the allowed cache and returns 409.

@riderx riderx closed this May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants