Develop#8
Conversation
- add upload preflight/chunk/merge APIs and UploadService - introduce object storage client and upload-related settings - support merge conflict strategy and frontend type field - add upload service tests and setting defaults test - include python-multipart dependency
There was a problem hiding this comment.
Pull request overview
This PR syncs branches by bringing in a large set of features spanning the web UI (public share access + archive extraction), API typings/clients, mock handlers, and substantial backend additions (background job framework, worker consumer, archive/upload/share services), plus new Postgres init scripts and local docker-compose files.
Changes:
- Web: add a public share access route/page with download/preview/save-to-my-space flows and share password management UI.
- Web: add archive preview/extract UI (dialog + context menu action) backed by new file/job API calls and mock job handlers.
- Backend/infra: introduce background job model, Redis-stream queue, worker runtime, archive tasks/effects, upload endpoints, and new Postgres init SQL + docker dev compose.
Reviewed changes
Copilot reviewed 71 out of 82 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| web/src/utils/http.ts | Adds skipAuth request option and safer Authorization header injection. |
| web/src/types/share.d.ts | Extends share settings/update requests; adds save-share request/response types. |
| web/src/types/file.d.ts | Adds merge conflict strategy typing and archive preview/extract + job typings. |
| web/src/router/routes.ts | Adds /share/:shareLink route for public share access. |
| web/src/router/gurad.ts | Allows ShareAccess route to bypass email verification redirect. |
| web/src/pages/share/index.ts | Exports the new ShareAccess page entrypoint. |
| web/src/pages/share/ShareAccess.vue | New public share access page: load/access, download, preview, save-to-space. |
| web/src/pages/files/components/FileItemsView.vue | Adds “Extract…” menu option for archive-like files; emits extractArchive. |
| web/src/pages/files/components/ExtractArchiveDialog.vue | New dialog for archive preview + extract job orchestration via polling. |
| web/src/pages/files/MyFiles.vue | Wires ExtractArchiveDialog into My Files page and handles extract action. |
| web/src/mock/state.ts | Adds mock job store and includes mock share password field. |
| web/src/mock/index.ts | Registers new jobs mock handlers. |
| web/src/mock/handlers/share.ts | Adds share download/preview/save mocks; password handling + sanitization. |
| web/src/mock/handlers/jobs.ts | Adds mock GET /jobs/:id handler. |
| web/src/mock/handlers/file.ts | Adds mock archive preview/extract endpoints producing “job” records. |
| web/src/main.ts | Makes mock loading conditional via VITE_ENABLE_MOCKS. |
| web/src/components/common/ShareDialog.vue | Adds password edit/regenerate/copy UI; updates share settings save logic. |
| web/src/components/common/SelectFolderDialog.vue | New folder picker dialog used for saving shared items. |
| web/src/components/common/MoveItemDialog.vue | Adds configurable title/prompt/confirm/root label (reused by extract dialog). |
| web/src/api/share.ts | Public share endpoints now skipAuth; adds save/download/preview API calls. |
| web/src/api/job.ts | Adds job polling API client. |
| web/src/api/file.ts | Adds archive preview/extract job request APIs. |
| docs/schema-summary.sql | Removes old schema summary entrypoint SQL. |
| docker/postgresql-16/init/90_comments.sql | Adds table/column comments for many entities/views. |
| docker/postgresql-16/init/50_maintenance_views.sql | Adds maintenance functions/triggers and rebuild logic + various views. |
| docker/postgresql-16/init/40_audit_security.sql | Adds audit log, notifications, security events, moderation/scan result tables. |
| docker/postgresql-16/init/30_access_share.sql | Adds ACL/sharing/favorites and related indexes/triggers. |
| docker/postgresql-16/init/25_worker.sql | Adds background_job table and indexes/triggers. |
| docker/postgresql-16/init/20_storage.sql | Adds storage objects, file/folder tables, upload task tables, preview/media tables. |
| docker/postgresql-16/init/10_identity.sql | Adds identity tables (users, sessions, tokens, groups). |
| docker/postgresql-16/init/00_base.sql | Adds enum types, pg_trgm extension, and common trigger function. |
| docker/postgresql-16/docker-compose.yaml | Adds local Postgres 16 compose setup + init mounting. |
| docker/minio/docker-compose.yaml | Adds local MinIO compose setup. |
| app/tests/test_workers.py | Tests task dispatch unknown type + retry delay behavior. |
| app/tests/test_upload_service.py | Adds upload service tests (preflight/chunk/merge edge cases). |
| app/tests/test_tasks_transcode.py | Tests ffmpeg command building and transcode flow with mocked subprocess. |
| app/tests/test_tasks_scan.py | Tests heuristic scan behavior for benign and disguised executable files. |
| app/tests/test_tasks_archive.py | Tests archive detection, preview, extraction, and path sanitization/limits. |
| app/tests/test_share_service.py | Tests share token roundtrip and key share service behaviors. |
| app/tests/test_settings.py | Tests settings defaults for upload-related configuration. |
| app/tests/test_effects_archive_extract.py | Tests DB effects application for archive extract manifest processing. |
| app/src/workers/repository.py | Adds job status transitions (running/succeeded/failed/retrying) + backoff. |
| app/src/workers/effects.py | Adds DB side-effects for scan/transcode/archive extract results. |
| app/src/workers/dispatcher.py | Adds task execution bridge to the task registry. |
| app/src/workers/contracts.py | Adds WorkerJobMessage contract. |
| app/src/workers/consumer.py | Adds Redis-stream worker consumer with process pool execution and retries. |
| app/src/workers/bootstrap.py | Adds worker runtime config and process pool creation. |
| app/src/workers/init.py | Exports worker entrypoints. |
| app/src/tasks/transcode.py | Implements ffprobe/ffmpeg based transcode task. |
| app/src/tasks/scan.py | Implements heuristic dangerous file scan task. |
| app/src/tasks/registry.py | Adds task registry + dispatch with helpful error for unknown task types. |
| app/src/tasks/init.py | Exposes tasks/dispatch publicly. |
| app/src/services/job_queue.py | Implements Redis stream publisher/consumer wrapper. |
| app/src/services/background_jobs.py | Implements enqueueing + publishing jobs with error handling. |
| app/src/services/archive.py | Adds archive preview/extract job creation service. |
| app/src/services/init.py | Re-exports newly added services. |
| app/src/schemas/share.py | Adds password/regenerate fields + save-to-space request/response schemas. |
| app/src/schemas/job.py | Adds BackgroundJobResponse schema + conversion helper. |
| app/src/schemas/file.py | Adds merge conflict strategy field to MergeChunksRequest. |
| app/src/schemas/archive.py | Adds ArchiveExtractRequest schema. |
| app/src/schemas/init.py | Exports new share save schemas. |
| app/src/s3/minio_client.py | Adds MinIO object storage client wrapper. |
| app/src/s3/init.py | Exports MinIO client types. |
| app/src/routers/uploads.py | Adds upload preflight/chunk/merge endpoints. |
| app/src/routers/shares.py | Adds share access/save + shared file streaming download/preview endpoints. |
| app/src/routers/jobs.py | Adds job status endpoint scoped to requesting user. |
| app/src/routers/files.py | Adds archive preview/extract job creation endpoints. |
| app/src/routers/init.py | Registers new routers in the API. |
| app/src/models/tables_worker.py | Adds SQLAlchemy model for background_job. |
| app/src/models/tables.py | Exports BackgroundJob in models tables module. |
| app/src/models/init.py | Exposes BackgroundJob from models package. |
| app/src/core/settings.py | Adds worker/object-storage/upload/archive settings and derived properties. |
| app/src/core/security.py | Adds share access token create/decode helpers. |
| app/src/core/errors.py | Extends ApiError to carry data through standard error responses. |
| app/src/core/deps.py | Wires new services (storage, jobs, share/upload/archive) into FastAPI deps. |
| app/src/.env.example | Documents new env vars for storage/upload/worker configuration. |
| app/pyproject.toml | Adds dependencies for multipart uploads and 7z support. |
| .gitignore | Ignores additional local cache/artifact directories. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| async function bootstrap() { | ||
| const enableMocks = import.meta.env.VITE_ENABLE_MOCKS !== 'false'; | ||
| if (enableMocks) { | ||
| await import('./mock'); | ||
| } | ||
|
|
||
| const app = createApp(App); | ||
| const pinia = createPinia(); | ||
| const app = createApp(App); | ||
| const pinia = createPinia(); | ||
|
|
||
| app.use(pinia); | ||
| app.use(router); | ||
| app.use(pinia); | ||
| app.use(router); | ||
|
|
||
| app.mount('#app'); | ||
| } | ||
|
|
||
| bootstrap(); |
There was a problem hiding this comment.
bootstrap() is async and is invoked without awaiting or a .catch handler. If the dynamic mock import fails (or any other bootstrap error occurs), it will create an unhandled promise rejection and the app may never mount. Consider calling bootstrap().catch(...) (or wrapping in an IIFE with try/catch) to ensure failures are surfaced and don’t silently break startup.
| environment: | ||
| POSTGRES_USER: admin | ||
| POSTGRES_PASSWORD: psgl-ff-db | ||
| POSTGRES_DB: fileflash | ||
| TZ: Asia/Shanghai |
There was a problem hiding this comment.
This docker-compose file hardcodes database credentials (POSTGRES_PASSWORD) directly in the repo. Even for local dev, this pattern increases the chance of accidental reuse in non-local environments. Prefer using an external .env file (not committed) and referencing variables here, or use clearly non-sensitive placeholder values and document how to override them.
| environment: | ||
| MINIO_ROOT_USER: "admin" | ||
| MINIO_ROOT_PASSWORD: "minio-admin" | ||
| command: server /data --console-address ":9001" |
There was a problem hiding this comment.
This docker-compose file hardcodes MinIO root credentials in-repo. Recommend sourcing these from environment variables / a local .env (not committed) to reduce the risk of credential reuse or accidental exposure outside local development.
| } else if ( | ||
| needsVerification && | ||
| to.name !== VERIFY_EMAIL_ROUTE_NAME && | ||
| to.name !== PROFILE_ROUTE_NAME && | ||
| to.name !== SHARE_ACCESS_ROUTE_NAME | ||
| ) { | ||
| next({ name: VERIFY_EMAIL_ROUTE_NAME }); | ||
| } else if (requiresAdmin && !hasAdminAccess) { |
There was a problem hiding this comment.
The newly added block is indented with tab characters and is inconsistent with the rest of the file’s 2-space indentation (and likely the project formatter). This will cause noisy diffs and may fail lint/format checks. Reformat this block to match the existing indentation style (spaces).
| @@ -66,13 +67,14 @@ const fetchPermissions = async () => { | |||
| }; | |||
|
|
|||
| const hydrateShareSettings = (share: Share) => { | |||
| shareSettings.value = { | |||
| passwordProtected: share.settings.passwordProtected, | |||
| expireAt: share.settings.expireAt ? share.settings.expireAt.slice(0, 10) : '', | |||
| allowDownload: share.settings.allowDownload, | |||
| allowPreview: share.settings.allowPreview, | |||
| }; | |||
| }; | |||
| shareSettings.value = { | |||
| passwordProtected: share.settings.passwordProtected, | |||
| password: share.settings.password || '', | |||
| expireAt: share.settings.expireAt ? share.settings.expireAt.slice(0, 10) : '', | |||
| allowDownload: share.settings.allowDownload, | |||
| allowPreview: share.settings.allowPreview, | |||
| }; | |||
| }; | |||
There was a problem hiding this comment.
This section introduces tab-indented lines (e.g., in the shareSettings object and hydrateShareSettings), which is inconsistent with the surrounding 2-space indentation used throughout the file. Please reformat these lines to the project’s standard spacing to avoid formatter/linter churn.
sync branches