-
Notifications
You must be signed in to change notification settings - Fork 11
Streamline self-hosting and Zero local startup #461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
3a6fd22
Self-hosting and Zero startup flow
urjitc a5a9405
Fix storage mode fallback and local file serving
urjitc f71b91d
Harden local file serving and upload guards
urjitc 3c5917b
Merge origin/main into codex/self-host-phase1
urjitc c43bd22
Scope local files to the uploading user
urjitc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,64 +1,81 @@ | ||
| # Application | ||
| # Core self-host runtime | ||
| NEXT_PUBLIC_APP_URL=http://localhost:3000 | ||
|
|
||
| # Better Auth (Authentication) | ||
| BETTER_AUTH_SECRET=your-better-auth-secret | ||
| BETTER_AUTH_URL=http://localhost:3000 | ||
| NEXT_PUBLIC_BETTER_AUTH_URL=http://localhost:3000 | ||
| BETTER_AUTH_SECRET=your-better-auth-secret | ||
|
|
||
| # Google OAuth (for Better Auth) | ||
| GOOGLE_CLIENT_ID=... | ||
| GOOGLE_CLIENT_SECRET=... | ||
|
|
||
| # Database Configuration | ||
| # Database | ||
| # For Docker PostgreSQL (recommended): | ||
| # DATABASE_URL=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex | ||
| # For local PostgreSQL: | ||
| # DATABASE_URL=postgresql://user:password@localhost:5432/thinkex | ||
| DATABASE_URL=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex | ||
|
|
||
| # PostgreSQL Docker Configuration (used by docker-compose.yml) | ||
| # PostgreSQL Docker defaults used by docker-compose.yml and setup.sh | ||
| POSTGRES_USER=thinkex | ||
| POSTGRES_PASSWORD=thinkex_password_change_me | ||
| POSTGRES_DB=thinkex | ||
| POSTGRES_PORT=5432 | ||
|
|
||
| # File storage (Supabase Storage — required for uploads) | ||
| # Create a public bucket named `file-upload` in the Supabase dashboard. | ||
| # Zero cache (required for workspace sync in local development) | ||
| NEXT_PUBLIC_ZERO_SERVER=http://localhost:4848 | ||
| ZERO_UPSTREAM_DB=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex | ||
| ZERO_QUERY_URL=http://localhost:3000/api/zero/query | ||
| ZERO_MUTATE_URL=http://localhost:3000/api/zero/mutate | ||
| ZERO_MUTATE_FORWARD_COOKIES=true | ||
| ZERO_QUERY_FORWARD_COOKIES=true | ||
| ZERO_APP_PUBLICATIONS=zero_pub | ||
| ZERO_ADMIN_PASSWORD=change-me-before-sharing | ||
| ZERO_APP_ID=zero_local_dev | ||
| ZERO_COOKIE_DOMAIN=localhost | ||
|
|
||
| # Storage | ||
| # Core self-host mode uses local files. This supports uploads/viewing inside | ||
| # ThinkEx, but media processing features that call third-party providers still | ||
| # require provider-reachable object storage. | ||
| STORAGE_TYPE=local | ||
| UPLOADS_DIR=./uploads | ||
|
|
||
| # Optional provider-reachable storage (required for OCR/audio/doc conversion) | ||
| NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co | ||
| NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ... | ||
| SUPABASE_SERVICE_ROLE_KEY=eyJ... | ||
|
|
||
| # Google AI | ||
| # AI backend | ||
| # Chat and autogen use the AI backend configured for your deployment. In local | ||
| # setups that use Google directly, set GOOGLE_GENERATIVE_AI_API_KEY. | ||
| GOOGLE_GENERATIVE_AI_API_KEY=AIza... | ||
|
|
||
| # AssemblyAI (audio transcription) | ||
| ASSEMBLYAI_API_KEY=your-assemblyai-api-key | ||
|
|
||
| # E2B Code Execution Sandbox | ||
| E2B_API_KEY=e2b_... | ||
|
|
||
| # Firecrawl Web Scraping (Optional) | ||
| # Get your API key from firecrawl.dev | ||
| FIRECRAWL_API_KEY=fc_... | ||
|
|
||
| # FastAPI Service (optional - file conversion, doc-to-markdown, audio/video analysis) | ||
| # FASTAPI_BASE_URL=https://your-fastapi-service.com | ||
| # FASTAPI_API_KEY=your-service-api-key | ||
| # Optional OAuth and export integrations | ||
| GOOGLE_CLIENT_ID=... | ||
| GOOGLE_CLIENT_SECRET=... | ||
| NEXT_PUBLIC_GOOGLE_CLIENT_ID=... | ||
|
|
||
| # Mistral OCR | ||
| # Optional media and retrieval integrations | ||
| MISTRAL_API_KEY=your-mistral-api-key | ||
| MISTRAL_OCR_ENDPOINT=https://api.mistral.ai/v1/ocr | ||
| MISTRAL_OCR_MODEL=mistral-ocr-latest | ||
| ASSEMBLYAI_API_KEY=your-assemblyai-api-key | ||
| FASTAPI_BASE_URL=https://your-fastapi-service.com | ||
| FASTAPI_API_KEY=your-service-api-key | ||
| FIRECRAWL_API_KEY=fc_... | ||
| YOUTUBE_API_KEY=... | ||
|
|
||
| # Supermemory (memory layer for chat — https://console.supermemory.ai) | ||
| # Optional: when unset, the memory toggle is a no-op server-side. | ||
| # Optional chat integrations | ||
| E2B_API_KEY=e2b_... | ||
| SUPERMEMORY_API_KEY=sm_... | ||
|
|
||
| # Kill switch for the Supermemory wrapper. Must be exactly "true" to enable. | ||
| # `@supermemory/tools@1.4.3+` contains the Proxy-based fix for AI SDK v6 | ||
| # (https://github.com/supermemoryai/supermemory/pull/854). If you downgrade | ||
| # the dep or upstream regresses, set this to "false" to neutralize the | ||
| # wrapper without ripping out any UI/transport plumbing. | ||
| SUPERMEMORY_ENABLED=true | ||
| # Optional: MISTRAL_OCR_ENDPOINT=https://api.mistral.ai/v1/ocr | ||
| # Optional: MISTRAL_OCR_MODEL=mistral-ocr-latest | ||
| # Zero Sync | ||
| NEXT_PUBLIC_ZERO_SERVER=http://localhost:4848 | ||
| ZERO_COOKIE_DOMAIN=localhost | ||
|
|
||
| # Optional observability and email | ||
| NEXT_PUBLIC_POSTHOG_HOST=https://us.i.posthog.com | ||
| NEXT_PUBLIC_POSTHOG_TOKEN=phc_... | ||
| POSTHOG_HOST=https://us.i.posthog.com | ||
| POSTHOG_PROJECT_TOKEN=phc_... | ||
| POSTHOG_PERSONAL_API_KEY=phx_... | ||
| POSTHOG_PROJECT_ID=... | ||
| POSTHOG_RELEASE_NAME=thinkex | ||
| POSTHOG_RELEASE_VERSION= | ||
| RESEND_API_KEY=re_... | ||
|
|
||
| # Optional platform-specific settings | ||
| CHROME_EXTENSION_ID= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # Self-Hosting ThinkEx | ||
|
|
||
| This repo currently supports a **core self-host** profile for local development: | ||
|
|
||
| - PostgreSQL | ||
| - Better Auth secret + app URL | ||
| - Zero cache server | ||
| - Local filesystem storage | ||
|
|
||
| `pnpm dev` is the supported one-command entrypoint. It starts: | ||
|
|
||
| - Next.js | ||
| - the AI SDK devtools | ||
| - Zero | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ```bash | ||
| git clone https://github.com/ThinkEx-OSS/thinkex.git | ||
| cd thinkex | ||
| ./setup.sh | ||
| pnpm dev | ||
| ``` | ||
|
|
||
| If you prefer to bootstrap manually: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| cp .env.example .env | ||
| docker-compose up -d postgres | ||
| pnpm db:push | ||
| pnpm dev | ||
| ``` | ||
|
|
||
| ## Core Self-Host Environment | ||
|
|
||
| The minimum local profile needs these groups configured: | ||
|
|
||
| - App/auth: | ||
| - `NEXT_PUBLIC_APP_URL` | ||
| - `BETTER_AUTH_URL` | ||
| - `NEXT_PUBLIC_BETTER_AUTH_URL` | ||
| - `BETTER_AUTH_SECRET` | ||
| - Database: | ||
| - `DATABASE_URL` | ||
| - Zero: | ||
| - `NEXT_PUBLIC_ZERO_SERVER` | ||
| - `ZERO_UPSTREAM_DB` | ||
| - `ZERO_QUERY_URL` | ||
| - `ZERO_MUTATE_URL` | ||
| - `ZERO_MUTATE_FORWARD_COOKIES` | ||
| - `ZERO_QUERY_FORWARD_COOKIES` | ||
| - `ZERO_APP_PUBLICATIONS` | ||
| - `ZERO_ADMIN_PASSWORD` | ||
| - `ZERO_APP_ID` | ||
| - `ZERO_COOKIE_DOMAIN` | ||
| - Storage: | ||
| - `STORAGE_TYPE=local` | ||
| - `UPLOADS_DIR` | ||
|
|
||
| `./setup.sh` populates sensible local defaults for the core profile and creates the `zero_pub` publication used by the local Zero process. | ||
|
|
||
| ## Zero | ||
|
|
||
| Zero is required for workspace sync in local development. ThinkEx does not currently support a non-Zero workspace mode. | ||
|
|
||
| Use: | ||
|
|
||
| ```bash | ||
| pnpm dev | ||
| ``` | ||
|
|
||
| The repo also includes `pnpm run dev:zero` as a debugging hook, but `pnpm dev` is the supported workflow. | ||
|
|
||
| ## Local File Storage | ||
|
|
||
| ThinkEx uses `STORAGE_TYPE=local` by default for the core self-host profile. That supports: | ||
|
|
||
| - local uploads | ||
| - viewing uploaded files inside the app | ||
| - deleting uploaded files | ||
|
|
||
| It does **not** make files reachable to third-party providers. | ||
|
|
||
| ## Unsupported in Core Local Mode | ||
|
|
||
| These features require provider-reachable object storage and the relevant API keys: | ||
|
|
||
| - OCR | ||
| - audio transcription | ||
| - office document conversion | ||
|
|
||
| In local-file core mode, those flows are intentionally blocked with explicit capability errors. | ||
|
|
||
| ## Optional Integrations | ||
|
|
||
| You can still configure optional integrations from `.env.example`, including: | ||
|
|
||
| - Supabase storage for provider-reachable uploads | ||
| - Google OAuth | ||
| - Mistral OCR | ||
| - AssemblyAI | ||
| - FastAPI conversion | ||
| - Firecrawl | ||
| - E2B | ||
| - PostHog | ||
| - Resend | ||
| - Supermemory | ||
| - YouTube metadata | ||
|
|
||
| Those are not required to boot the core self-host profile. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mention Compose v2 (
docker compose) syntax.docker-compose(v1) reached end-of-life in mid-2023; fresh Docker installs only shipdocker compose(v2).setup.shalready falls back between both, but this manual snippet only documents v1, which will fail on common environments.📝 Suggested wording
🤖 Prompt for AI Agents