Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 57 additions & 40 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,64 +1,81 @@
# Application
# Core self-host runtime
NEXT_PUBLIC_APP_URL=http://localhost:3000

# Better Auth (Authentication)
BETTER_AUTH_SECRET=your-better-auth-secret
BETTER_AUTH_URL=http://localhost:3000
NEXT_PUBLIC_BETTER_AUTH_URL=http://localhost:3000
BETTER_AUTH_SECRET=your-better-auth-secret

# Google OAuth (for Better Auth)
GOOGLE_CLIENT_ID=...
GOOGLE_CLIENT_SECRET=...

# Database Configuration
# Database
# For Docker PostgreSQL (recommended):
# DATABASE_URL=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex
# For local PostgreSQL:
# DATABASE_URL=postgresql://user:password@localhost:5432/thinkex
DATABASE_URL=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex

# PostgreSQL Docker Configuration (used by docker-compose.yml)
# PostgreSQL Docker defaults used by docker-compose.yml and setup.sh
POSTGRES_USER=thinkex
POSTGRES_PASSWORD=thinkex_password_change_me
POSTGRES_DB=thinkex
POSTGRES_PORT=5432

# File storage (Supabase Storage — required for uploads)
# Create a public bucket named `file-upload` in the Supabase dashboard.
# Zero cache (required for workspace sync in local development)
NEXT_PUBLIC_ZERO_SERVER=http://localhost:4848
ZERO_UPSTREAM_DB=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex
ZERO_QUERY_URL=http://localhost:3000/api/zero/query
ZERO_MUTATE_URL=http://localhost:3000/api/zero/mutate
ZERO_MUTATE_FORWARD_COOKIES=true
ZERO_QUERY_FORWARD_COOKIES=true
ZERO_APP_PUBLICATIONS=zero_pub
ZERO_ADMIN_PASSWORD=change-me-before-sharing
ZERO_APP_ID=zero_local_dev
ZERO_COOKIE_DOMAIN=localhost

# Storage
# Core self-host mode uses local files. This supports uploads/viewing inside
# ThinkEx, but media processing features that call third-party providers still
# require provider-reachable object storage.
STORAGE_TYPE=local
UPLOADS_DIR=./uploads

# Optional provider-reachable storage (required for OCR/audio/doc conversion)
NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
SUPABASE_SERVICE_ROLE_KEY=eyJ...

# Google AI
# AI backend
# Chat and autogen use the AI backend configured for your deployment. In local
# setups that use Google directly, set GOOGLE_GENERATIVE_AI_API_KEY.
GOOGLE_GENERATIVE_AI_API_KEY=AIza...

# AssemblyAI (audio transcription)
ASSEMBLYAI_API_KEY=your-assemblyai-api-key

# E2B Code Execution Sandbox
E2B_API_KEY=e2b_...

# Firecrawl Web Scraping (Optional)
# Get your API key from firecrawl.dev
FIRECRAWL_API_KEY=fc_...

# FastAPI Service (optional - file conversion, doc-to-markdown, audio/video analysis)
# FASTAPI_BASE_URL=https://your-fastapi-service.com
# FASTAPI_API_KEY=your-service-api-key
# Optional OAuth and export integrations
GOOGLE_CLIENT_ID=...
GOOGLE_CLIENT_SECRET=...
NEXT_PUBLIC_GOOGLE_CLIENT_ID=...

# Mistral OCR
# Optional media and retrieval integrations
MISTRAL_API_KEY=your-mistral-api-key
MISTRAL_OCR_ENDPOINT=https://api.mistral.ai/v1/ocr
MISTRAL_OCR_MODEL=mistral-ocr-latest
ASSEMBLYAI_API_KEY=your-assemblyai-api-key
FASTAPI_BASE_URL=https://your-fastapi-service.com
FASTAPI_API_KEY=your-service-api-key
FIRECRAWL_API_KEY=fc_...
YOUTUBE_API_KEY=...

# Supermemory (memory layer for chat — https://console.supermemory.ai)
# Optional: when unset, the memory toggle is a no-op server-side.
# Optional chat integrations
E2B_API_KEY=e2b_...
SUPERMEMORY_API_KEY=sm_...

# Kill switch for the Supermemory wrapper. Must be exactly "true" to enable.
# `@supermemory/tools@1.4.3+` contains the Proxy-based fix for AI SDK v6
# (https://github.com/supermemoryai/supermemory/pull/854). If you downgrade
# the dep or upstream regresses, set this to "false" to neutralize the
# wrapper without ripping out any UI/transport plumbing.
SUPERMEMORY_ENABLED=true
# Optional: MISTRAL_OCR_ENDPOINT=https://api.mistral.ai/v1/ocr
# Optional: MISTRAL_OCR_MODEL=mistral-ocr-latest
# Zero Sync
NEXT_PUBLIC_ZERO_SERVER=http://localhost:4848
ZERO_COOKIE_DOMAIN=localhost

# Optional observability and email
NEXT_PUBLIC_POSTHOG_HOST=https://us.i.posthog.com
NEXT_PUBLIC_POSTHOG_TOKEN=phc_...
POSTHOG_HOST=https://us.i.posthog.com
POSTHOG_PROJECT_TOKEN=phc_...
POSTHOG_PERSONAL_API_KEY=phx_...
POSTHOG_PROJECT_ID=...
POSTHOG_RELEASE_NAME=thinkex
POSTHOG_RELEASE_VERSION=
RESEND_API_KEY=re_...

# Optional platform-specific settings
CHROME_EXTENSION_ID=
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Contributions should prioritize:
- Node.js **v20+**
- pnpm
- PostgreSQL (local or Docker)
- Zero cache server configuration from [SELF_HOSTING.md](SELF_HOSTING.md)

### Quick Setup (Recommended)

Expand All @@ -31,6 +32,7 @@ Run the interactive setup script:
git clone https://github.com/ThinkEx-OSS/thinkex.git
cd thinkex
./setup.sh
pnpm dev
```

### Manual Setup
Expand All @@ -42,3 +44,5 @@ cp .env.example .env
pnpm db:push
pnpm dev
```

Use [SELF_HOSTING.md](SELF_HOSTING.md) as the source of truth for environment variables, Zero startup, local storage, and optional media integrations.
126 changes: 31 additions & 95 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,112 +58,48 @@ Nothing disappears into a black box. You see what AI sees and control what it wo

## Self-Hosting

ThinkEx can be self hosted for local development. The setup uses Docker for PostgreSQL (recommended) while running the Next.js app locally.
ThinkEx supports a **core self-host** path for local development:

### Quick Start

#### Prerequisites
- PostgreSQL
- Better Auth secret + app URL
- Zero cache server
- Local filesystem storage

* [Node.js](https://nodejs.org/) (v20+)
* [pnpm](https://pnpm.io/) (will be installed automatically if missing)
* [Docker](https://docs.docker.com/get-docker/) (recommended for PostgreSQL) OR [PostgreSQL](https://www.postgresql.org/download/) (v12+) installed locally
* **Required API Keys:**
* **Google AI**: API key from [Google AI Studio](https://aistudio.google.com/app/apikey)
* `GOOGLE_GENERATIVE_AI_API_KEY`
* **Supabase** (file uploads): Project URL and keys from [Supabase](https://supabase.com)
* `NEXT_PUBLIC_SUPABASE_URL`, `NEXT_PUBLIC_SUPABASE_ANON_KEY`, `SUPABASE_SERVICE_ROLE_KEY`
* **Optional API Keys:**
* **Google OAuth**: Get credentials from [Google Cloud Console](https://console.cloud.google.com/apis/credentials) (for OAuth login)
* `GOOGLE_CLIENT_ID`
* `GOOGLE_CLIENT_SECRET`
* **Other optional keys** (web scraping, OCR, external conversion service): see [`.env.example`](.env.example) (`FIRECRAWL_API_KEY`, `MISTRAL_API_KEY`, `FASTAPI_*`, `SCRAPING_MODE`, etc.)
`pnpm dev` is the supported one-command entrypoint for this setup. It starts:

#### Automated Setup
- Next.js
- the AI SDK devtools
- Zero

Run the interactive setup script:
### Quick Start

```bash
git clone https://github.com/ThinkEx-OSS/thinkex.git
cd thinkex
./setup.sh
pnpm dev
```

The script will:
- Check prerequisites (Node.js, pnpm, Docker)
- Create `.env` file from template
- Generate `BETTER_AUTH_SECRET` automatically
- Start PostgreSQL in Docker (or use local PostgreSQL if Docker is not available)
- Configure database connection
- Install dependencies
- Initialize the database schema
- Start the development server automatically

Access ThinkEx at [http://localhost:3000](http://localhost:3000)

**PostgreSQL Docker Commands:**
- Stop PostgreSQL: `docker-compose down`
- Start PostgreSQL: `docker-compose up -d`
- View logs: `docker-compose logs -f postgres`

#### Manual Setup

1. **Clone the repository**
```bash
git clone https://github.com/ThinkEx-OSS/thinkex.git
cd thinkex
```

2. **Start PostgreSQL (Docker)**
```bash
docker-compose up -d postgres
```

Or use your local PostgreSQL installation.

3. **Install dependencies**
```bash
pnpm install
```

4. **Configure environment variables**
```bash
cp .env.example .env
```

Edit `.env` and configure:

* **Database**: Set `DATABASE_URL` to your PostgreSQL connection string
```bash
# For Docker PostgreSQL:
DATABASE_URL=postgresql://thinkex:thinkex_password_change_me@localhost:5432/thinkex

# For local PostgreSQL:
DATABASE_URL=postgresql://user:password@localhost:5432/thinkex
```
* **Better Auth**: Generate `BETTER_AUTH_SECRET` with `openssl rand -base64 32`. For a public URL (not localhost), set `BETTER_AUTH_URL` and `NEXT_PUBLIC_APP_URL` to that origin.
* **Google OAuth**: Get credentials from [Google Cloud Console](https://console.cloud.google.com/apis/credentials)
* **Supabase**: Your Supabase project URL and keys (required for file uploads)
* **Google AI**: API key from [Google AI Studio](https://aistudio.google.com/app/apikey)

5. **Initialize the database**
```bash
pnpm db:push
```

6. **Start the development server**
```bash
pnpm dev
```

7. **Access the application**
Open [http://localhost:3000](http://localhost:3000) in your browser.

#### File storage (Supabase)

Uploads go to **Supabase Storage**. Configure:

- `NEXT_PUBLIC_SUPABASE_URL`, `NEXT_PUBLIC_SUPABASE_ANON_KEY`, `SUPABASE_SERVICE_ROLE_KEY`
- A storage bucket named `file-upload` set to **Public**
Access ThinkEx at [http://localhost:3000](http://localhost:3000).

### What Core Self-Host Includes

- Local uploads, viewing, and deletion using `STORAGE_TYPE=local`
- Workspace sync through Zero
- The main app shell and workspace flows

### What Core Self-Host Does Not Include

OCR, audio transcription, and office document conversion require provider-reachable object storage plus the relevant provider credentials. Those features are intentionally not supported in local-file core mode and will fail with explicit messages.

### Full Setup Guide

See [SELF_HOSTING.md](SELF_HOSTING.md) for:

- required local environment variables
- Zero configuration
- local storage behavior
- optional integrations such as Supabase, OCR, audio, analytics, and email

## Contributing

Expand Down
111 changes: 111 additions & 0 deletions SELF_HOSTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Self-Hosting ThinkEx

This repo currently supports a **core self-host** profile for local development:

- PostgreSQL
- Better Auth secret + app URL
- Zero cache server
- Local filesystem storage

`pnpm dev` is the supported one-command entrypoint. It starts:

- Next.js
- the AI SDK devtools
- Zero

## Quick Start

```bash
git clone https://github.com/ThinkEx-OSS/thinkex.git
cd thinkex
./setup.sh
pnpm dev
```

If you prefer to bootstrap manually:

```bash
pnpm install
cp .env.example .env
docker-compose up -d postgres
pnpm db:push
pnpm dev
```
Comment on lines +27 to +33

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Mention Compose v2 (docker compose) syntax.

docker-compose (v1) reached end-of-life in mid-2023; fresh Docker installs only ship docker compose (v2). setup.sh already falls back between both, but this manual snippet only documents v1, which will fail on common environments.

📝 Suggested wording
 pnpm install
 cp .env.example .env
-docker-compose up -d postgres
+docker compose up -d postgres   # or: docker-compose up -d postgres
 pnpm db:push
 pnpm dev
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@SELF_HOSTING.md` around lines 27 - 33, Docs show the deprecated
docker-compose (v1) command; update the example that currently lists
"docker-compose up -d postgres" to either use the Compose v2 "docker compose up
-d postgres" (preferred) or show both variants, and optionally note that
setup.sh already falls back between them so users on recent Docker installs
should run the v2 form; change the command in the code block in SELF_HOSTING.md
accordingly.


## Core Self-Host Environment

The minimum local profile needs these groups configured:

- App/auth:
- `NEXT_PUBLIC_APP_URL`
- `BETTER_AUTH_URL`
- `NEXT_PUBLIC_BETTER_AUTH_URL`
- `BETTER_AUTH_SECRET`
- Database:
- `DATABASE_URL`
- Zero:
- `NEXT_PUBLIC_ZERO_SERVER`
- `ZERO_UPSTREAM_DB`
- `ZERO_QUERY_URL`
- `ZERO_MUTATE_URL`
- `ZERO_MUTATE_FORWARD_COOKIES`
- `ZERO_QUERY_FORWARD_COOKIES`
- `ZERO_APP_PUBLICATIONS`
- `ZERO_ADMIN_PASSWORD`
- `ZERO_APP_ID`
- `ZERO_COOKIE_DOMAIN`
- Storage:
- `STORAGE_TYPE=local`
- `UPLOADS_DIR`

`./setup.sh` populates sensible local defaults for the core profile and creates the `zero_pub` publication used by the local Zero process.

## Zero

Zero is required for workspace sync in local development. ThinkEx does not currently support a non-Zero workspace mode.

Use:

```bash
pnpm dev
```

The repo also includes `pnpm run dev:zero` as a debugging hook, but `pnpm dev` is the supported workflow.

## Local File Storage

ThinkEx uses `STORAGE_TYPE=local` by default for the core self-host profile. That supports:

- local uploads
- viewing uploaded files inside the app
- deleting uploaded files

It does **not** make files reachable to third-party providers.

## Unsupported in Core Local Mode

These features require provider-reachable object storage and the relevant API keys:

- OCR
- audio transcription
- office document conversion

In local-file core mode, those flows are intentionally blocked with explicit capability errors.

## Optional Integrations

You can still configure optional integrations from `.env.example`, including:

- Supabase storage for provider-reachable uploads
- Google OAuth
- Mistral OCR
- AssemblyAI
- FastAPI conversion
- Firecrawl
- E2B
- PostHog
- Resend
- Supermemory
- YouTube metadata

Those are not required to boot the core self-host profile.
Loading
Loading