diff --git a/packages/process-compose/docs/architecture.md b/packages/process-compose/docs/architecture.md index 4e652d1237..e5f6635f8d 100644 --- a/packages/process-compose/docs/architecture.md +++ b/packages/process-compose/docs/architecture.md @@ -58,7 +58,9 @@ one of three conditions: `startService(name)` starts the requested definition and its transitive dependencies. `stopService(name)` and `restartService(name)` also include active dependents so a caller cannot -leave an already-running dependent attached to a restarted dependency. +leave an already-running dependent attached to a restarted dependency. The pure restart-closure +calculation preserves inactive connector services on a dependency path to an active descendant; +otherwise restarting the descendant without its connector would violate graph order. `updateServiceDefinition(name, replacement)` validates a graph with the replacement and then swaps the graph used for subsequent starts and restarts. It does not mutate a process generation that is @@ -110,7 +112,8 @@ redundant publications must compare before writing. `ServiceTransition` is the only normal path for observed-status changes. `applyEvent()` rejects illegal `(status, event)` pairs by returning `null`; `transition()` applies a legal event atomically through `SubscriptionRef.modifyEffect`. Races such as a late health callback during shutdown are -therefore ignored without corrupting state. +therefore ignored without corrupting state. The transition classification is keyed by every event +tag, so adding an event fails type checking until its legal source statuses are defined. ## Lifecycle of one process generation @@ -121,9 +124,9 @@ For each requested definition, `Orchestrator` runs this sequence: backoff, allowing a caller to reserve external resources. 3. Wait for dependencies, bounded by `dependencyTimeoutSeconds` (default: 120 seconds). 4. Transition to `Starting`. -5. Call `beforeSpawn` immediately before every spawn. The stack uses it to release a reserved port - as late as possible. For a supervised Docker command there is still a spawn-to-container-bind - window because no supervisor/container bind handshake exists. +5. Call `beforeSpawn` immediately before every spawn. A caller can use it to release a reserved port + as late as possible. For a supervised child there is still a spawn-to-bind window because no + supervisor/child bind handshake exists. 6. Spawn either the configured process or its optional supervisor. 7. Register a scoped finalizer before exposing `Running`. 8. Run `started` hooks sequentially. Only successful hooks allow `ProcessSpawned` to publish @@ -137,6 +140,10 @@ For each requested definition, `Orchestrator` runs this sequence: The service keeps the same state stream across restart generations. Restart backoff is `min(30 seconds, 2^(restartCount - 1))`. +A no-health-check, `restart: "no"` process is treated as one-shot work. A small isolated poll of +`ChildProcessHandle.isRunning` compensates for adapters that can report process completion before +their `exitCode` Effect becomes observable; it is not part of the general lifecycle loop. + ### Lifecycle hooks Hooks are caller-supplied Effects triggered on `started` or `healthy`. Hooks for one trigger run in @@ -212,9 +219,15 @@ In-process cleanup and orphan supervision solve different failure modes: owner's stdin closes, its PID disappears, the supervisor receives a termination signal, or the managed child exits while cleanup is configured. -`ExternalCleanupAction` currently supports removing a Docker container or a filesystem path. The -supervisor validates the decoded configuration and ignores individual cleanup failures so cleanup -remains best-effort and idempotent. +`ExternalCleanupAction` supports a shell-free `RunCommand` with an executable, argument array, and +optional timeout (default 5 seconds), plus `RemovePath` for filesystem cleanup. Cleanup commands +receive the managed child's sanitized environment, without the supervisor self-dispatch protocol +variables. The supervisor rejects a malformed decoded cleanup contract before spawning the child, +while individual execution failures remain best-effort. Callers must choose idempotent commands +because owner-loss signals can race. `RunCommand` actions execute serially, so their aggregate +duration can approach the sum of their individual timeouts. Each command runs in its own process +group on Unix, while Windows uses `taskkill /T`; a timeout therefore terminates the command tree +rather than only its root. Filesystem retry delays may overlap command execution. ## Supervisor runtime @@ -247,7 +260,8 @@ virtual filesystem. The package therefore uses three internal environment variab The CLI entrypoint calls `enableSupervisorSelfDispatchForCompiledBun()` before normal command dispatch, checks `isSupervisorRuntimeRequested()`, and invokes `runSupervisorRuntimeFromEnv()`. The supervisor removes all three variables before starting the managed command, preventing the -protocol from leaking recursively into a service. +protocol from leaking recursively into a service. Contract tests run the same encoded supervisor +configuration through both the source-file argument path and environment-based self-dispatch path. The stack daemon has a separate Supabase-owned marker, `SUPABASE_STACK_RUN_DAEMON`; it is documented in [the stack detach-mode guide](../../stack/docs/detach-mode.md#compiled-executable-re-entry). @@ -264,15 +278,13 @@ plus live per-service and merged `PubSub` streams. `historyAll` can filter by se Streams contain new entries only; callers explicitly request history when they need replay. When a process exits unexpectedly or becomes unhealthy, the orchestrator appends recent buffered -output to its diagnostics. `truncate` currently clears both a service's history and its entries in -the merged history; it has no production caller. +output to its diagnostics. ## Error model Graph construction can fail with `MissingDependencyError` or `CyclicDependencyError`. Lifecycle lookup uses `ServiceNotFoundError`; spawn preparation uses `SpawnError`; readiness uses -`ServiceReadyError`. Global shutdown timeout is logged and force-cleared, so the exported -`ShutdownTimeoutError` does not represent a current failure path. +`ServiceReadyError`. Global shutdown timeout is logged and force-cleared. ## Testing through Interfaces diff --git a/packages/process-compose/src/LogBuffer.ts b/packages/process-compose/src/LogBuffer.ts index 53a247098c..b561d7e6fe 100644 --- a/packages/process-compose/src/LogBuffer.ts +++ b/packages/process-compose/src/LogBuffer.ts @@ -24,7 +24,6 @@ export class LogBuffer extends Context.Service< limit?: number, services?: ReadonlyArray, ) => Effect.Effect>; - readonly truncate: (service: string) => Effect.Effect; } >()("process-compose/LogBuffer") { static layer = Layer.effect( @@ -97,15 +96,6 @@ export class LogBuffer extends Context.Service< return filtered.slice(-limit); }), - - truncate: (service) => - Effect.gen(function* () { - const { buffer } = yield* getOrCreate(service); - yield* Ref.set(buffer, []); - yield* Ref.update(globalBuffer, (entries) => - entries.filter((entry) => entry.service !== service), - ); - }), }; }), ); diff --git a/packages/process-compose/src/LogBuffer.unit.test.ts b/packages/process-compose/src/LogBuffer.unit.test.ts index b4e26774a8..c758785707 100644 --- a/packages/process-compose/src/LogBuffer.unit.test.ts +++ b/packages/process-compose/src/LogBuffer.unit.test.ts @@ -69,17 +69,6 @@ describe("LogBuffer", () => { }).pipe(Effect.provide(layer)); }); - it.live("truncate clears buffer", () => - Effect.gen(function* () { - const log = yield* LogBuffer; - yield* log.append("svc", "stdout", "line1"); - yield* log.append("svc", "stdout", "line2"); - yield* log.truncate("svc"); - const entries = yield* log.history("svc"); - expect(entries).toHaveLength(0); - }).pipe(Effect.provide(layer)), - ); - it.live("subscribeAll receives entries from all services", () => Effect.gen(function* () { const log = yield* LogBuffer; diff --git a/packages/process-compose/src/Orchestrator.ts b/packages/process-compose/src/Orchestrator.ts index dd1014cbd7..2a5f845e11 100644 --- a/packages/process-compose/src/Orchestrator.ts +++ b/packages/process-compose/src/Orchestrator.ts @@ -17,6 +17,7 @@ import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process"; import { buildGraph, type ResolvedGraph } from "./DependencyGraph.ts"; import { type HealthProbeCallbacks, runHealthProbe } from "./HealthProbe.ts"; import { LogBuffer } from "./LogBuffer.ts"; +import { restartClosureFor } from "./RestartClosure.ts"; import { decideRestart, type LifecycleCause, @@ -55,6 +56,8 @@ const willRestartAfterExit = (def: ServiceDef, state: ServiceState): boolean => ); }; +// Some one-shot adapters report `isRunning: false` before their exit-code Effect is observable. +// Keep the compensating poll isolated here so the ordinary process-exit path remains event-driven. const waitForProcessToStop = (handle: { readonly isRunning: Effect.Effect; }): Effect.Effect => @@ -316,10 +319,10 @@ export class Orchestrator extends Context.Service< // Release external resources such as port reservations only // once dependencies are satisfied and spawning is imminent. - // For supervised Docker services, this still leaves a wider - // spawn-to-bind window because the supervisor starts before - // the container binds its published ports. Closing that gap - // would require an explicit supervisor/container handshake. + // For supervised external processes, this can still leave a wider + // spawn-to-bind window because the supervisor starts before its + // child binds published ports. Closing that gap would require an + // explicit supervisor/child handshake. yield* options?.beforeSpawn?.(def.name) ?? Effect.void; // Spawn the process @@ -632,29 +635,30 @@ export class Orchestrator extends Context.Service< yield* sendEvent(name, { _tag: "ProcessExited", exitCode: 143 }); }); - const restartClosureFor = (name: string): ReadonlyArray => { - const names = new Set([name]); - const visited = new Set(); - const collectDependents = (current: string): boolean => { - if (visited.has(current)) return names.has(current); - visited.add(current); - let hasActiveDependent = false; - for (const dependent of graph.dependentsOf(current)) { - const dependentService = services.get(dependent.name); - const dependentIsActive = - FiberMap.hasUnsafe(fibers, dependent.name) || - (dependentService !== undefined && - SubscriptionRef.getUnsafe(dependentService.state).desired === "running"); - const descendantIsActive = collectDependents(dependent.name); - if (dependentIsActive || descendantIsActive) { - names.add(dependent.name); - hasActiveDependent = true; - } - } - return hasActiveDependent; - }; - collectDependents(name); - return graph.startOrder.filter((def) => names.has(def.name)); + const restartClosure = (name: string): ReadonlyArray => { + const activeServices = new Set( + graph.startOrder + .filter((def) => { + const service = services.get(def.name); + return ( + FiberMap.hasUnsafe(fibers, def.name) || + (service !== undefined && + SubscriptionRef.getUnsafe(service.state).desired === "running") + ); + }) + .map((def) => def.name), + ); + const closure = new Set( + restartClosureFor( + { + order: graph.startOrder.map((def) => def.name), + dependentsOf: (service) => graph.dependentsOf(service).map((def) => def.name), + }, + name, + activeServices, + ), + ); + return graph.startOrder.filter((def) => closure.has(def.name)); }; const waitReadySingle = (def: ServiceDef): Effect.Effect => @@ -861,7 +865,7 @@ export class Orchestrator extends Context.Service< if (lookupDef(name) === undefined) { return yield* Effect.fail(new ServiceNotFoundError({ name })); } - const affected = restartClosureFor(name); + const affected = restartClosure(name); for (const affectedDef of [...affected].reverse()) { yield* setDesired(affectedDef.name, "stopped"); yield* sendEvent(affectedDef.name, { _tag: "StopRequested" }); @@ -876,7 +880,7 @@ export class Orchestrator extends Context.Service< if (def === undefined) { return yield* Effect.fail(new ServiceNotFoundError({ name })); } - const affected = restartClosureFor(name); + const affected = restartClosure(name); for (const affectedDef of [...affected].reverse()) { yield* stopForRestart(affectedDef.name); diff --git a/packages/process-compose/src/Orchestrator.unit.test.ts b/packages/process-compose/src/Orchestrator.unit.test.ts index 954ce355b6..520375e86a 100644 --- a/packages/process-compose/src/Orchestrator.unit.test.ts +++ b/packages/process-compose/src/Orchestrator.unit.test.ts @@ -48,7 +48,6 @@ function mockLogBuffer() { line: entry.line, })); }), - truncate: () => Effect.void, }), get entries() { return entries; @@ -455,11 +454,17 @@ describe("Orchestrator", () => { it.live("supervised services spawn the supervisor runtime", () => { const { layer, proc } = setupOrchestrator([ - svc("postgres", { - command: "docker", - args: ["run", "--rm", "postgres"], + svc("database", { + command: "container-runtime", + args: ["run", "database"], supervision: { - orphanCleanup: [{ _tag: "DockerRemove", containerName: "supabase-postgres-test" }], + orphanCleanup: [ + { + _tag: "RunCommand", + executable: "container-runtime", + args: ["remove", "database-test"], + }, + ], }, }), ]); @@ -476,11 +481,17 @@ describe("Orchestrator", () => { it.live("stopping a supervisor during its spawn handshake cleans it up", () => { const { layer, proc } = setupOrchestrator( [ - svc("postgres", { - command: "docker", - args: ["run", "--rm", "postgres"], + svc("database", { + command: "container-runtime", + args: ["run", "database"], supervision: { - orphanCleanup: [{ _tag: "DockerRemove", containerName: "supabase-postgres-test" }], + orphanCleanup: [ + { + _tag: "RunCommand", + executable: "container-runtime", + args: ["remove", "database-test"], + }, + ], }, }), ], @@ -488,10 +499,10 @@ describe("Orchestrator", () => { ); return Effect.gen(function* () { const orc = yield* Orchestrator; - yield* orc.startService("postgres", { beforeSpawn: () => Effect.void }); + yield* orc.startService("database", { beforeSpawn: () => Effect.void }); yield* proc.waitForSpawnCount(1); - yield* orc.stopService("postgres"); + yield* orc.stopService("database"); yield* proc.waitForKillCount(1); expect(proc.killed[0]?.command).toBe(process.execPath); diff --git a/packages/process-compose/src/RestartClosure.ts b/packages/process-compose/src/RestartClosure.ts new file mode 100644 index 0000000000..2871e38fed --- /dev/null +++ b/packages/process-compose/src/RestartClosure.ts @@ -0,0 +1,37 @@ +export interface RestartClosureGraph { + readonly order: ReadonlyArray; + readonly dependentsOf: (name: string) => ReadonlyArray; +} + +/** + * Returns the requested service and every connector on a path to an active dependent. + * The result preserves dependency start order so callers can stop it in reverse safely. + */ +export const restartClosureFor = ( + graph: RestartClosureGraph, + name: string, + activeServices: ReadonlySet, +): ReadonlyArray => { + const closure = new Set([name]); + const visited = new Set(); + + const collectDependents = (current: string): boolean => { + if (visited.has(current)) { + return closure.has(current); + } + visited.add(current); + + let connectsToActiveDependent = false; + for (const dependent of graph.dependentsOf(current)) { + const descendantConnectsToActive = collectDependents(dependent); + if (activeServices.has(dependent) || descendantConnectsToActive) { + closure.add(dependent); + connectsToActiveDependent = true; + } + } + return connectsToActiveDependent; + }; + + collectDependents(name); + return graph.order.filter((service) => closure.has(service)); +}; diff --git a/packages/process-compose/src/RestartClosure.unit.test.ts b/packages/process-compose/src/RestartClosure.unit.test.ts new file mode 100644 index 0000000000..306d926c9b --- /dev/null +++ b/packages/process-compose/src/RestartClosure.unit.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from "vitest"; +import { restartClosureFor, type RestartClosureGraph } from "./RestartClosure.ts"; + +const graph = ( + order: ReadonlyArray, + dependents: Readonly>>, +): RestartClosureGraph => ({ + order, + dependentsOf: (name) => dependents[name] ?? [], +}); + +describe("restartClosureFor", () => { + const cases: ReadonlyArray<{ + readonly name: string; + readonly graph: RestartClosureGraph; + readonly root: string; + readonly active: ReadonlyArray; + readonly expected: ReadonlyArray; + }> = [ + { + name: "keeps only the requested service when no dependent is active", + graph: graph(["db", "api", "web"], { db: ["api"], api: ["web"] }), + root: "db", + active: [], + expected: ["db"], + }, + { + name: "includes active direct dependents in dependency order", + graph: graph(["db", "api", "worker"], { db: ["api", "worker"] }), + root: "db", + active: ["worker", "api"], + expected: ["db", "api", "worker"], + }, + { + name: "preserves inactive connectors leading to an active descendant", + graph: graph(["db", "gateway", "api", "web"], { + db: ["gateway"], + gateway: ["api"], + api: ["web"], + }), + root: "db", + active: ["web"], + expected: ["db", "gateway", "api", "web"], + }, + { + name: "continues through active dependents to include active descendants", + graph: graph(["db", "api", "web"], { db: ["api"], api: ["web"] }), + root: "db", + active: ["api", "web"], + expected: ["db", "api", "web"], + }, + { + name: "handles shared connectors without duplicating diamond nodes", + graph: graph(["db", "api", "worker", "web"], { + db: ["api", "worker"], + api: ["web"], + worker: ["web"], + }), + root: "db", + active: ["web"], + expected: ["db", "api", "worker", "web"], + }, + ]; + + for (const testCase of cases) { + it(testCase.name, () => { + expect(restartClosureFor(testCase.graph, testCase.root, new Set(testCase.active))).toEqual( + testCase.expected, + ); + }); + } +}); diff --git a/packages/process-compose/src/ServiceDef.ts b/packages/process-compose/src/ServiceDef.ts index d015fcdae1..6cdc03150b 100644 --- a/packages/process-compose/src/ServiceDef.ts +++ b/packages/process-compose/src/ServiceDef.ts @@ -55,8 +55,10 @@ export interface LifecycleHook { export type ExternalCleanupAction = | { - readonly _tag: "DockerRemove"; - readonly containerName: string; + readonly _tag: "RunCommand"; + readonly executable: string; + readonly args: ReadonlyArray; + readonly timeoutMs?: number; } | { readonly _tag: "RemovePath"; diff --git a/packages/process-compose/src/ServiceTransition.ts b/packages/process-compose/src/ServiceTransition.ts index 94c28528ed..ea9212a754 100644 --- a/packages/process-compose/src/ServiceTransition.ts +++ b/packages/process-compose/src/ServiceTransition.ts @@ -28,52 +28,43 @@ export type ServiceEvent = | { readonly _tag: "HookFailed"; readonly error: string }; // --------------------------------------------------------------------------- -// Transition table — set of (fromStatus, eventTag) pairs that are legal +// Transition table — every event must classify its legal source statuses // --------------------------------------------------------------------------- -const allowed = new Set<`${ServiceStatus}:${ServiceEvent["_tag"]}`>([ - "Pending:DependenciesSatisfied", - "Pending:DependencyFailed", - "Pending:SpawnFailed", - "Pending:StopRequested", - "Starting:ProcessSpawned", - "Starting:SpawnFailed", - "Starting:StopRequested", - "Starting:HookFailed", - "Running:HealthCheckPassed", - "Running:HealthCheckFailed", - "Running:ProcessExited", - "Running:StopRequested", - "Healthy:HealthCheckPassed", - "Healthy:HealthCheckFailed", - "Healthy:ProcessExited", - "Healthy:StopRequested", - "Unhealthy:HealthCheckPassed", - "Unhealthy:ProcessExited", - "Unhealthy:ProcessTerminated", - "Unhealthy:StopRequested", - "Stopping:ProcessExited", - "Stopped:RestartTriggered", - "Failed:RestartTriggered", - "Failed:ProcessExited", - "Failed:StopRequested", - "Unhealthy:RestartTriggered", - "Unhealthy:UnhealthyRestartExhausted", - "Restarting:StopRequested", - "Restarting:SpawnFailed", - "Restarting:BackoffElapsed", - "Running:HookFailed", - "Healthy:HookFailed", - "Unhealthy:HookFailed", -]); +type TransitionTable = { + readonly [Tag in ServiceEvent["_tag"]]: ReadonlySet; +}; + +const transitions: TransitionTable = { + DependenciesSatisfied: new Set(["Pending"]), + DependencyFailed: new Set(["Pending"]), + SpawnFailed: new Set(["Pending", "Starting", "Restarting"]), + ProcessSpawned: new Set(["Starting"]), + HealthCheckPassed: new Set(["Running", "Healthy", "Unhealthy"]), + HealthCheckFailed: new Set(["Running", "Healthy"]), + ProcessTerminated: new Set(["Unhealthy"]), + UnhealthyRestartExhausted: new Set(["Unhealthy"]), + ProcessExited: new Set(["Running", "Healthy", "Unhealthy", "Stopping", "Failed"]), + StopRequested: new Set([ + "Pending", + "Starting", + "Running", + "Healthy", + "Unhealthy", + "Restarting", + "Failed", + ]), + RestartTriggered: new Set(["Stopped", "Failed", "Unhealthy"]), + BackoffElapsed: new Set(["Restarting"]), + HookFailed: new Set(["Starting", "Running", "Healthy", "Unhealthy"]), +}; // --------------------------------------------------------------------------- // applyEvent — pure function, returns new ServiceState or null if invalid // --------------------------------------------------------------------------- export const applyEvent = (state: ServiceState, event: ServiceEvent): ServiceState | null => { - const key = `${state.status}:${event._tag}` as const; - if (!allowed.has(key)) return null; + if (!transitions[event._tag].has(state.status)) return null; switch (event._tag) { case "DependenciesSatisfied": diff --git a/packages/process-compose/src/ServiceTransition.unit.test.ts b/packages/process-compose/src/ServiceTransition.unit.test.ts index 58b41524f6..c34b260ce2 100644 --- a/packages/process-compose/src/ServiceTransition.unit.test.ts +++ b/packages/process-compose/src/ServiceTransition.unit.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; -import { applyEvent } from "./ServiceTransition.ts"; -import { ServiceState, initial } from "./ServiceState.ts"; +import { applyEvent, type ServiceEvent } from "./ServiceTransition.ts"; +import { ServiceState, initial, type ServiceStatus } from "./ServiceState.ts"; const make = ( name: string, @@ -19,6 +19,73 @@ const make = ( }); describe("ServiceTransition", () => { + it("classifies every event against every service status", () => { + const statuses: ReadonlyArray = [ + "Pending", + "Starting", + "Running", + "Healthy", + "Unhealthy", + "Stopping", + "Stopped", + "Failed", + "Restarting", + ]; + const events: { readonly [Tag in ServiceEvent["_tag"]]: Extract } = + { + DependenciesSatisfied: { _tag: "DependenciesSatisfied" }, + DependencyFailed: { _tag: "DependencyFailed", error: "dependency failed" }, + SpawnFailed: { _tag: "SpawnFailed", error: "spawn failed" }, + ProcessSpawned: { _tag: "ProcessSpawned", pid: 1234, startedAt: 1000 }, + HealthCheckPassed: { _tag: "HealthCheckPassed" }, + HealthCheckFailed: { _tag: "HealthCheckFailed" }, + ProcessTerminated: { _tag: "ProcessTerminated" }, + UnhealthyRestartExhausted: { + _tag: "UnhealthyRestartExhausted", + error: "restart exhausted", + }, + ProcessExited: { _tag: "ProcessExited", exitCode: 1 }, + StopRequested: { _tag: "StopRequested" }, + RestartTriggered: { _tag: "RestartTriggered", restartCount: 1 }, + BackoffElapsed: { _tag: "BackoffElapsed" }, + HookFailed: { _tag: "HookFailed", error: "hook failed" }, + }; + const legalStatuses: { + readonly [Tag in ServiceEvent["_tag"]]: ReadonlyArray; + } = { + DependenciesSatisfied: ["Pending"], + DependencyFailed: ["Pending"], + SpawnFailed: ["Pending", "Starting", "Restarting"], + ProcessSpawned: ["Starting"], + HealthCheckPassed: ["Running", "Healthy", "Unhealthy"], + HealthCheckFailed: ["Running", "Healthy"], + ProcessTerminated: ["Unhealthy"], + UnhealthyRestartExhausted: ["Unhealthy"], + ProcessExited: ["Running", "Healthy", "Unhealthy", "Stopping", "Failed"], + StopRequested: [ + "Pending", + "Starting", + "Running", + "Healthy", + "Unhealthy", + "Failed", + "Restarting", + ], + RestartTriggered: ["Unhealthy", "Stopped", "Failed"], + BackoffElapsed: ["Restarting"], + HookFailed: ["Starting", "Running", "Healthy", "Unhealthy"], + }; + + for (const event of Object.values(events)) { + for (const status of statuses) { + const state = make("service", { status, pid: 1234 }); + expect(applyEvent(state, event) !== null, `${status} + ${event._tag}`).toBe( + legalStatuses[event._tag].includes(status), + ); + } + } + }); + describe("valid transitions", () => { it("Pending + DependenciesSatisfied → Starting", () => { const state = make("db"); diff --git a/packages/process-compose/src/SupervisorRuntime.unit.test.ts b/packages/process-compose/src/SupervisorRuntime.unit.test.ts index 77b7b61b5d..4cdd6321eb 100644 --- a/packages/process-compose/src/SupervisorRuntime.unit.test.ts +++ b/packages/process-compose/src/SupervisorRuntime.unit.test.ts @@ -2,10 +2,38 @@ import { spawn } from "node:child_process"; import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import path from "node:path"; -import { fileURLToPath } from "node:url"; -import { describe, test } from "vitest"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { describe, expect, test } from "vitest"; +import { makeSupervisorRuntimeEnv, withoutSupervisorRuntimeEnv } from "./supervisor-protocol.ts"; const supervisorRuntimePath = fileURLToPath(new URL("./supervisor-runtime.ts", import.meta.url)); +const supervisorProtocolPath = fileURLToPath(new URL("./supervisor-protocol.ts", import.meta.url)); + +type SupervisorEntry = "source path" | "compiled self-dispatch"; + +const spawnSupervisor = (entry: SupervisorEntry, encodedConfig: string) => { + if (entry === "source path") { + return spawn(process.execPath, [supervisorRuntimePath, encodedConfig], { + stdio: ["pipe", "ignore", "ignore"], + }); + } + + const runtimeUrl = pathToFileURL(supervisorRuntimePath).href; + const protocolUrl = pathToFileURL(supervisorProtocolPath).href; + const dispatch = [ + `import { runSupervisorRuntimeFromEnv } from ${JSON.stringify(runtimeUrl)};`, + `import { isSupervisorRuntimeRequested } from ${JSON.stringify(protocolUrl)};`, + `if (!isSupervisorRuntimeRequested()) throw new Error("supervisor dispatch not requested");`, + `runSupervisorRuntimeFromEnv();`, + ].join("\n"); + return spawn(process.execPath, ["--eval", dispatch], { + env: makeSupervisorRuntimeEnv(encodedConfig, { + ...process.env, + PROCESS_COMPOSE_SUPERVISOR_SELF_DISPATCH: "1", + }), + stdio: ["pipe", "ignore", "ignore"], + }); +}; const waitFor = async ( predicate: () => boolean, @@ -39,12 +67,14 @@ const isPidAlive = (pid: number): boolean => { }; describe("supervisor-runtime", () => { - test( - "kills the child tree and runs orphan cleanup when parent stdin closes", + test.each(["source path", "compiled self-dispatch"])( + "%s kills the child tree and runs validated orphan cleanup when parent stdin closes", { timeout: 15_000 }, - async () => { + async (entry) => { const tempDir = mkdtempSync(path.join(tmpdir(), "process-compose-supervisor-")); const cleanupDir = path.join(tempDir, "cleanup-dir"); + const cleanupMarker = path.join(tempDir, "cleanup-command-ran"); + const cleanupEnvironmentMarker = path.join(tempDir, "cleanup-environment.json"); const childPidFile = path.join(tempDir, "child.pid"); const grandchildPidFile = path.join(tempDir, "grandchild.pid"); const readyFile = path.join(tempDir, "ready"); @@ -72,13 +102,32 @@ describe("supervisor-runtime", () => { args: [childScriptPath], shutdownSignal: "SIGTERM", shutdownTimeoutMs: 100, - cleanup: [{ _tag: "RemovePath", path: cleanupDir, recursive: true }], + cleanup: [ + { _tag: "RemovePath", path: cleanupDir, recursive: true }, + { + _tag: "RunCommand", + executable: process.execPath, + args: [ + "-e", + [ + `const { writeFileSync } = require("node:fs");`, + `writeFileSync(process.argv[1], process.argv[2]);`, + `writeFileSync(process.argv[3], JSON.stringify({`, + ` run: process.env.PROCESS_COMPOSE_RUN_SUPERVISOR,`, + ` config: process.env.PROCESS_COMPOSE_SUPERVISOR_CONFIG,`, + ` dispatch: process.env.PROCESS_COMPOSE_SUPERVISOR_SELF_DISPATCH,`, + `}));`, + ].join("\n"), + cleanupMarker, + "literal; $(not-run) & value", + cleanupEnvironmentMarker, + ], + }, + ], }), ).toString("base64url"); - const supervisor = spawn(process.execPath, [supervisorRuntimePath, encodedConfig], { - stdio: ["pipe", "ignore", "ignore"], - }); + const supervisor = spawnSupervisor(entry, encodedConfig); try { await waitFor(() => existsSync(readyFile)); @@ -90,6 +139,9 @@ describe("supervisor-runtime", () => { await waitFor(() => supervisor.exitCode != null, { timeoutMs: 10_000 }); await waitFor(() => !existsSync(cleanupDir), { timeoutMs: 10_000 }); + await waitFor(() => existsSync(cleanupMarker), { timeoutMs: 10_000 }); + expect(readFileSync(cleanupMarker, "utf8")).toBe("literal; $(not-run) & value"); + expect(JSON.parse(readFileSync(cleanupEnvironmentMarker, "utf8"))).toEqual({}); await waitFor(() => !isPidAlive(childPid), { timeoutMs: 10_000 }); await waitFor(() => !isPidAlive(grandchildPid), { timeoutMs: 10_000 }); } finally { @@ -99,6 +151,142 @@ describe("supervisor-runtime", () => { }, ); + test.each([ + [ + "non-string command argument", + { _tag: "RunCommand", executable: process.execPath, args: [42] }, + ], + ["empty executable", { _tag: "RunCommand", executable: "", args: [] }], + [ + "non-positive timeout", + { _tag: "RunCommand", executable: process.execPath, args: [], timeoutMs: 0 }, + ], + ["invalid path option", { _tag: "RemovePath", path: "/tmp/example", recursive: "yes" }], + ])("rejects a malformed cleanup contract with %s before spawning", async (_name, cleanup) => { + const tempDir = mkdtempSync(path.join(tmpdir(), "process-compose-supervisor-invalid-")); + const childMarker = path.join(tempDir, "child-started"); + const encodedConfig = Buffer.from( + JSON.stringify({ + command: process.execPath, + args: ["-e", `require("node:fs").writeFileSync(${JSON.stringify(childMarker)}, "started")`], + cleanup: [cleanup], + }), + ).toString("base64url"); + const supervisor = spawnSupervisor("source path", encodedConfig); + + try { + await waitFor(() => supervisor.exitCode != null); + expect(supervisor.exitCode).not.toBe(0); + expect(existsSync(childMarker)).toBe(false); + } finally { + supervisor.kill("SIGKILL"); + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + test("removes supervisor protocol variables from the managed child environment", () => { + const childEnv = withoutSupervisorRuntimeEnv({ + KEEP_ME: "value", + PROCESS_COMPOSE_SUPERVISOR_SELF_DISPATCH: "1", + PROCESS_COMPOSE_RUN_SUPERVISOR: "1", + PROCESS_COMPOSE_SUPERVISOR_CONFIG: "encoded", + }); + + expect(childEnv).toEqual({ KEEP_ME: "value" }); + }); + + test("bounds a cleanup command tree by its timeout and continues remaining cleanup", async () => { + const tempDir = mkdtempSync(path.join(tmpdir(), "process-compose-supervisor-timeout-")); + const cleanupDir = path.join(tempDir, "cleanup-dir"); + const cleanupWorkerPidFile = path.join(tempDir, "cleanup-worker.pid"); + const childScriptPath = path.join(tempDir, "child.mjs"); + mkdirSync(cleanupDir); + writeFileSync(childScriptPath, "process.exit(0);\n"); + const encodedConfig = Buffer.from( + JSON.stringify({ + command: process.execPath, + args: [childScriptPath], + cleanup: [ + { + _tag: "RunCommand", + executable: process.execPath, + args: [ + "-e", + [ + `const { spawn } = require("node:child_process");`, + `const { writeFileSync } = require("node:fs");`, + `const worker = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" });`, + `writeFileSync(${JSON.stringify(cleanupWorkerPidFile)}, String(worker.pid));`, + `setInterval(() => {}, 1000);`, + ].join("\n"), + ], + timeoutMs: 100, + }, + { _tag: "RemovePath", path: cleanupDir, recursive: true }, + ], + }), + ).toString("base64url"); + const supervisor = spawnSupervisor("source path", encodedConfig); + + try { + await waitFor(() => supervisor.exitCode != null); + expect(supervisor.exitCode).toBe(0); + expect(existsSync(cleanupDir)).toBe(false); + const cleanupWorkerPid = Number.parseInt(readFileSync(cleanupWorkerPidFile, "utf8"), 10); + expect(Number.isSafeInteger(cleanupWorkerPid)).toBe(true); + await waitFor(() => !isPidAlive(cleanupWorkerPid)); + } finally { + supervisor.kill("SIGKILL"); + if (existsSync(cleanupWorkerPidFile)) { + try { + process.kill(Number.parseInt(readFileSync(cleanupWorkerPidFile, "utf8"), 10), "SIGKILL"); + } catch {} + } + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + test("bounds a cleanup command when no timeout is configured", { timeout: 12_000 }, async () => { + const tempDir = mkdtempSync(path.join(tmpdir(), "process-compose-supervisor-timeout-")); + const cleanupDir = path.join(tempDir, "cleanup-dir"); + const cleanupPidFile = path.join(tempDir, "cleanup.pid"); + const childScriptPath = path.join(tempDir, "child.mjs"); + mkdirSync(cleanupDir); + writeFileSync(childScriptPath, "process.exit(0);\n"); + const encodedConfig = Buffer.from( + JSON.stringify({ + command: process.execPath, + args: [childScriptPath], + cleanup: [ + { + _tag: "RunCommand", + executable: process.execPath, + args: [ + "-e", + `require("node:fs").writeFileSync(${JSON.stringify(cleanupPidFile)}, String(process.pid)); setInterval(() => {}, 1000)`, + ], + }, + { _tag: "RemovePath", path: cleanupDir, recursive: true }, + ], + }), + ).toString("base64url"); + const supervisor = spawnSupervisor("source path", encodedConfig); + + try { + await waitFor(() => supervisor.exitCode != null, { timeoutMs: 8_000 }); + expect(supervisor.exitCode).toBe(0); + expect(existsSync(cleanupDir)).toBe(false); + } finally { + supervisor.kill("SIGKILL"); + if (existsSync(cleanupPidFile)) { + try { + process.kill(Number.parseInt(readFileSync(cleanupPidFile, "utf8"), 10), "SIGKILL"); + } catch {} + } + rmSync(tempDir, { recursive: true, force: true }); + } + }); + test( "runs orphan cleanup when the configured owner pid is already gone", { timeout: 15_000 }, diff --git a/packages/process-compose/src/errors.ts b/packages/process-compose/src/errors.ts index 44bde7ff2b..65dc4ea66f 100644 --- a/packages/process-compose/src/errors.ts +++ b/packages/process-compose/src/errors.ts @@ -18,10 +18,6 @@ export class SpawnError extends Data.TaggedError("SpawnError")<{ readonly cause: unknown; }> {} -export class ShutdownTimeoutError extends Data.TaggedError("ShutdownTimeoutError")<{ - readonly service: string; -}> {} - export class ServiceReadyError extends Data.TaggedError("ServiceReadyError")<{ readonly name: string; readonly reason: string; diff --git a/packages/process-compose/src/errors.unit.test.ts b/packages/process-compose/src/errors.unit.test.ts index b2c2920595..073e5279f0 100644 --- a/packages/process-compose/src/errors.unit.test.ts +++ b/packages/process-compose/src/errors.unit.test.ts @@ -4,7 +4,6 @@ import { MissingDependencyError, ServiceNotFoundError, SpawnError, - ShutdownTimeoutError, } from "./errors.ts"; describe("errors", () => { @@ -34,10 +33,4 @@ describe("errors", () => { expect(err.service).toBe("postgres"); expect(err.cause).toBe(cause); }); - - it("ShutdownTimeoutError has correct tag and data", () => { - const err = new ShutdownTimeoutError({ service: "postgres" }); - expect(err._tag).toBe("ShutdownTimeoutError"); - expect(err.service).toBe("postgres"); - }); }); diff --git a/packages/process-compose/src/index.ts b/packages/process-compose/src/index.ts index 609f5db891..421763613f 100644 --- a/packages/process-compose/src/index.ts +++ b/packages/process-compose/src/index.ts @@ -25,7 +25,6 @@ export { ServiceNotFoundError, ServiceReadyError, SpawnError, - ShutdownTimeoutError, } from "./errors.ts"; export type { LogEntry } from "./LogBuffer.ts"; diff --git a/packages/process-compose/src/supervisor-runtime.ts b/packages/process-compose/src/supervisor-runtime.ts index 302961ffdf..f3114183f5 100644 --- a/packages/process-compose/src/supervisor-runtime.ts +++ b/packages/process-compose/src/supervisor-runtime.ts @@ -9,6 +9,7 @@ import { } from "./supervisor-protocol.ts"; type RemovePathAction = Extract; +type RunCommandAction = Extract; interface SupervisorRuntimeConfig { readonly command: string; @@ -24,6 +25,8 @@ interface ChildExit { readonly signal: NodeJS.Signals | null; } +const DEFAULT_CLEANUP_COMMAND_TIMEOUT_MS = 5_000; + const isMain = (() => { if (process.argv[1] == null) { return false; @@ -104,16 +107,35 @@ const cleanupActionFrom = (value: unknown): ExternalCleanupAction | undefined => } const tag = getField(value, "_tag"); - if (tag === "DockerRemove") { - const containerName = getField(value, "containerName"); - return typeof containerName === "string" ? { _tag: tag, containerName } : undefined; + if (tag === "RunCommand") { + const executable = getField(value, "executable"); + const args = stringArrayFrom(getField(value, "args")); + const timeoutMs = getField(value, "timeoutMs"); + if ( + typeof executable !== "string" || + executable.length === 0 || + args === undefined || + (timeoutMs !== undefined && + (typeof timeoutMs !== "number" || !Number.isFinite(timeoutMs) || timeoutMs <= 0)) + ) { + return undefined; + } + return { + _tag: tag, + executable, + args, + timeoutMs: typeof timeoutMs === "number" ? timeoutMs : undefined, + }; } if (tag === "RemovePath") { const path = getField(value, "path"); const recursive = getField(value, "recursive"); const force = getField(value, "force"); - return typeof path === "string" + return typeof path === "string" && + path.length > 0 && + (recursive === undefined || typeof recursive === "boolean") && + (force === undefined || typeof force === "boolean") ? { _tag: tag, path, @@ -148,6 +170,11 @@ const parseSupervisorRuntimeConfig = (encodedConfig: string): SupervisorRuntimeC const ownerPid = getField(value, "ownerPid"); const shutdownTimeoutMs = getField(value, "shutdownTimeoutMs"); + const cleanupValue = getField(value, "cleanup"); + const cleanup = cleanupValue === undefined ? undefined : cleanupActionsFrom(cleanupValue); + if (cleanupValue !== undefined && cleanup === undefined) { + throw new Error("Invalid supervisor cleanup"); + } return { command, @@ -155,7 +182,7 @@ const parseSupervisorRuntimeConfig = (encodedConfig: string): SupervisorRuntimeC ownerPid: typeof ownerPid === "number" ? ownerPid : undefined, shutdownSignal: signalFrom(getField(value, "shutdownSignal")), shutdownTimeoutMs: typeof shutdownTimeoutMs === "number" ? shutdownTimeoutMs : undefined, - cleanup: cleanupActionsFrom(getField(value, "cleanup")), + cleanup, }; }; @@ -207,14 +234,10 @@ export function runSupervisorRuntime(encodedConfig = process.argv[2]): void { } }; - const killChildTree = (signal: ChildProcess.Signal): void => { - if (child.pid == null) { - return; - } - + const killProcessTree = (pid: number, signal: ChildProcess.Signal): void => { if (isWindows) { try { - execFileSync("taskkill", ["/PID", String(child.pid), "/T", "/F"], { + execFileSync("taskkill", ["/PID", String(pid), "/T", "/F"], { stdio: "ignore", timeout: 5_000, }); @@ -224,16 +247,48 @@ export function runSupervisorRuntime(encodedConfig = process.argv[2]): void { } try { - process.kill(-child.pid, signal); + process.kill(-pid, signal); return; } catch {} try { - process.kill(child.pid, signal); + process.kill(pid, signal); } catch {} }; - const runCleanup = () => { + const killChildTree = (signal: ChildProcess.Signal): void => { + if (child.pid != null) { + killProcessTree(child.pid, signal); + } + }; + + const runCleanupCommand = (action: RunCommandAction): Promise => + new Promise((resolve) => { + const cleanupChild = spawn(action.executable, action.args, { + detached: !isWindows, + env: childEnv, + stdio: "ignore", + }); + let timeoutId: ReturnType | undefined; + + const finish = () => { + if (timeoutId != null) { + clearTimeout(timeoutId); + } + resolve(); + }; + + cleanupChild.once("error", finish); + cleanupChild.once("exit", finish); + timeoutId = setTimeout(() => { + if (cleanupChild.pid != null) { + killProcessTree(cleanupChild.pid, "SIGKILL"); + } + finish(); + }, action.timeoutMs ?? DEFAULT_CLEANUP_COMMAND_TIMEOUT_MS); + }); + + const runCleanup = async (): Promise => { const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); const removePathWithRetry = async (action: RemovePathAction): Promise => { for (let attempt = 0; attempt < 20; attempt++) { @@ -249,20 +304,26 @@ export function runSupervisorRuntime(encodedConfig = process.argv[2]): void { } }; - return Promise.all( - (config.cleanup ?? []).map(async (action) => { + const runCommands = async () => { + for (const action of config.cleanup ?? []) { + if (action._tag !== "RunCommand") { + continue; + } + try { - if (action._tag === "DockerRemove") { - execFileSync("docker", ["rm", "-f", action.containerName], { - stdio: "ignore", - timeout: 5_000, - }); - } else if (action._tag === "RemovePath") { - await removePathWithRetry(action); - } + await runCleanupCommand(action); } catch {} - }), - ).then(() => undefined); + } + }; + + // Commands serialize so their worst-case timeouts add together. Each runs in its own Unix + // process group (or uses taskkill on Windows), allowing a timeout to terminate its whole tree. + await Promise.all([ + runCommands(), + ...(config.cleanup ?? []).map((action) => + action._tag === "RemovePath" ? removePathWithRetry(action) : Promise.resolve(), + ), + ]); }; const shutdown = async (signal: ChildProcess.Signal): Promise => { diff --git a/packages/stack/README.md b/packages/stack/README.md index 85d2db88e1..50dd10f464 100644 --- a/packages/stack/README.md +++ b/packages/stack/README.md @@ -198,10 +198,11 @@ service and its projected status. ### Readiness ```typescript -await stack.ready(); // Wait for all services -await stack.ready({ timeout: 30_000 }); // With timeout (ms) -await stack.serviceReady("postgres"); // Wait for one service -await stack.serviceReady("auth", { timeout: 10_000 }); +await stack.ready(); // Inherit the stack's finite two-minute default +await stack.ready({ mode: "finite", timeoutMs: 30_000 }); +await stack.ready({ mode: "infinite" }); // Explicit debugging override +await stack.serviceReady("postgres"); +await stack.serviceReady("auth", { mode: "finite", timeoutMs: 10_000 }); ``` In eager mode, `start()` blocks until every enabled service is ready. In lazy mode it waits only @@ -276,8 +277,8 @@ await prefetch({ versions: { postgres: "17.4.1.045" } }); ## Service Versions Default versions are used when no per-service `version` field is specified. The authoritative, -exhaustive values are exported as `DEFAULT_VERSIONS` and live in -[`src/versions.ts`](./src/versions.ts); they are intentionally not copied into this README. +exhaustive values live in [`src/ServiceCatalog.ts`](./src/ServiceCatalog.ts) and are exported as +the derived `DEFAULT_VERSIONS` manifest; they are intentionally not copied into this README. Override versions per service: diff --git a/packages/stack/docs/architecture.md b/packages/stack/docs/architecture.md index 042f24cf7f..336964d1d2 100644 --- a/packages/stack/docs/architecture.md +++ b/packages/stack/docs/architecture.md @@ -20,7 +20,7 @@ socket HTTP implementations. `createStack.ts` remains platform-agnostic and rece ```mermaid flowchart LR - Input["StackConfig"] --> Resolve["resolveConfig"] + Input["StackConfig"] --> Resolve["StackConfigResolver"] Resolve --> Layer["foregroundLayer"] Layer --> Prepare["StackPreparation"] Prepare --> Builder["StackBuilder"] @@ -40,7 +40,7 @@ can use the same lifecycle calls against an in-process stack or a detached daemo top-level fields choose runtime mode, startup mode, cache/runtime roots, API keys, JWT secret, functions options, and per-service configuration. `false` disables an optional service. -`resolveConfig()`: +`StackConfigResolver.resolveConfig()`: 1. chooses cache, durable stack, runtime, and project roots; 2. allocates every required port through one port allocator; @@ -48,6 +48,12 @@ functions options, and per-service configuration. `false` disables an optional s 4. applies per-service defaults and current `DEFAULT_VERSIONS`; 5. records auto-managed paths for scoped cleanup. +Readiness policy is part of the resolved configuration. The package default is a finite two-minute +deadline; callers can choose a different finite deadline or explicit infinite waiting. Per-call +`ReadyOptions` take precedence over the stack policy, while `inherit` delegates to the stack +policy. The Promise `ready()` / `serviceReady()` Adapter applies that resolver today. The Effect +lifecycle Interface remains unbounded until deadline enforcement is centralized there. + The current zero-config stack enables PostgreSQL, PostgREST, Auth, and Edge Runtime. Realtime, Storage, imgproxy, Mailpit, Postgres Meta, Studio, Analytics, Vector, and Supavisor are enabled only when their corresponding configuration object is present. In `native` mode, Edge Runtime is also @@ -65,7 +71,8 @@ disabled by omission because the automatic artifact policy currently classifies Preparation is separate from topology construction: -- `ServiceArtifacts.ts` records native release providers and Docker image candidates. +- `ServiceCatalog.ts` is the exhaustive source for service identity, default version, runtime + support, artifact providers, activation policy, and allocated port fields. - `BinaryResolver` detects the platform, downloads and verifies archives, restores executable permissions, and publishes complete cache entries atomically. - `StackPreparation` resolves all enabled public services, emits download/pull progress, and @@ -76,7 +83,9 @@ Native cache identity includes service, provider, version, and asset name. `.com last, so an incomplete download is never treated as reusable. Supabase-owned Docker images are tried through ECR, Docker Hub, then GHCR; upstream images use their canonical repository. -`prefetch()` uses the same `StackPreparation` Interface without constructing a lifecycle runtime. +`ServiceResolution` belongs to this preparation domain. `prefetch()` uses the same +`StackPreparation` Interface and its binary-to-Docker fallback without constructing a lifecycle +runtime. ## Service coverage and topology @@ -131,12 +140,13 @@ process statuses. `start()` prepares artifacts, creates the runtime once, starts the appropriate services, and waits for their generic process-compose readiness. `stop()` preserves explicit per-service stop intent; -`dispose()` additionally closes the scoped runtime and executes cleanup. Current generic readiness -has no built-in deadline. +`dispose()` additionally closes the scoped runtime and executes cleanup. Generic process-compose +readiness remains intentionally policy-free and unbounded. ## Eager and lazy activation -`ServiceActivation.ts` is the declarative startup and companion-ownership policy: +`ServiceActivation.ts` evaluates the startup and companion-ownership metadata in +`ServiceCatalog.ts`: - eager: PostgreSQL, Realtime, Mailpit, Studio, and Pooler; - lazy: PostgREST, Auth, Edge Runtime, Storage, imgproxy, Postgres Meta, Analytics, and Vector; @@ -276,5 +286,6 @@ Callers may explicitly supply `projectStateRoot`, in which case durable stacks l - Targeted e2e tests own the expensive process/container Seam for full stack startup, parallel stacks, daemon lifecycle, and cleanup behavior. -The authoritative current service versions are `DEFAULT_VERSIONS` in `src/versions.ts`; package +The authoritative current service versions are the `defaultVersion` fields in +`src/ServiceCatalog.ts`; `DEFAULT_VERSIONS` is derived from that catalog, and package documentation should link to that source rather than copy its values. diff --git a/packages/stack/docs/service-versioning.md b/packages/stack/docs/service-versioning.md index d4c12efdfa..f8a3dd819c 100644 --- a/packages/stack/docs/service-versioning.md +++ b/packages/stack/docs/service-versioning.md @@ -38,10 +38,10 @@ The important separation is: ## Artifact Providers and Runtime Support -Every service in the Stack has one artifact definition, regardless of who maintains it. The -definition records its Docker image provider, tag convention, and whether a supported native -release is available. Runtime code consumes the resulting service resolution and does not need to -know which registry or release repository supplied it. +Every service in the Stack has one entry in `ServiceCatalog.ts`, regardless of who maintains it. +The entry records its default version, Docker image provider, tag convention, runtime support, and +any native release source. Runtime code consumes the resulting service resolution and does not +need to know which registry or release repository supplied it. Supabase-managed images currently retain their ECR, Docker Hub, and GHCR candidates. External services such as imgproxy, Mailpit, and Vector retain their upstream Docker images and are marked @@ -63,8 +63,8 @@ service definitions. The old Go CLI used `pkg/config/templates/Dockerfile` as a version manifest so Dependabot could bump image tags automatically. -The TypeScript stack exports a typed `DEFAULT_VERSIONS` manifest instead. That constant is the -built-in default version set for a given CLI release. +The TypeScript stack derives its typed `DEFAULT_VERSIONS` manifest from `ServiceCatalog.ts`. The +catalog values are the built-in default version set for a given CLI release. These defaults are the fallback for: diff --git a/packages/stack/scripts/sync-versions-from-dockerfile.ts b/packages/stack/scripts/sync-versions-from-dockerfile.ts index 39e4592ea2..a35e4db24d 100644 --- a/packages/stack/scripts/sync-versions-from-dockerfile.ts +++ b/packages/stack/scripts/sync-versions-from-dockerfile.ts @@ -11,7 +11,7 @@ import { const scriptDir = path.dirname(fileURLToPath(import.meta.url)); const repoRoot = path.resolve(scriptDir, "../../.."); const dockerfilePath = path.join(repoRoot, "apps/cli-go/pkg/config/templates/Dockerfile"); -const versionsPath = path.join(repoRoot, "packages/stack/src/versions.ts"); +const catalogPath = path.join(repoRoot, "packages/stack/src/ServiceCatalog.ts"); const fromLinePattern = /^FROM\s+(.+):([^:\s]+)\s+AS\s+([^\s#]+)/i; @@ -76,43 +76,40 @@ export function readVersionManifestFromDockerfile(dockerfile: string): VersionMa return versions; } -function renderManifestKey(service: ServiceName): string { - return /^[a-zA-Z_$][\w$]*$/.test(service) ? service : JSON.stringify(service); -} - -export function renderDefaultVersions(versions: VersionManifest): string { - const lines = SERVICE_NAMES.map( - (service) => ` ${renderManifestKey(service)}: ${JSON.stringify(versions[service])},`, - ); - return ["export const DEFAULT_VERSIONS: VersionManifest = {", ...lines, "} as const;"].join("\n"); -} - export function syncDefaultVersionsSource(source: string, versions: VersionManifest): string { - const startMarker = "export const DEFAULT_VERSIONS: VersionManifest = {"; - const endMarker = "\n} as const;"; - const start = source.indexOf(startMarker); - if (start === -1) { - throw new Error("Could not find DEFAULT_VERSIONS declaration."); - } + let updated = source; + for (const service of SERVICE_NAMES) { + const nameMarker = ` name: ${JSON.stringify(service)},`; + const entryStart = updated.indexOf(nameMarker); + if (entryStart === -1) { + throw new Error(`Could not find catalog entry for '${service}'.`); + } - const end = source.indexOf(endMarker, start); - if (end === -1) { - throw new Error("Could not find DEFAULT_VERSIONS declaration end."); - } + const versionMarker = " defaultVersion: "; + const versionStart = updated.indexOf(versionMarker, entryStart + nameMarker.length); + if (versionStart === -1 || versionStart - entryStart > 300) { + throw new Error(`Could not find defaultVersion for '${service}'.`); + } + const versionEnd = updated.indexOf("\n", versionStart); + if (versionEnd === -1) { + throw new Error(`Could not find defaultVersion line end for '${service}'.`); + } - return `${source.slice(0, start)}${renderDefaultVersions(versions)}${source.slice( - end + endMarker.length, - )}`; + updated = `${updated.slice(0, versionStart)}${versionMarker}${JSON.stringify( + versions[service], + )},${updated.slice(versionEnd)}`; + } + return updated; } async function main() { const checkOnly = process.argv.includes("--check"); const dockerfile = await readFile(dockerfilePath, "utf8"); - const versionsSource = await readFile(versionsPath, "utf8"); + const catalogSource = await readFile(catalogPath, "utf8"); const versions = readVersionManifestFromDockerfile(dockerfile); - const syncedSource = syncDefaultVersionsSource(versionsSource, versions); + const syncedSource = syncDefaultVersionsSource(catalogSource, versions); - if (syncedSource === versionsSource) { + if (syncedSource === catalogSource) { console.log("DEFAULT_VERSIONS is already synced with the Dockerfile manifest."); return; } @@ -123,7 +120,7 @@ async function main() { return; } - await Bun.write(versionsPath, syncedSource); + await Bun.write(catalogPath, syncedSource); console.log("Synced DEFAULT_VERSIONS with the Dockerfile manifest."); } diff --git a/packages/stack/src/BinaryResolver.integration.test.ts b/packages/stack/src/BinaryResolver.integration.test.ts index b7ff757b7d..5dc58fe38d 100644 --- a/packages/stack/src/BinaryResolver.integration.test.ts +++ b/packages/stack/src/BinaryResolver.integration.test.ts @@ -18,7 +18,7 @@ import { afterEach } from "vitest"; import { BinaryResolver } from "./BinaryResolver.ts"; import { DownloadError } from "./errors.ts"; import { detectPlatform } from "./Platform.ts"; -import { nativeReleaseForService } from "./ServiceArtifacts.ts"; +import { nativeReleaseForService } from "./ServiceCatalog.ts"; import { DEFAULT_VERSIONS } from "./versions.ts"; const tempRoots: string[] = []; diff --git a/packages/stack/src/BinaryResolver.ts b/packages/stack/src/BinaryResolver.ts index dbd68113b7..f6ee4070d4 100644 --- a/packages/stack/src/BinaryResolver.ts +++ b/packages/stack/src/BinaryResolver.ts @@ -8,8 +8,8 @@ import { nativeReleaseForService, type ArchiveFormat, type NativeReleaseArtifact, -} from "./ServiceArtifacts.ts"; -import type { ServiceName } from "./versions.ts"; +} from "./ServiceCatalog.ts"; +import type { ServiceName } from "./ServiceName.ts"; export interface BinarySpec { readonly service: ServiceName; diff --git a/packages/stack/src/BinaryResolver.unit.test.ts b/packages/stack/src/BinaryResolver.unit.test.ts index 3aabc93834..d0576010bd 100644 --- a/packages/stack/src/BinaryResolver.unit.test.ts +++ b/packages/stack/src/BinaryResolver.unit.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "@effect/vitest"; import { BinaryResolver } from "./BinaryResolver.ts"; -import { nativeReleaseForService } from "./ServiceArtifacts.ts"; +import { nativeReleaseForService } from "./ServiceCatalog.ts"; import { DEFAULT_VERSIONS } from "./versions.ts"; const postgresVersion = DEFAULT_VERSIONS.postgres; diff --git a/packages/stack/src/ServiceActivation.ts b/packages/stack/src/ServiceActivation.ts index 0edb4dafb9..ae5c05189b 100644 --- a/packages/stack/src/ServiceActivation.ts +++ b/packages/stack/src/ServiceActivation.ts @@ -3,40 +3,11 @@ import type { ServiceReadyError } from "@supabase/process-compose"; import { Context, Effect, Layer } from "effect"; import { StackBuildError, StackNotRunningError } from "./errors.ts"; import { stackServiceStartupBudgetSeconds } from "./services/health-budgets.ts"; -import { SERVICE_NAMES, type ServiceName } from "./versions.ts"; - -export interface ServiceActivationPolicy { - /** Whether the public service must already be running when lazy startup completes. */ - readonly startup: "eager" | "lazy"; - /** Other public services required when this service is activated. */ - readonly activates?: ReadonlyArray; - /** Private companions whose lifecycle is exclusively owned by this service. */ - readonly owns?: ReadonlyArray; -} - -/** - * Central ownership map for lazy startup. Services with a direct TCP or HTTP - * endpoint must be running before that endpoint is published. Companion - * services are activated with the public service that consumes them. - */ -export const SERVICE_ACTIVATION_POLICY: Readonly> = { - postgres: { startup: "eager" }, - postgrest: { startup: "lazy" }, - auth: { startup: "lazy" }, - "edge-runtime": { startup: "lazy" }, - realtime: { startup: "eager" }, - storage: { startup: "lazy", activates: ["imgproxy"], owns: ["imgproxy"] }, - imgproxy: { startup: "lazy" }, - mailpit: { startup: "eager" }, - pgmeta: { startup: "lazy" }, - studio: { startup: "eager", activates: ["analytics"] }, - analytics: { startup: "lazy", activates: ["vector"], owns: ["vector"] }, - vector: { startup: "lazy" }, - pooler: { startup: "eager" }, -}; +import { SERVICE_NAMES, serviceMetadata } from "./ServiceCatalog.ts"; +import type { ServiceName } from "./ServiceName.ts"; export const eagerServices = (enabled: ReadonlyArray): ReadonlyArray => - enabled.filter((service) => SERVICE_ACTIVATION_POLICY[service].startup === "eager"); + enabled.filter((service) => serviceMetadata(service).activation.startup === "eager"); export const activationTargetsForService = ( enabledServices: ReadonlyArray, @@ -46,7 +17,7 @@ export const activationTargetsForService = ( const targets = new Set(); const addWithCompanions = (target: ServiceName): void => { if (!enabled.has(target) || targets.has(target)) return; - for (const activated of SERVICE_ACTIVATION_POLICY[target].activates ?? []) { + for (const activated of serviceMetadata(target).activation.activates) { addWithCompanions(activated); } targets.add(target); @@ -88,7 +59,7 @@ export const lifecycleTargetsForService = ( const addWithOwnedCompanions = (target: ServiceName): void => { add(target); - for (const owned of SERVICE_ACTIVATION_POLICY[target].owns ?? []) { + for (const owned of serviceMetadata(target).activation.owns) { addWithOwnedCompanions(owned); } }; diff --git a/packages/stack/src/ServiceActivation.unit.test.ts b/packages/stack/src/ServiceActivation.unit.test.ts index 83336d573d..7b8a09de96 100644 --- a/packages/stack/src/ServiceActivation.unit.test.ts +++ b/packages/stack/src/ServiceActivation.unit.test.ts @@ -4,13 +4,12 @@ import { activationTargetsForService, eagerServices, lifecycleTargetsForService, - SERVICE_ACTIVATION_POLICY, } from "./ServiceActivation.ts"; -import { SERVICE_NAMES } from "./versions.ts"; +import { SERVICE_CATALOG, SERVICE_NAMES } from "./ServiceCatalog.ts"; describe("service activation", () => { it("defines an access policy for every stack service", () => { - expect(Object.keys(SERVICE_ACTIVATION_POLICY).sort()).toEqual([...SERVICE_NAMES].sort()); + expect(Object.keys(SERVICE_CATALOG).sort()).toEqual([...SERVICE_NAMES].sort()); }); it("starts direct endpoints eagerly", () => { diff --git a/packages/stack/src/ServiceArtifacts.ts b/packages/stack/src/ServiceArtifacts.ts deleted file mode 100644 index 65921365c7..0000000000 --- a/packages/stack/src/ServiceArtifacts.ts +++ /dev/null @@ -1,213 +0,0 @@ -import { - authAssetName, - edgeRuntimeAssetName, - postgresAssetName, - postgrestAssetName, - type PlatformInfo, -} from "./Platform.ts"; -import type { ServiceName } from "./versions.ts"; - -type ArtifactOwnership = "supabase" | "upstream"; -type ServiceRuntimeSupport = "native-preferred" | "docker-only"; -export type ArchiveFormat = "tar.gz" | "tar.xz" | "zip"; - -export interface NativeReleaseArtifact { - readonly provider: string; - readonly assetName: string; - readonly archive: ArchiveFormat; - readonly downloadUrl: string; - readonly checksumUrl: string | null; - readonly stripComponents: boolean; -} - -interface NativeReleaseSource { - readonly provider: string; - readonly resolve: (version: string, platform: PlatformInfo) => NativeReleaseArtifact | undefined; -} - -interface DockerImageSource { - readonly ownership: ArtifactOwnership; - readonly repository: string; - readonly tagPrefix?: string; -} - -export interface ServiceArtifactDefinition { - readonly runtimeSupport: ServiceRuntimeSupport; - readonly docker: DockerImageSource; - readonly native?: NativeReleaseSource; -} - -const SUPABASE_ECR_REGISTRY = "public.ecr.aws/supabase"; -const SUPABASE_DOCKER_HUB_REGISTRY = "supabase"; -const SUPABASE_GHCR_REGISTRY = "ghcr.io/supabase"; - -const nativeRelease = ( - provider: string, - assetName: string | null, - archive: ArchiveFormat, - downloadUrl: string, - options?: { - readonly checksumUrl?: string; - readonly stripComponents?: boolean; - }, -): NativeReleaseArtifact | undefined => - assetName === null - ? undefined - : { - provider, - assetName, - archive, - downloadUrl, - checksumUrl: options?.checksumUrl ?? null, - stripComponents: options?.stripComponents ?? false, - }; - -const authReleaseTag = (version: string): string => - version.includes("-rc.") ? `rc${version}` : `v${version}`; - -export const SERVICE_ARTIFACTS: Record = { - postgres: { - runtimeSupport: "native-preferred", - docker: { ownership: "supabase", repository: "postgres" }, - native: { - provider: "github.com/supabase/postgres", - resolve: (version, platform) => { - const assetName = postgresAssetName(platform); - const cliVersion = `${version}-cli`; - const url = `https://github.com/supabase/postgres/releases/download/v${cliVersion}/supabase-postgres-v${cliVersion}-${assetName}.tar.gz`; - return nativeRelease("github.com/supabase/postgres", assetName, "tar.gz", url, { - checksumUrl: `${url}.sha256`, - stripComponents: true, - }); - }, - }, - }, - postgrest: { - runtimeSupport: "native-preferred", - docker: { ownership: "supabase", repository: "postgrest", tagPrefix: "v" }, - native: { - provider: "github.com/PostgREST/postgrest", - resolve: (version, platform) => { - const assetName = postgrestAssetName(platform); - const archive = assetName?.startsWith("windows") === true ? "zip" : "tar.xz"; - return nativeRelease( - "github.com/PostgREST/postgrest", - assetName, - archive, - `https://github.com/PostgREST/postgrest/releases/download/v${version}/postgrest-v${version}-${assetName}.${archive}`, - ); - }, - }, - }, - auth: { - runtimeSupport: "native-preferred", - docker: { ownership: "supabase", repository: "gotrue", tagPrefix: "v" }, - native: { - provider: "github.com/supabase/auth", - resolve: (version, platform) => { - const assetName = authAssetName(platform); - return nativeRelease( - "github.com/supabase/auth", - assetName, - "tar.gz", - `https://github.com/supabase/auth/releases/download/${authReleaseTag(version)}/auth-v${version}-${assetName}.tar.gz`, - ); - }, - }, - }, - "edge-runtime": { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "edge-runtime", tagPrefix: "v" }, - native: { - provider: "github.com/supabase/edge-runtime", - resolve: (version, platform) => { - const assetName = edgeRuntimeAssetName(platform); - return nativeRelease( - "github.com/supabase/edge-runtime", - assetName, - "tar.gz", - `https://github.com/supabase/edge-runtime/releases/download/v${version}/edge-runtime-v${version}-${assetName}.tar.gz`, - ); - }, - }, - }, - realtime: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "realtime", tagPrefix: "v" }, - }, - storage: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "storage-api", tagPrefix: "v" }, - }, - imgproxy: { - runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "darthsim/imgproxy" }, - }, - mailpit: { - runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "axllent/mailpit" }, - }, - pgmeta: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "postgres-meta", tagPrefix: "v" }, - }, - studio: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "studio" }, - }, - analytics: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "logflare" }, - }, - vector: { - runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "timberio/vector" }, - }, - pooler: { - runtimeSupport: "docker-only", - docker: { ownership: "supabase", repository: "supavisor" }, - }, -}; - -export const nativeReleaseForService = ( - service: ServiceName, - version: string, - platform: PlatformInfo, -): NativeReleaseArtifact | undefined => - SERVICE_ARTIFACTS[service].native?.resolve(version, platform); - -export const isDockerOnlyService = (service: ServiceName): boolean => - SERVICE_ARTIFACTS[service].runtimeSupport === "docker-only"; - -const dockerTag = (service: ServiceName, version: string): string => { - const source = SERVICE_ARTIFACTS[service].docker; - return `${source.tagPrefix ?? ""}${version}`; -}; - -export const dockerImageForArtifact = (service: ServiceName, version: string): string => { - const source = SERVICE_ARTIFACTS[service].docker; - const repository = - source.ownership === "supabase" - ? `${SUPABASE_ECR_REGISTRY}/${source.repository}` - : source.repository; - return `${repository}:${dockerTag(service, version)}`; -}; - -export const dockerImageCandidatesForArtifact = ( - service: ServiceName, - version: string, -): ReadonlyArray => { - const source = SERVICE_ARTIFACTS[service].docker; - const tag = dockerTag(service, version); - if (source.ownership === "upstream") { - return [`${source.repository}:${tag}`]; - } - return [ - `${SUPABASE_ECR_REGISTRY}/${source.repository}:${tag}`, - `${SUPABASE_DOCKER_HUB_REGISTRY}/${source.repository}:${tag}`, - `${SUPABASE_GHCR_REGISTRY}/${source.repository}:${tag}`, - ]; -}; - -export const imageTagPrefixForService = (service: ServiceName): string | undefined => - SERVICE_ARTIFACTS[service].docker.tagPrefix; diff --git a/packages/stack/src/ServiceCatalog.ts b/packages/stack/src/ServiceCatalog.ts new file mode 100644 index 0000000000..aa004a98cf --- /dev/null +++ b/packages/stack/src/ServiceCatalog.ts @@ -0,0 +1,337 @@ +import { Record } from "effect"; +import { + authAssetName, + edgeRuntimeAssetName, + postgresAssetName, + postgrestAssetName, + type PlatformInfo, +} from "./Platform.ts"; +import type { PortField } from "./PortAllocator.ts"; +import type { ServiceName } from "./ServiceName.ts"; + +type ArtifactOwnership = "supabase" | "upstream"; +type ServiceRuntimeSupport = "native-preferred" | "docker-only"; +export type ArchiveFormat = "tar.gz" | "tar.xz" | "zip"; + +export interface NativeReleaseArtifact { + readonly provider: string; + readonly assetName: string; + readonly archive: ArchiveFormat; + readonly downloadUrl: string; + readonly checksumUrl: string | null; + readonly stripComponents: boolean; +} + +interface NativeReleaseSource { + readonly provider: string; + readonly resolve: (version: string, platform: PlatformInfo) => NativeReleaseArtifact | undefined; +} + +interface DockerImageSource { + readonly ownership: ArtifactOwnership; + readonly repository: string; + readonly tagPrefix?: string; +} + +interface ServiceArtifactDefinition { + readonly docker: DockerImageSource; + readonly native?: NativeReleaseSource; +} + +interface ServiceActivationPolicy { + /** Whether the public service must already be running when lazy startup completes. */ + readonly startup: "eager" | "lazy"; + /** Other public services required when this service is activated. */ + readonly activates: ReadonlyArray; + /** Private companions whose lifecycle is exclusively owned by this service. */ + readonly owns: ReadonlyArray; +} + +type ServiceConfigKey = + | "postgres" + | "postgrest" + | "auth" + | "edgeRuntime" + | "realtime" + | "storage" + | "imgproxy" + | "mailpit" + | "pgmeta" + | "studio" + | "analytics" + | "vector" + | "pooler"; + +export interface ServiceCatalogEntry { + readonly name: Name; + readonly configKey: ServiceConfigKey; + readonly defaultVersion: string; + readonly runtimeSupport: ServiceRuntimeSupport; + readonly artifact: ServiceArtifactDefinition; + readonly activation: ServiceActivationPolicy; + readonly portFields: ReadonlyArray; +} + +const SUPABASE_ECR_REGISTRY = "public.ecr.aws/supabase"; +const SUPABASE_DOCKER_HUB_REGISTRY = "supabase"; +const SUPABASE_GHCR_REGISTRY = "ghcr.io/supabase"; + +const nativeRelease = ( + provider: string, + assetName: string | null, + archive: ArchiveFormat, + downloadUrl: string, + options?: { + readonly checksumUrl?: string; + readonly stripComponents?: boolean; + }, +): NativeReleaseArtifact | undefined => + assetName === null + ? undefined + : { + provider, + assetName, + archive, + downloadUrl, + checksumUrl: options?.checksumUrl ?? null, + stripComponents: options?.stripComponents ?? false, + }; + +const authReleaseTag = (version: string): string => + version.includes("-rc.") ? `rc${version}` : `v${version}`; + +/** + * Exhaustive static identity and capability metadata for public stack services. + * Cross-service topology and process definitions deliberately remain in StackBuilder. + */ +export const SERVICE_CATALOG = { + postgres: { + name: "postgres", + configKey: "postgres", + defaultVersion: "17.6.1.158", + runtimeSupport: "native-preferred", + artifact: { + docker: { ownership: "supabase", repository: "postgres" }, + native: { + provider: "github.com/supabase/postgres", + resolve: (version, platform) => { + const assetName = postgresAssetName(platform); + const cliVersion = `${version}-cli`; + const url = `https://github.com/supabase/postgres/releases/download/v${cliVersion}/supabase-postgres-v${cliVersion}-${assetName}.tar.gz`; + return nativeRelease("github.com/supabase/postgres", assetName, "tar.gz", url, { + checksumUrl: `${url}.sha256`, + stripComponents: true, + }); + }, + }, + }, + activation: { startup: "eager", activates: [], owns: [] }, + portFields: ["dbPort"], + }, + postgrest: { + name: "postgrest", + configKey: "postgrest", + defaultVersion: "14.16", + runtimeSupport: "native-preferred", + artifact: { + docker: { ownership: "supabase", repository: "postgrest", tagPrefix: "v" }, + native: { + provider: "github.com/PostgREST/postgrest", + resolve: (version, platform) => { + const assetName = postgrestAssetName(platform); + const archive = assetName?.startsWith("windows") === true ? "zip" : "tar.xz"; + return nativeRelease( + "github.com/PostgREST/postgrest", + assetName, + archive, + `https://github.com/PostgREST/postgrest/releases/download/v${version}/postgrest-v${version}-${assetName}.${archive}`, + ); + }, + }, + }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: ["postgrestPort", "postgrestAdminPort"], + }, + auth: { + name: "auth", + configKey: "auth", + defaultVersion: "2.195.0", + runtimeSupport: "native-preferred", + artifact: { + docker: { ownership: "supabase", repository: "gotrue", tagPrefix: "v" }, + native: { + provider: "github.com/supabase/auth", + resolve: (version, platform) => { + const assetName = authAssetName(platform); + return nativeRelease( + "github.com/supabase/auth", + assetName, + "tar.gz", + `https://github.com/supabase/auth/releases/download/${authReleaseTag(version)}/auth-v${version}-${assetName}.tar.gz`, + ); + }, + }, + }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: ["authPort"], + }, + "edge-runtime": { + name: "edge-runtime", + configKey: "edgeRuntime", + defaultVersion: "1.74.3", + runtimeSupport: "docker-only", + artifact: { + docker: { ownership: "supabase", repository: "edge-runtime", tagPrefix: "v" }, + native: { + provider: "github.com/supabase/edge-runtime", + resolve: (version, platform) => { + const assetName = edgeRuntimeAssetName(platform); + return nativeRelease( + "github.com/supabase/edge-runtime", + assetName, + "tar.gz", + `https://github.com/supabase/edge-runtime/releases/download/v${version}/edge-runtime-v${version}-${assetName}.tar.gz`, + ); + }, + }, + }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: ["edgeRuntimePort", "edgeRuntimeInspectorPort"], + }, + realtime: { + name: "realtime", + configKey: "realtime", + defaultVersion: "2.123.1", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "supabase", repository: "realtime", tagPrefix: "v" } }, + activation: { startup: "eager", activates: [], owns: [] }, + portFields: ["realtimePort"], + }, + storage: { + name: "storage", + configKey: "storage", + defaultVersion: "1.68.1", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "supabase", repository: "storage-api", tagPrefix: "v" } }, + activation: { startup: "lazy", activates: ["imgproxy"], owns: ["imgproxy"] }, + portFields: ["storagePort"], + }, + imgproxy: { + name: "imgproxy", + configKey: "imgproxy", + defaultVersion: "v3.8.0", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "upstream", repository: "darthsim/imgproxy" } }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: ["imgproxyPort"], + }, + mailpit: { + name: "mailpit", + configKey: "mailpit", + defaultVersion: "v1.30.2", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "upstream", repository: "axllent/mailpit" } }, + activation: { startup: "eager", activates: [], owns: [] }, + portFields: ["mailpitPort", "mailpitSmtpPort", "mailpitPop3Port"], + }, + pgmeta: { + name: "pgmeta", + configKey: "pgmeta", + defaultVersion: "0.96.6", + runtimeSupport: "docker-only", + artifact: { + docker: { ownership: "supabase", repository: "postgres-meta", tagPrefix: "v" }, + }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: ["pgmetaPort"], + }, + studio: { + name: "studio", + configKey: "studio", + defaultVersion: "2026.08.03-sha-022b374", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "supabase", repository: "studio" } }, + activation: { startup: "eager", activates: ["analytics"], owns: [] }, + portFields: ["studioPort"], + }, + analytics: { + name: "analytics", + configKey: "analytics", + defaultVersion: "1.49.2", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "supabase", repository: "logflare" } }, + activation: { startup: "lazy", activates: ["vector"], owns: ["vector"] }, + portFields: ["analyticsPort"], + }, + vector: { + name: "vector", + configKey: "vector", + defaultVersion: "0.53.0-alpine", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "upstream", repository: "timberio/vector" } }, + activation: { startup: "lazy", activates: [], owns: [] }, + portFields: [], + }, + pooler: { + name: "pooler", + configKey: "pooler", + defaultVersion: "2.9.7", + runtimeSupport: "docker-only", + artifact: { docker: { ownership: "supabase", repository: "supavisor" } }, + activation: { startup: "eager", activates: [], owns: [] }, + portFields: ["poolerPort", "poolerApiPort"], + }, +} satisfies { readonly [Name in ServiceName]: ServiceCatalogEntry }; + +export const SERVICE_NAMES: ReadonlyArray = Record.keys(SERVICE_CATALOG); + +export const DEFAULT_VERSIONS: Readonly> = Record.map( + SERVICE_CATALOG, + (metadata) => metadata.defaultVersion, +); + +export const serviceMetadata = (service: ServiceName): ServiceCatalogEntry => + SERVICE_CATALOG[service]; + +export const nativeReleaseForService = ( + service: ServiceName, + version: string, + platform: PlatformInfo, +): NativeReleaseArtifact | undefined => + serviceMetadata(service).artifact.native?.resolve(version, platform); + +export const isDockerOnlyService = (service: ServiceName): boolean => + SERVICE_CATALOG[service].runtimeSupport === "docker-only"; + +const dockerTag = (service: ServiceName, version: string): string => { + const source = serviceMetadata(service).artifact.docker; + return `${source.tagPrefix ?? ""}${version}`; +}; + +export const dockerImageForArtifact = (service: ServiceName, version: string): string => { + const source = SERVICE_CATALOG[service].artifact.docker; + const repository = + source.ownership === "supabase" + ? `${SUPABASE_ECR_REGISTRY}/${source.repository}` + : source.repository; + return `${repository}:${dockerTag(service, version)}`; +}; + +export const dockerImageCandidatesForArtifact = ( + service: ServiceName, + version: string, +): ReadonlyArray => { + const source = SERVICE_CATALOG[service].artifact.docker; + const tag = dockerTag(service, version); + if (source.ownership === "upstream") { + return [`${source.repository}:${tag}`]; + } + return [ + `${SUPABASE_ECR_REGISTRY}/${source.repository}:${tag}`, + `${SUPABASE_DOCKER_HUB_REGISTRY}/${source.repository}:${tag}`, + `${SUPABASE_GHCR_REGISTRY}/${source.repository}:${tag}`, + ]; +}; + +export const imageTagPrefixForService = (service: ServiceName): string | undefined => + serviceMetadata(service).artifact.docker.tagPrefix; diff --git a/packages/stack/src/ServiceCatalog.unit.test.ts b/packages/stack/src/ServiceCatalog.unit.test.ts new file mode 100644 index 0000000000..d00f5a177b --- /dev/null +++ b/packages/stack/src/ServiceCatalog.unit.test.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from "vitest"; +import { DEFAULT_VERSIONS, SERVICE_CATALOG, SERVICE_NAMES } from "./ServiceCatalog.ts"; + +describe("ServiceCatalog", () => { + it("derives exhaustive iteration and defaults from catalog entries", () => { + expect(SERVICE_NAMES).toEqual(Object.keys(SERVICE_CATALOG)); + expect(Object.keys(DEFAULT_VERSIONS)).toEqual(SERVICE_NAMES); + + for (const service of SERVICE_NAMES) { + expect(SERVICE_CATALOG[service].name).toBe(service); + expect(DEFAULT_VERSIONS[service]).toBe(SERVICE_CATALOG[service].defaultVersion); + } + }); + + it("references only catalog services in activation relationships", () => { + const knownServices = new Set(SERVICE_NAMES); + for (const service of SERVICE_NAMES) { + const { activates, owns } = SERVICE_CATALOG[service].activation; + expect([...activates, ...owns].every((related) => knownServices.has(related))).toBe(true); + } + }); +}); diff --git a/packages/stack/src/ServiceName.ts b/packages/stack/src/ServiceName.ts new file mode 100644 index 0000000000..f9524cacd2 --- /dev/null +++ b/packages/stack/src/ServiceName.ts @@ -0,0 +1,15 @@ +/** Public Supabase services represented in a local stack. */ +export type ServiceName = + | "postgres" + | "postgrest" + | "auth" + | "edge-runtime" + | "realtime" + | "storage" + | "imgproxy" + | "mailpit" + | "pgmeta" + | "studio" + | "analytics" + | "vector" + | "pooler"; diff --git a/packages/stack/src/ServicePorts.ts b/packages/stack/src/ServicePorts.ts index 27a0b125c0..cb61925149 100644 --- a/packages/stack/src/ServicePorts.ts +++ b/packages/stack/src/ServicePorts.ts @@ -1,31 +1,16 @@ import type { PortField } from "./PortAllocator.ts"; -import { enabledServicesForConfig, type ResolvedStackConfig } from "./StackBuilder.ts"; -import { SERVICE_NAMES, type ServiceName } from "./versions.ts"; +import { enabledServicesForConfig } from "./StackBuilder.ts"; +import { SERVICE_CATALOG, SERVICE_NAMES } from "./ServiceCatalog.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; export const allocatedPortFieldsForConfig = ( config: ResolvedStackConfig, ): ReadonlyArray => [ "apiPort", - ...enabledServicesForConfig(config).flatMap((service) => SERVICE_PORT_FIELDS[service]), + ...enabledServicesForConfig(config).flatMap((service) => SERVICE_CATALOG[service].portFields), ]; -const SERVICE_PORT_FIELDS = { - postgres: ["dbPort"], - postgrest: ["postgrestPort", "postgrestAdminPort"], - auth: ["authPort"], - "edge-runtime": ["edgeRuntimePort", "edgeRuntimeInspectorPort"], - realtime: ["realtimePort"], - storage: ["storagePort"], - imgproxy: ["imgproxyPort"], - mailpit: ["mailpitPort", "mailpitSmtpPort", "mailpitPop3Port"], - pgmeta: ["pgmetaPort"], - studio: ["studioPort"], - analytics: ["analyticsPort"], - vector: [], - pooler: ["poolerPort", "poolerApiPort"], -} as const satisfies Readonly>>; - export const portFieldsForService = (name: string): ReadonlyArray => { const service = SERVICE_NAMES.find((candidate) => candidate === name); - return service === undefined ? [] : SERVICE_PORT_FIELDS[service]; + return service === undefined ? [] : SERVICE_CATALOG[service].portFields; }; diff --git a/packages/stack/src/Stack.ts b/packages/stack/src/Stack.ts index 06ab6abd44..dbe2ce70e0 100644 --- a/packages/stack/src/Stack.ts +++ b/packages/stack/src/Stack.ts @@ -4,7 +4,7 @@ import { Effect, Layer, Schema, Context, Stream } from "effect"; import { StackBuildError } from "./errors.ts"; import type { FunctionsConfig } from "./functions.ts"; import { StackLifecycleCoordinator } from "./StackLifecycleCoordinator.ts"; -import type { EdgeRuntimeConfig, ResolvedStackConfig } from "./StackBuilder.ts"; +import type { EdgeRuntimeConfig, ResolvedStackConfig } from "./StackConfig.ts"; import { StackServiceState } from "./StackServiceState.ts"; export interface StackInfo { diff --git a/packages/stack/src/Stack.unit.test.ts b/packages/stack/src/Stack.unit.test.ts index 7a959e37bc..189a7f9e11 100644 --- a/packages/stack/src/Stack.unit.test.ts +++ b/packages/stack/src/Stack.unit.test.ts @@ -11,7 +11,7 @@ import { StackLifecycleCoordinator } from "./StackLifecycleCoordinator.ts"; import { StackMetadataPersistence } from "./StackMetadataPersistence.ts"; import { StackPreparation } from "./StackPreparation.ts"; import { StackBuilder } from "./StackBuilder.ts"; -import type { ResolvedStackConfig } from "./StackBuilder.ts"; +import { DEFAULT_STACK_READINESS_POLICY, type ResolvedStackConfig } from "./StackConfig.ts"; import { DEFAULT_VERSIONS } from "./versions.ts"; const testJwtSecret = "super-secret-jwt-token-with-at-least-32-characters-long"; @@ -44,6 +44,7 @@ const defaultConfig: ResolvedStackConfig = { projectDir: "/tmp/supabase-project", mode: "native", startupMode: "eager", + readiness: DEFAULT_STACK_READINESS_POLICY, jwtSecret: testJwtSecret, ports: defaultPorts, apiPort: 54321, diff --git a/packages/stack/src/StackBuilder.ts b/packages/stack/src/StackBuilder.ts index dc95e58583..17f5e75ed4 100644 --- a/packages/stack/src/StackBuilder.ts +++ b/packages/stack/src/StackBuilder.ts @@ -3,7 +3,6 @@ import type { ResolvedGraph, ServiceDef } from "@supabase/process-compose"; import { Effect, Layer, Context } from "effect"; import type { CleanupTargets } from "./CleanupTargets.ts"; import { StackBuildError } from "./errors.ts"; -import type { FunctionsConfig, ResolvedFunctionsConfig } from "./functions.ts"; import { generateJwks } from "./JwtGenerator.ts"; import { detectPlatform, @@ -11,7 +10,6 @@ import { dockerNetworkArgs, dockerPortMapArgs, } from "./Platform.ts"; -import type { ServiceResolution } from "./resolve.ts"; import { analyticsDockerRuntimeNetwork, makeAnalyticsServiceDocker } from "./services/analytics.ts"; import { makeAuthServiceDocker, makeAuthServiceNative } from "./services/auth.ts"; import { @@ -38,271 +36,12 @@ import { dependencyTimeoutSecondsForServices, POSTGRES_INIT_COMPLETION_BUDGET_SECONDS, } from "./services/health-budgets.ts"; -import type { PreparedStackArtifacts } from "./StackPreparation.ts"; +import type { PreparedStackArtifacts, ServiceResolution } from "./StackPreparation.ts"; import type { StackServiceProjectionCatalog } from "./StackStateProjection.ts"; -import type { AllocatedPorts } from "./PortAllocator.ts"; -import type { ServiceName, VersionManifest } from "./versions.ts"; - -export interface PostgresConfig { - readonly port?: number; - readonly dataDir?: string; - readonly version?: string; - /** - * When true (default), the bundled initial schema GRANTs that expose new tables, views, - * sequences, and functions in `public` to the Data API roles (`anon`, `authenticated`, - * `service_role`) are kept in place. When false, those default privileges are revoked so the - * local stack matches the new cloud default and requires explicit GRANTs to surface entities - * through the Data API. - */ - readonly autoExposeNewTables?: boolean; -} - -export interface PostgrestConfig { - readonly schemas?: ReadonlyArray; - readonly extraSearchPath?: ReadonlyArray; - readonly maxRows?: number; - readonly version?: string; -} - -export interface AuthConfig { - readonly port?: number; - readonly siteUrl?: string; - readonly jwtExpiry?: number; - readonly externalUrl?: string; - readonly version?: string; -} - -export interface RealtimeConfig { - readonly port?: number; - readonly version?: string; - readonly tenantId?: string; - readonly encryptionKey?: string; - readonly secretKeyBase?: string; - readonly maxHeaderLength?: number; -} - -export interface EdgeRuntimeConfig { - readonly enabled?: boolean; - readonly port?: number; - readonly inspectorPort?: number; - readonly policy?: "oneshot" | "per_worker"; - readonly version?: string; - readonly env?: Readonly>; -} - -export interface StorageConfig { - readonly port?: number; - readonly dataDir?: string; - readonly fileSizeLimit?: string; - readonly s3ProtocolEnabled?: boolean; - readonly version?: string; -} - -export interface ImgproxyConfig { - readonly port?: number; - readonly version?: string; -} - -export interface MailpitConfig { - readonly port?: number; - readonly smtpPort?: number; - readonly pop3Port?: number; - readonly version?: string; - readonly adminEmail?: string; - readonly senderName?: string; -} - -export interface PgmetaConfig { - readonly port?: number; - readonly version?: string; -} - -export interface StudioConfig { - readonly port?: number; - readonly apiUrl?: string; - readonly version?: string; -} - -export interface AnalyticsConfig { - readonly port?: number; - readonly version?: string; - readonly backend?: "postgres" | "bigquery"; - readonly apiKey?: string; -} - -export interface VectorConfig { - readonly version?: string; -} - -export interface PoolerConfig { - readonly port?: number; - readonly apiPort?: number; - readonly mode?: "transaction" | "session"; - readonly version?: string; - readonly tenantId?: string; - readonly encryptionKey?: string; - readonly secretKeyBase?: string; - readonly defaultPoolSize?: number; - readonly maxClientConn?: number; -} - -export interface StackConfig { - readonly cacheRoot?: string; - readonly stackRoot?: string; - readonly runtimeRoot?: string; - readonly projectDir?: string; - readonly mode?: "native" | "auto" | "docker"; - /** Start all services immediately, or defer proxied services until first use. */ - readonly startupMode?: "eager" | "lazy"; - readonly jwtSecret?: string; - readonly port?: number; - readonly publishableKey?: string; - readonly secretKey?: string; - readonly functions?: FunctionsConfig | false; - readonly postgres?: PostgresConfig; - readonly postgrest?: PostgrestConfig | false; - readonly auth?: AuthConfig | false; - readonly edgeRuntime?: EdgeRuntimeConfig | false; - readonly realtime?: RealtimeConfig | false; - readonly storage?: StorageConfig | false; - readonly imgproxy?: ImgproxyConfig | false; - readonly mailpit?: MailpitConfig | false; - readonly pgmeta?: PgmetaConfig | false; - readonly studio?: StudioConfig | false; - readonly analytics?: AnalyticsConfig | false; - readonly vector?: VectorConfig | false; - readonly pooler?: PoolerConfig | false; -} - -export interface ResolvedPostgresConfig { - readonly port: number; - readonly dataDir: string; - readonly version: string; - readonly autoExposeNewTables: boolean; -} - -export interface ResolvedPostgrestConfig { - readonly port: number; - readonly adminPort: number; - readonly schemas: ReadonlyArray; - readonly extraSearchPath: ReadonlyArray; - readonly maxRows: number; - readonly version: string; -} - -export interface ResolvedAuthConfig { - readonly port: number; - readonly siteUrl: string; - readonly jwtExpiry: number; - readonly externalUrl: string; - readonly version: string; -} - -export interface ResolvedRealtimeConfig { - readonly port: number; - readonly version: string; - readonly tenantId: string; - readonly encryptionKey: string; - readonly secretKeyBase: string; - readonly maxHeaderLength: number; -} - -export interface ResolvedEdgeRuntimeConfig { - readonly enabled: boolean; - readonly port: number; - readonly inspectorPort: number; - readonly policy: "oneshot" | "per_worker"; - readonly version: string; - readonly env: Readonly>; -} - -export interface ResolvedStorageConfig { - readonly port: number; - readonly version: string; - readonly dataDir: string; - readonly fileSizeLimit: string; - readonly s3ProtocolEnabled: boolean; -} - -export interface ResolvedImgproxyConfig { - readonly port: number; - readonly version: string; -} - -export interface ResolvedMailpitConfig { - readonly port: number; - readonly smtpPort: number; - readonly pop3Port: number; - readonly version: string; - readonly adminEmail: string; - readonly senderName: string; -} - -export interface ResolvedPgmetaConfig { - readonly port: number; - readonly version: string; -} - -export interface ResolvedStudioConfig { - readonly port: number; - readonly version: string; - readonly apiUrl: string; -} - -export interface ResolvedAnalyticsConfig { - readonly port: number; - readonly version: string; - readonly backend: "postgres" | "bigquery"; - readonly apiKey: string; -} - -export interface ResolvedVectorConfig { - readonly version: string; -} - -export interface ResolvedPoolerConfig { - readonly port: number; - readonly apiPort: number; - readonly mode: "transaction" | "session"; - readonly version: string; - readonly tenantId: string; - readonly encryptionKey: string; - readonly secretKeyBase: string; - readonly defaultPoolSize: number; - readonly maxClientConn: number; -} - -export interface ResolvedStackConfig { - readonly cacheRoot: string; - readonly stackRoot: string; - readonly runtimeRoot: string; - readonly projectDir: string; - readonly mode: "native" | "auto" | "docker"; - readonly startupMode: "eager" | "lazy"; - readonly jwtSecret: string; - readonly ports: AllocatedPorts; - readonly apiPort: number; - readonly dbPort: number; - readonly publishableKey: string; - readonly secretKey: string; - readonly functions: ResolvedFunctionsConfig | false; - readonly autoManagedPaths: ReadonlyArray; - readonly anonJwt: string; - readonly serviceRoleJwt: string; - readonly postgres: ResolvedPostgresConfig; - readonly postgrest: ResolvedPostgrestConfig | false; - readonly auth: ResolvedAuthConfig | false; - readonly edgeRuntime: ResolvedEdgeRuntimeConfig | false; - readonly realtime: ResolvedRealtimeConfig | false; - readonly storage: ResolvedStorageConfig | false; - readonly imgproxy: ResolvedImgproxyConfig | false; - readonly mailpit: ResolvedMailpitConfig | false; - readonly pgmeta: ResolvedPgmetaConfig | false; - readonly studio: ResolvedStudioConfig | false; - readonly analytics: ResolvedAnalyticsConfig | false; - readonly vector: ResolvedVectorConfig | false; - readonly pooler: ResolvedPoolerConfig | false; -} +import { SERVICE_NAMES, serviceMetadata } from "./ServiceCatalog.ts"; +import type { ServiceName } from "./ServiceName.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; +import type { VersionManifest } from "./versions.ts"; export interface BuildResult { readonly graph: ResolvedGraph; @@ -310,18 +49,9 @@ export interface BuildResult { readonly serviceProjection: StackServiceProjectionCatalog; } -const dockerOnlyServices = [ - "edge-runtime", - "realtime", - "storage", - "imgproxy", - "mailpit", - "pgmeta", - "studio", - "analytics", - "vector", - "pooler", -] as const; +const dockerOnlyServices = SERVICE_NAMES.filter( + (service) => serviceMetadata(service).runtimeSupport === "docker-only", +); const dependsOnPostgres = (hasPostgresInit: boolean): ReadonlyArray => hasPostgresInit @@ -372,10 +102,8 @@ const hasAutoManagedPath = (config: ResolvedStackConfig, path: string) => path.startsWith(`${managedPath}\\`), ); -const resolvedConfigForService = ( - config: ResolvedStackConfig, - service: Exclude, -) => (service === "edge-runtime" ? config.edgeRuntime : config[service]); +const resolvedConfigForService = (config: ResolvedStackConfig, service: ServiceName) => + config[serviceMetadata(service).configKey]; export const validateResolvedConfig = ( config: ResolvedStackConfig, @@ -419,67 +147,22 @@ export const validateResolvedConfig = ( } }); -export const enabledServicesForConfig = ( - config: ResolvedStackConfig, -): ReadonlyArray => { - const services: ServiceName[] = ["postgres"]; +export const enabledServicesForConfig = (config: ResolvedStackConfig): ReadonlyArray => + SERVICE_NAMES.filter( + (service) => service === "postgres" || resolvedConfigForService(config, service) !== false, + ); - if (config.postgrest !== false) { - services.push("postgrest"); - } - if (config.auth !== false) { - services.push("auth"); - } - if (config.edgeRuntime !== false) { - services.push("edge-runtime"); - } - if (config.realtime !== false) { - services.push("realtime"); - } - if (config.storage !== false) { - services.push("storage"); - } - if (config.imgproxy !== false) { - services.push("imgproxy"); - } - if (config.mailpit !== false) { - services.push("mailpit"); - } - if (config.pgmeta !== false) { - services.push("pgmeta"); - } - if (config.studio !== false) { - services.push("studio"); - } - if (config.analytics !== false) { - services.push("analytics"); - } - if (config.vector !== false) { - services.push("vector"); - } - if (config.pooler !== false) { - services.push("pooler"); +export const versionsForConfig = (config: ResolvedStackConfig): Partial => { + const versions: Partial> = {}; + for (const service of enabledServicesForConfig(config)) { + const serviceConfig = resolvedConfigForService(config, service); + if (serviceConfig !== false) { + versions[service] = serviceConfig.version; + } } - - return services; + return versions; }; -export const versionsForConfig = (config: ResolvedStackConfig): Partial => ({ - postgres: config.postgres.version, - ...(config.postgrest === false ? {} : { postgrest: config.postgrest.version }), - ...(config.auth === false ? {} : { auth: config.auth.version }), - ...(config.edgeRuntime === false ? {} : { "edge-runtime": config.edgeRuntime.version }), - ...(config.realtime === false ? {} : { realtime: config.realtime.version }), - ...(config.storage === false ? {} : { storage: config.storage.version }), - ...(config.imgproxy === false ? {} : { imgproxy: config.imgproxy.version }), - ...(config.mailpit === false ? {} : { mailpit: config.mailpit.version }), - ...(config.pgmeta === false ? {} : { pgmeta: config.pgmeta.version }), - ...(config.studio === false ? {} : { studio: config.studio.version }), - ...(config.analytics === false ? {} : { analytics: config.analytics.version }), - ...(config.vector === false ? {} : { vector: config.vector.version }), - ...(config.pooler === false ? {} : { pooler: config.pooler.version }), -}); - const requirePreparedResolution = ( prepared: PreparedStackArtifacts, service: ServiceName, diff --git a/packages/stack/src/StackBuilder.unit.test.ts b/packages/stack/src/StackBuilder.unit.test.ts index 75935f65ac..3344b5f2e9 100644 --- a/packages/stack/src/StackBuilder.unit.test.ts +++ b/packages/stack/src/StackBuilder.unit.test.ts @@ -5,7 +5,7 @@ import { mockBinaryResolver } from "../tests/helpers/mocks.ts"; import { defaultPublishableKey, defaultSecretKey, generateJwt } from "./JwtGenerator.ts"; import { StackBuilder } from "./StackBuilder.ts"; import type { BuildResult } from "./StackBuilder.ts"; -import type { ResolvedStackConfig } from "./StackBuilder.ts"; +import { DEFAULT_STACK_READINESS_POLICY, type ResolvedStackConfig } from "./StackConfig.ts"; import { enabledServicesForConfig, versionsForConfig } from "./StackBuilder.ts"; import { nativePostgresNeedsDockerAccess } from "./StackBuilder.ts"; import type { AllocatedPorts } from "./PortAllocator.ts"; @@ -47,6 +47,7 @@ const baseConfig: ResolvedStackConfig = { projectDir: "/tmp/supabase-project", mode: "auto", startupMode: "eager", + readiness: DEFAULT_STACK_READINESS_POLICY, jwtSecret: testJwtSecret, ports: basePorts, apiPort: 3000, diff --git a/packages/stack/src/StackConfig.ts b/packages/stack/src/StackConfig.ts new file mode 100644 index 0000000000..cbf6552fc8 --- /dev/null +++ b/packages/stack/src/StackConfig.ts @@ -0,0 +1,295 @@ +import type { FunctionsConfig, ResolvedFunctionsConfig } from "./functions.ts"; +import type { AllocatedPorts } from "./PortAllocator.ts"; + +type StackMode = "native" | "auto" | "docker"; +type StackStartupMode = "eager" | "lazy"; + +export type ReadinessPolicy = + | { readonly mode: "finite"; readonly timeoutMs: number } + | { readonly mode: "infinite" }; + +export type ReadyOptions = { readonly mode: "inherit" } | ReadinessPolicy; + +/** Standalone stacks wait at most two minutes unless a caller or launch Adapter chooses otherwise. */ +export const DEFAULT_STACK_READINESS_POLICY: ReadinessPolicy = { + mode: "finite", + timeoutMs: 120_000, +}; + +/** Resolve readiness with per-call policy taking precedence over stack policy and package default. */ +export const resolveReadinessPolicy = (input: { + readonly readyOptions?: ReadyOptions; + readonly stackPolicy?: ReadinessPolicy; +}): ReadinessPolicy => + input.readyOptions === undefined || input.readyOptions.mode === "inherit" + ? (input.stackPolicy ?? DEFAULT_STACK_READINESS_POLICY) + : input.readyOptions; + +export interface PostgresConfig { + readonly port?: number; + readonly dataDir?: string; + readonly version?: string; + /** + * When true (default), the bundled initial schema GRANTs that expose new tables, views, + * sequences, and functions in `public` to the Data API roles (`anon`, `authenticated`, + * `service_role`) are kept in place. When false, those default privileges are revoked so the + * local stack matches the new cloud default and requires explicit GRANTs to surface entities + * through the Data API. + */ + readonly autoExposeNewTables?: boolean; +} + +export interface PostgrestConfig { + readonly schemas?: ReadonlyArray; + readonly extraSearchPath?: ReadonlyArray; + readonly maxRows?: number; + readonly version?: string; +} + +export interface AuthConfig { + readonly port?: number; + readonly siteUrl?: string; + readonly jwtExpiry?: number; + readonly externalUrl?: string; + readonly version?: string; +} + +export interface RealtimeConfig { + readonly port?: number; + readonly version?: string; + readonly tenantId?: string; + readonly encryptionKey?: string; + readonly secretKeyBase?: string; + readonly maxHeaderLength?: number; +} + +export interface EdgeRuntimeConfig { + readonly enabled?: boolean; + readonly port?: number; + readonly inspectorPort?: number; + readonly policy?: "oneshot" | "per_worker"; + readonly version?: string; + readonly env?: Readonly>; +} + +export interface StorageConfig { + readonly port?: number; + readonly dataDir?: string; + readonly fileSizeLimit?: string; + readonly s3ProtocolEnabled?: boolean; + readonly version?: string; +} + +export interface ImgproxyConfig { + readonly port?: number; + readonly version?: string; +} + +export interface MailpitConfig { + readonly port?: number; + readonly smtpPort?: number; + readonly pop3Port?: number; + readonly version?: string; + readonly adminEmail?: string; + readonly senderName?: string; +} + +export interface PgmetaConfig { + readonly port?: number; + readonly version?: string; +} + +export interface StudioConfig { + readonly port?: number; + readonly apiUrl?: string; + readonly version?: string; +} + +export interface AnalyticsConfig { + readonly port?: number; + readonly version?: string; + readonly backend?: "postgres" | "bigquery"; + readonly apiKey?: string; +} + +export interface VectorConfig { + readonly version?: string; +} + +export interface PoolerConfig { + readonly port?: number; + readonly apiPort?: number; + readonly mode?: "transaction" | "session"; + readonly version?: string; + readonly tenantId?: string; + readonly encryptionKey?: string; + readonly secretKeyBase?: string; + readonly defaultPoolSize?: number; + readonly maxClientConn?: number; +} + +export interface StackConfig { + readonly cacheRoot?: string; + readonly stackRoot?: string; + readonly runtimeRoot?: string; + readonly projectDir?: string; + readonly mode?: StackMode; + /** Start all services immediately, or defer proxied services until first use. */ + readonly startupMode?: StackStartupMode; + /** Stack-wide readiness policy. Per-call ReadyOptions take precedence. */ + readonly readiness?: ReadinessPolicy; + readonly jwtSecret?: string; + readonly port?: number; + readonly publishableKey?: string; + readonly secretKey?: string; + readonly functions?: FunctionsConfig | false; + readonly postgres?: PostgresConfig; + readonly postgrest?: PostgrestConfig | false; + readonly auth?: AuthConfig | false; + readonly edgeRuntime?: EdgeRuntimeConfig | false; + readonly realtime?: RealtimeConfig | false; + readonly storage?: StorageConfig | false; + readonly imgproxy?: ImgproxyConfig | false; + readonly mailpit?: MailpitConfig | false; + readonly pgmeta?: PgmetaConfig | false; + readonly studio?: StudioConfig | false; + readonly analytics?: AnalyticsConfig | false; + readonly vector?: VectorConfig | false; + readonly pooler?: PoolerConfig | false; +} + +export interface ResolvedPostgresConfig { + readonly port: number; + readonly dataDir: string; + readonly version: string; + readonly autoExposeNewTables: boolean; +} + +export interface ResolvedPostgrestConfig { + readonly port: number; + readonly adminPort: number; + readonly schemas: ReadonlyArray; + readonly extraSearchPath: ReadonlyArray; + readonly maxRows: number; + readonly version: string; +} + +export interface ResolvedAuthConfig { + readonly port: number; + readonly siteUrl: string; + readonly jwtExpiry: number; + readonly externalUrl: string; + readonly version: string; +} + +export interface ResolvedRealtimeConfig { + readonly port: number; + readonly version: string; + readonly tenantId: string; + readonly encryptionKey: string; + readonly secretKeyBase: string; + readonly maxHeaderLength: number; +} + +export interface ResolvedEdgeRuntimeConfig { + readonly enabled: boolean; + readonly port: number; + readonly inspectorPort: number; + readonly policy: "oneshot" | "per_worker"; + readonly version: string; + readonly env: Readonly>; +} + +export interface ResolvedStorageConfig { + readonly port: number; + readonly version: string; + readonly dataDir: string; + readonly fileSizeLimit: string; + readonly s3ProtocolEnabled: boolean; +} + +export interface ResolvedImgproxyConfig { + readonly port: number; + readonly version: string; +} + +export interface ResolvedMailpitConfig { + readonly port: number; + readonly smtpPort: number; + readonly pop3Port: number; + readonly version: string; + readonly adminEmail: string; + readonly senderName: string; +} + +export interface ResolvedPgmetaConfig { + readonly port: number; + readonly version: string; +} + +export interface ResolvedStudioConfig { + readonly port: number; + readonly version: string; + readonly apiUrl: string; +} + +export interface ResolvedAnalyticsConfig { + readonly port: number; + readonly version: string; + readonly backend: "postgres" | "bigquery"; + readonly apiKey: string; +} + +export interface ResolvedVectorConfig { + readonly version: string; +} + +export interface ResolvedPoolerConfig { + readonly port: number; + readonly apiPort: number; + readonly mode: "transaction" | "session"; + readonly version: string; + readonly tenantId: string; + readonly encryptionKey: string; + readonly secretKeyBase: string; + readonly defaultPoolSize: number; + readonly maxClientConn: number; +} + +export interface ResolvedStackConfig { + readonly cacheRoot: string; + readonly stackRoot: string; + readonly runtimeRoot: string; + readonly projectDir: string; + readonly mode: StackMode; + readonly startupMode: StackStartupMode; + readonly readiness: ReadinessPolicy; + readonly jwtSecret: string; + readonly ports: AllocatedPorts; + readonly apiPort: number; + readonly dbPort: number; + readonly publishableKey: string; + readonly secretKey: string; + readonly functions: ResolvedFunctionsConfig | false; + readonly autoManagedPaths: ReadonlyArray; + readonly anonJwt: string; + readonly serviceRoleJwt: string; + readonly postgres: ResolvedPostgresConfig; + readonly postgrest: ResolvedPostgrestConfig | false; + readonly auth: ResolvedAuthConfig | false; + readonly edgeRuntime: ResolvedEdgeRuntimeConfig | false; + readonly realtime: ResolvedRealtimeConfig | false; + readonly storage: ResolvedStorageConfig | false; + readonly imgproxy: ResolvedImgproxyConfig | false; + readonly mailpit: ResolvedMailpitConfig | false; + readonly pgmeta: ResolvedPgmetaConfig | false; + readonly studio: ResolvedStudioConfig | false; + readonly analytics: ResolvedAnalyticsConfig | false; + readonly vector: ResolvedVectorConfig | false; + readonly pooler: ResolvedPoolerConfig | false; +} + +export interface ResolvedDaemonConfig extends ResolvedStackConfig { + readonly name: string; + readonly projectDir: string; +} diff --git a/packages/stack/src/StackConfig.unit.test.ts b/packages/stack/src/StackConfig.unit.test.ts new file mode 100644 index 0000000000..9c3d3b0a84 --- /dev/null +++ b/packages/stack/src/StackConfig.unit.test.ts @@ -0,0 +1,59 @@ +import { describe, expect, it } from "vitest"; +import { + DEFAULT_STACK_READINESS_POLICY, + resolveReadinessPolicy, + type ReadinessPolicy, +} from "./StackConfig.ts"; + +const finite = (timeoutMs: number): ReadinessPolicy => ({ mode: "finite", timeoutMs }); + +describe("resolveReadinessPolicy", () => { + it("uses the finite package default when neither level chooses a policy", () => { + expect(resolveReadinessPolicy({})).toEqual(DEFAULT_STACK_READINESS_POLICY); + }); + + it("inherits finite and infinite stack policies", () => { + expect( + resolveReadinessPolicy({ + readyOptions: { mode: "inherit" }, + stackPolicy: finite(120_000), + }), + ).toEqual(finite(120_000)); + expect( + resolveReadinessPolicy({ + readyOptions: { mode: "inherit" }, + stackPolicy: { mode: "infinite" }, + }), + ).toEqual({ mode: "infinite" }); + }); + + it("allows shorter and longer per-call finite policies", () => { + expect( + resolveReadinessPolicy({ + readyOptions: finite(5_000), + stackPolicy: finite(120_000), + }), + ).toEqual(finite(5_000)); + expect( + resolveReadinessPolicy({ + readyOptions: finite(300_000), + stackPolicy: finite(120_000), + }), + ).toEqual(finite(300_000)); + }); + + it("allows either level to opt into or override infinite waiting", () => { + expect( + resolveReadinessPolicy({ + readyOptions: { mode: "infinite" }, + stackPolicy: finite(120_000), + }), + ).toEqual({ mode: "infinite" }); + expect( + resolveReadinessPolicy({ + readyOptions: finite(30_000), + stackPolicy: { mode: "infinite" }, + }), + ).toEqual(finite(30_000)); + }); +}); diff --git a/packages/stack/src/StackConfigResolver.ts b/packages/stack/src/StackConfigResolver.ts new file mode 100644 index 0000000000..88cd888abf --- /dev/null +++ b/packages/stack/src/StackConfigResolver.ts @@ -0,0 +1,613 @@ +import { mkdtempSync } from "node:fs"; +import { readdir, readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Effect, Schema } from "effect"; +import { toStackError } from "./errors.ts"; +import { + defaultJwtSecret, + defaultPublishableKey, + defaultSecretKey, + generateJwt, +} from "./JwtGenerator.ts"; +import { + DEFAULT_MANAGED_STACK_NAME, + defaultCacheRoot, + defaultManagedProjectsRoot, + defaultManagedRuntimeRoot, + defaultManagedStackRoot, + shortTempPrefixRoot, +} from "./paths.ts"; +import { + allocatePorts, + DEFAULT_PORTS, + PORT_FIELDS, + type AllocatedPorts, + type PortAllocationError, + type PortInput, + type PortSelectionOptions, +} from "./PortAllocator.ts"; +import { StackMetadataSchema } from "./StackMetadata.ts"; +import { resolveReadinessPolicy } from "./StackConfig.ts"; +import type { + AnalyticsConfig, + AuthConfig, + EdgeRuntimeConfig, + ImgproxyConfig, + MailpitConfig, + PgmetaConfig, + PoolerConfig, + PostgrestConfig, + RealtimeConfig, + ResolvedAnalyticsConfig, + ResolvedAuthConfig, + ResolvedDaemonConfig, + ResolvedEdgeRuntimeConfig, + ResolvedImgproxyConfig, + ResolvedMailpitConfig, + ResolvedPgmetaConfig, + ResolvedPoolerConfig, + ResolvedPostgrestConfig, + ResolvedRealtimeConfig, + ResolvedStackConfig, + ResolvedStorageConfig, + ResolvedStudioConfig, + ResolvedVectorConfig, + StackConfig, + StorageConfig, + StudioConfig, + VectorConfig, +} from "./StackConfig.ts"; +import { DEFAULT_VERSIONS } from "./ServiceCatalog.ts"; + +const StackMetadataFileSchema = Schema.fromJsonString(StackMetadataSchema); +const decodeStackMetadataFile = Schema.decodeUnknownSync(StackMetadataFileSchema); + +export function defaultManagedStackName(_cwd: string): string { + return DEFAULT_MANAGED_STACK_NAME; +} + +export interface ResolveConfigOptions { + readonly stackRoot?: string; + readonly runtimeRoot?: string; + readonly preferredPorts?: Partial; + readonly reservedPorts?: ReadonlySet; + readonly portAllocator?: ( + input: PortInput, + options: PortSelectionOptions, + ) => Effect.Effect; +} + +interface ResolvedRoots { + readonly cacheRoot: string; + readonly stackRoot: string; + readonly runtimeRoot: string; + readonly autoManagedPaths: ReadonlyArray; +} + +const makeTempRoot = (prefix: string) => mkdtempSync(join(shortTempPrefixRoot(), prefix)); + +const resolveRoots = (config: StackConfig, opts: ResolveConfigOptions): ResolvedRoots => { + const cacheRoot = config.cacheRoot ?? defaultCacheRoot(); + const autoManagedPaths: string[] = []; + + const stackRoot = + opts.stackRoot ?? + config.stackRoot ?? + (() => { + const dir = makeTempRoot("sb-stack-"); + autoManagedPaths.push(dir); + return dir; + })(); + + const runtimeRoot = + opts.runtimeRoot ?? + config.runtimeRoot ?? + (() => { + const dir = makeTempRoot("sb-run-"); + autoManagedPaths.push(dir); + return dir; + })(); + + return { + cacheRoot, + stackRoot, + runtimeRoot, + autoManagedPaths, + }; +}; + +const resolveDataDir = ( + explicitDir: string | undefined, + stackRoot: string, + suffix: string, +): string => explicitDir ?? join(stackRoot, "data", suffix); + +async function readStackMetadataFile(filePath: string) { + try { + const content = await readFile(filePath, "utf8"); + return decodeStackMetadataFile(content); + } catch { + return undefined; + } +} + +async function readOwnedPorts(stackRoot: string): Promise { + const metadata = await readStackMetadataFile(join(stackRoot, "stack.json")); + return metadata?.ports; +} + +async function readReservedPorts( + projectsRoot: string, + currentStackRoot: string, +): Promise> { + const reserved = new Set(); + + let projectEntries: Array<{ isDirectory(): boolean; name: string }>; + try { + projectEntries = await readdir(projectsRoot, { withFileTypes: true }); + } catch { + return reserved; + } + + await Promise.all( + projectEntries.map(async (projectEntry) => { + if (!projectEntry.isDirectory()) { + return; + } + + const stacksRoot = join(projectsRoot, projectEntry.name, "stacks"); + let stackEntries: Array<{ isDirectory(): boolean; name: string }>; + try { + stackEntries = await readdir(stacksRoot, { withFileTypes: true }); + } catch { + return; + } + + await Promise.all( + stackEntries.map(async (stackEntry) => { + if (!stackEntry.isDirectory()) { + return; + } + + const stackRoot = join(stacksRoot, stackEntry.name); + if (stackRoot === currentStackRoot) { + return; + } + + const ports = (await readStackMetadataFile(join(stackRoot, "stack.json")))?.ports; + if (ports === undefined) { + return; + } + + for (const field of PORT_FIELDS) { + reserved.add(ports[field]); + } + }), + ); + }), + ); + + return reserved; +} + +async function readReservedPortsInStacksRoot( + stacksRoot: string, + currentStackRoot: string, +): Promise> { + const reserved = new Set(); + + let stackEntries: Array<{ isDirectory(): boolean; name: string }>; + try { + stackEntries = await readdir(stacksRoot, { withFileTypes: true }); + } catch { + return reserved; + } + + await Promise.all( + stackEntries.map(async (stackEntry) => { + if (!stackEntry.isDirectory()) { + return; + } + + const stackRoot = join(stacksRoot, stackEntry.name); + if (stackRoot === currentStackRoot) { + return; + } + + const ports = (await readStackMetadataFile(join(stackRoot, "stack.json")))?.ports; + if (ports === undefined) { + return; + } + + for (const field of PORT_FIELDS) { + reserved.add(ports[field]); + } + }), + ); + + return reserved; +} + +function resolvePostgrestConfig( + input: PostgrestConfig | undefined, + raw: PostgrestConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedPostgrestConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.postgrestPort, + adminPort: ports.postgrestAdminPort, + schemas: cfg.schemas ?? ["public", "graphql_public"], + extraSearchPath: cfg.extraSearchPath ?? ["public", "extensions"], + maxRows: cfg.maxRows ?? 1000, + version: cfg.version ?? DEFAULT_VERSIONS.postgrest, + }; +} + +function resolveAuthConfig( + input: AuthConfig | undefined, + raw: AuthConfig | false | undefined, + ports: AllocatedPorts, + apiPort: number, +): ResolvedAuthConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.authPort, + siteUrl: cfg.siteUrl ?? "http://localhost:3000", + jwtExpiry: cfg.jwtExpiry ?? 3600, + externalUrl: cfg.externalUrl ?? `http://127.0.0.1:${apiPort}`, + version: cfg.version ?? DEFAULT_VERSIONS.auth, + }; +} + +function resolveRealtimeConfig( + input: RealtimeConfig | undefined, + raw: RealtimeConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedRealtimeConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.realtimePort, + version: cfg.version ?? DEFAULT_VERSIONS.realtime, + tenantId: cfg.tenantId ?? "realtime-dev", + encryptionKey: cfg.encryptionKey ?? "supabaserealtime", + secretKeyBase: + cfg.secretKeyBase ?? "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG", + maxHeaderLength: cfg.maxHeaderLength ?? 4096, + }; +} + +function resolveEdgeRuntimeConfig( + input: EdgeRuntimeConfig | undefined, + raw: EdgeRuntimeConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedEdgeRuntimeConfig | false { + if (raw === false || raw?.enabled === false) return false; + const cfg = input ?? {}; + return { + enabled: cfg.enabled ?? true, + port: ports.edgeRuntimePort, + inspectorPort: ports.edgeRuntimeInspectorPort, + policy: cfg.policy ?? "per_worker", + version: cfg.version ?? DEFAULT_VERSIONS["edge-runtime"], + env: cfg.env ?? {}, + }; +} + +function resolveFunctionsConfig(config: StackConfig) { + if (config.functions === false) return false; + return { + envFile: config.functions?.envFile, + noVerifyJwt: config.functions?.noVerifyJwt ?? false, + }; +} + +function resolveStorageConfig( + input: StorageConfig | undefined, + raw: StorageConfig | false | undefined, + ports: AllocatedPorts, + opts: ResolveConfigOptions, +): ResolvedStorageConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.storagePort, + version: cfg.version ?? DEFAULT_VERSIONS.storage, + dataDir: resolveDataDir(cfg.dataDir, opts.stackRoot!, "storage"), + fileSizeLimit: cfg.fileSizeLimit ?? "50MiB", + s3ProtocolEnabled: cfg.s3ProtocolEnabled ?? true, + }; +} + +function resolveImgproxyConfig( + input: ImgproxyConfig | undefined, + raw: ImgproxyConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedImgproxyConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.imgproxyPort, + version: cfg.version ?? DEFAULT_VERSIONS.imgproxy, + }; +} + +function resolveMailpitConfig( + input: MailpitConfig | undefined, + raw: MailpitConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedMailpitConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.mailpitPort, + smtpPort: ports.mailpitSmtpPort, + pop3Port: ports.mailpitPop3Port, + version: cfg.version ?? DEFAULT_VERSIONS.mailpit, + adminEmail: cfg.adminEmail ?? "admin@email.com", + senderName: cfg.senderName ?? "Admin", + }; +} + +function resolvePgmetaConfig( + input: PgmetaConfig | undefined, + raw: PgmetaConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedPgmetaConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.pgmetaPort, + version: cfg.version ?? DEFAULT_VERSIONS.pgmeta, + }; +} + +function resolveStudioConfig( + input: StudioConfig | undefined, + raw: StudioConfig | false | undefined, + ports: AllocatedPorts, + apiPort: number, +): ResolvedStudioConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.studioPort, + version: cfg.version ?? DEFAULT_VERSIONS.studio, + apiUrl: cfg.apiUrl ?? `http://127.0.0.1:${apiPort}`, + }; +} + +function resolveAnalyticsConfig( + input: AnalyticsConfig | undefined, + raw: AnalyticsConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedAnalyticsConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.analyticsPort, + version: cfg.version ?? DEFAULT_VERSIONS.analytics, + backend: cfg.backend ?? "postgres", + apiKey: cfg.apiKey ?? "api-key", + }; +} + +function resolveVectorConfig( + input: VectorConfig | undefined, + raw: VectorConfig | false | undefined, +): ResolvedVectorConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + version: cfg.version ?? DEFAULT_VERSIONS.vector, + }; +} + +function resolvePoolerConfig( + input: PoolerConfig | undefined, + raw: PoolerConfig | false | undefined, + ports: AllocatedPorts, +): ResolvedPoolerConfig | false { + if (raw === false) return false; + const cfg = input ?? {}; + return { + port: ports.poolerPort, + apiPort: ports.poolerApiPort, + mode: cfg.mode ?? "transaction", + version: cfg.version ?? DEFAULT_VERSIONS.pooler, + tenantId: cfg.tenantId ?? "pooler-dev", + encryptionKey: cfg.encryptionKey ?? "12345678901234567890123456789032", + secretKeyBase: + cfg.secretKeyBase ?? "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG", + defaultPoolSize: cfg.defaultPoolSize ?? 20, + maxClientConn: cfg.maxClientConn ?? 100, + }; +} + +export async function resolveConfig( + input?: StackConfig, + opts: ResolveConfigOptions = {}, +): Promise { + const config = input ?? {}; + const projectDir = config.projectDir ?? process.cwd(); + const resolvedMode = config.mode ?? "auto"; + const roots = resolveRoots(config, opts); + const postgresInput = config.postgres ?? {}; + const postgrestInput = config.postgrest !== false ? (config.postgrest ?? undefined) : undefined; + const authInput = config.auth !== false ? (config.auth ?? undefined) : undefined; + const edgeRuntimeEnabled = + !(resolvedMode === "native" && config.edgeRuntime === undefined) && + config.edgeRuntime !== false && + (config.edgeRuntime?.enabled ?? true) !== false; + const realtimeEnabled = config.realtime !== undefined && config.realtime !== false; + const storageEnabled = config.storage !== undefined && config.storage !== false; + const imgproxyEnabled = config.imgproxy !== undefined && config.imgproxy !== false; + const mailpitEnabled = config.mailpit !== undefined && config.mailpit !== false; + const pgmetaEnabled = config.pgmeta !== undefined && config.pgmeta !== false; + const studioEnabled = config.studio !== undefined && config.studio !== false; + const analyticsEnabled = config.analytics !== undefined && config.analytics !== false; + const vectorEnabled = config.vector !== undefined && config.vector !== false; + const poolerEnabled = config.pooler !== undefined && config.pooler !== false; + const edgeRuntimeInput = edgeRuntimeEnabled ? (config.edgeRuntime ?? undefined) : undefined; + const realtimeInput = realtimeEnabled ? (config.realtime ?? undefined) : undefined; + const storageInput = storageEnabled ? (config.storage ?? undefined) : undefined; + const imgproxyInput = imgproxyEnabled ? (config.imgproxy ?? undefined) : undefined; + const mailpitInput = mailpitEnabled ? (config.mailpit ?? undefined) : undefined; + const pgmetaInput = pgmetaEnabled ? (config.pgmeta ?? undefined) : undefined; + const studioInput = studioEnabled ? (config.studio ?? undefined) : undefined; + const analyticsInput = analyticsEnabled ? (config.analytics ?? undefined) : undefined; + const vectorInput = vectorEnabled ? (config.vector ?? undefined) : undefined; + const poolerInput = poolerEnabled ? (config.pooler ?? undefined) : undefined; + + const postgresDataDir = resolveDataDir(postgresInput.dataDir, roots.stackRoot, "postgres"); + + const ports = await Effect.runPromise( + (opts.portAllocator ?? allocatePorts)( + { + apiPort: config.port, + dbPort: postgresInput.port, + authPort: authInput?.port, + postgrestPort: undefined, + postgrestAdminPort: undefined, + edgeRuntimePort: edgeRuntimeInput?.port, + edgeRuntimeInspectorPort: edgeRuntimeInput?.inspectorPort, + realtimePort: realtimeInput?.port, + storagePort: storageInput?.port, + imgproxyPort: imgproxyInput?.port, + mailpitPort: mailpitInput?.port, + mailpitSmtpPort: mailpitInput?.smtpPort, + mailpitPop3Port: mailpitInput?.pop3Port, + pgmetaPort: pgmetaInput?.port, + studioPort: studioInput?.port, + analyticsPort: analyticsInput?.port, + poolerPort: poolerInput?.port, + poolerApiPort: poolerInput?.apiPort, + }, + { + preferred: opts.preferredPorts, + reserved: opts.reservedPorts, + }, + ), + ).catch((error: unknown) => { + throw toStackError(error); + }); + + const jwtSecret = config.jwtSecret ?? defaultJwtSecret; + const anonJwt = generateJwt(jwtSecret, "anon"); + const serviceRoleJwt = generateJwt(jwtSecret, "service_role"); + + return { + cacheRoot: roots.cacheRoot, + stackRoot: roots.stackRoot, + runtimeRoot: roots.runtimeRoot, + projectDir, + mode: resolvedMode, + startupMode: config.startupMode ?? "eager", + readiness: resolveReadinessPolicy({ stackPolicy: config.readiness }), + jwtSecret, + ports, + apiPort: ports.apiPort, + dbPort: ports.dbPort, + publishableKey: config.publishableKey ?? defaultPublishableKey, + secretKey: config.secretKey ?? defaultSecretKey, + functions: resolveFunctionsConfig(config), + autoManagedPaths: roots.autoManagedPaths, + anonJwt, + serviceRoleJwt, + postgres: { + port: ports.dbPort, + dataDir: postgresDataDir, + version: postgresInput.version ?? DEFAULT_VERSIONS.postgres, + autoExposeNewTables: postgresInput.autoExposeNewTables ?? true, + }, + postgrest: resolvePostgrestConfig(postgrestInput, config.postgrest, ports), + auth: resolveAuthConfig(authInput, config.auth, ports, ports.apiPort), + edgeRuntime: edgeRuntimeEnabled + ? resolveEdgeRuntimeConfig(edgeRuntimeInput, config.edgeRuntime, ports) + : false, + realtime: realtimeEnabled + ? resolveRealtimeConfig(realtimeInput, config.realtime, ports) + : false, + storage: storageEnabled + ? resolveStorageConfig(storageInput, config.storage, ports, { + ...opts, + stackRoot: roots.stackRoot, + }) + : false, + imgproxy: imgproxyEnabled + ? resolveImgproxyConfig(imgproxyInput, config.imgproxy, ports) + : false, + mailpit: mailpitEnabled ? resolveMailpitConfig(mailpitInput, config.mailpit, ports) : false, + pgmeta: pgmetaEnabled ? resolvePgmetaConfig(pgmetaInput, config.pgmeta, ports) : false, + studio: studioEnabled + ? resolveStudioConfig(studioInput, config.studio, ports, ports.apiPort) + : false, + analytics: analyticsEnabled + ? resolveAnalyticsConfig(analyticsInput, config.analytics, ports) + : false, + vector: vectorEnabled ? resolveVectorConfig(vectorInput, config.vector) : false, + pooler: poolerEnabled ? resolvePoolerConfig(poolerInput, config.pooler, ports) : false, + }; +} + +export type DaemonConfigInput = StackConfig & { + readonly cwd: string; + readonly name?: string; + readonly projectDir?: string; + readonly projectStateRoot?: string; +}; + +export async function resolveDaemonConfig( + input: DaemonConfigInput, + opts: Pick = {}, +): Promise { + const { cwd, name, projectDir, projectStateRoot, ...stackConfig } = input; + if (stackConfig.stackRoot !== undefined || stackConfig.runtimeRoot !== undefined) { + throw new Error("Managed daemon stacks derive stackRoot and runtimeRoot automatically"); + } + const effectiveProjectDir = projectDir ?? cwd; + const resolvedName = name ?? defaultManagedStackName(effectiveProjectDir); + const cacheRoot = stackConfig.cacheRoot ?? defaultCacheRoot(); + const stackRoot = + projectStateRoot !== undefined + ? join(projectStateRoot, "stacks", resolvedName) + : defaultManagedStackRoot(cacheRoot, effectiveProjectDir, resolvedName); + const runtimeRoot = defaultManagedRuntimeRoot(stackRoot); + const savedPorts = await readOwnedPorts(stackRoot); + const reservedPortSets = await Promise.all([ + readReservedPorts(defaultManagedProjectsRoot(cacheRoot), stackRoot), + projectStateRoot === undefined + ? Promise.resolve>(new Set()) + : readReservedPortsInStacksRoot(join(projectStateRoot, "stacks"), stackRoot), + ]); + const reservedPorts = new Set(); + for (const ports of reservedPortSets) { + for (const port of ports) { + reservedPorts.add(port); + } + } + const resolved = await resolveConfig( + { + ...stackConfig, + cacheRoot, + stackRoot, + runtimeRoot, + projectDir: effectiveProjectDir, + }, + { + stackRoot, + runtimeRoot, + preferredPorts: savedPorts ?? DEFAULT_PORTS, + reservedPorts, + portAllocator: opts.portAllocator, + }, + ); + return { + ...resolved, + name: resolvedName, + projectDir: effectiveProjectDir, + }; +} diff --git a/packages/stack/src/StackLifecycleCoordinator.ts b/packages/stack/src/StackLifecycleCoordinator.ts index 86b4097bae..5127288e21 100644 --- a/packages/stack/src/StackLifecycleCoordinator.ts +++ b/packages/stack/src/StackLifecycleCoordinator.ts @@ -34,12 +34,13 @@ import { StackBuilder, validateResolvedConfig, versionsForConfig, - type ResolvedStackConfig, } from "./StackBuilder.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; import { projectStackStates, type StackServiceProjectionCatalog } from "./StackStateProjection.ts"; import { StackServiceState } from "./StackServiceState.ts"; import type { EdgeRuntimeReloadConfig, StackInfo } from "./Stack.ts"; -import { SERVICE_NAMES, type ServiceName } from "./versions.ts"; +import { SERVICE_NAMES } from "./ServiceCatalog.ts"; +import type { ServiceName } from "./ServiceName.ts"; type LifecyclePhase = | "idle" diff --git a/packages/stack/src/StackMetadata.ts b/packages/stack/src/StackMetadata.ts index 1704993751..aa4a3fb0d9 100644 --- a/packages/stack/src/StackMetadata.ts +++ b/packages/stack/src/StackMetadata.ts @@ -1,7 +1,8 @@ import { Schema } from "effect"; import { CleanupTargetsSchema, type CleanupTargets } from "./CleanupTargets.ts"; import { AllocatedPortsSchema, type AllocatedPorts } from "./PortAllocator.ts"; -import type { ResolvedStackConfig } from "./StackBuilder.ts"; +import { serviceMetadata } from "./ServiceCatalog.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; import { SERVICE_NAMES, type ServiceName, type VersionManifest } from "./versions.ts"; const VersionManifestSchema = Schema.Struct({ @@ -76,15 +77,10 @@ export const STACK_METADATA_SCHEMA_VERSION = 1; export function runningServiceVersionsForConfig( config: ResolvedStackConfig, ): PartialVersionManifest { - const versions: Partial> = { - postgres: config.postgres.version, - }; + const versions: Partial> = {}; for (const service of SERVICE_NAMES) { - if (service === "postgres") { - continue; - } - const serviceConfig = service === "edge-runtime" ? config.edgeRuntime : config[service]; + const serviceConfig = config[serviceMetadata(service).configKey]; if (serviceConfig !== false) { versions[service] = serviceConfig.version; } diff --git a/packages/stack/src/StackPreparation.ts b/packages/stack/src/StackPreparation.ts index a363d40a78..c0feb4924d 100644 --- a/packages/stack/src/StackPreparation.ts +++ b/packages/stack/src/StackPreparation.ts @@ -3,8 +3,7 @@ import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process"; import { BinaryResolver } from "./BinaryResolver.ts"; import type { ChecksumMismatchError } from "./errors.ts"; import { DockerPullError } from "./errors.ts"; -import type { ServiceResolution } from "./resolve.ts"; -import { isDockerOnlyService } from "./ServiceArtifacts.ts"; +import { isDockerOnlyService } from "./ServiceCatalog.ts"; import { DEFAULT_VERSIONS, SERVICE_NAMES, @@ -17,6 +16,10 @@ export interface PreparedStackArtifacts { readonly resolutions: Partial>; } +export type ServiceResolution = + | { readonly type: "binary"; readonly path: string } + | { readonly type: "docker"; readonly image: string }; + export interface StackPreparationInput { readonly versions?: Partial; readonly services?: ReadonlyArray; @@ -133,9 +136,11 @@ export const prepareAssetsWithDependencies = ( concurrency: "unbounded", }); - const artifacts = { - resolutions: Object.fromEntries(results) as PreparedStackArtifacts["resolutions"], - } satisfies PreparedStackArtifacts; + const resolutions: Partial> = {}; + for (const [service, resolution] of results) { + resolutions[service] = resolution; + } + const artifacts = { resolutions } satisfies PreparedStackArtifacts; yield* publishEvent?.(new PreparationCompleted({ artifacts })) ?? Effect.void; return artifacts; }); diff --git a/packages/stack/src/bun.ts b/packages/stack/src/bun.ts index 2642e1b6c6..ebc92eb69c 100644 --- a/packages/stack/src/bun.ts +++ b/packages/stack/src/bun.ts @@ -16,7 +16,7 @@ import { } from "./prefetch.ts"; import { defaultCacheRoot } from "./paths.ts"; import { StackPreparation } from "./StackPreparation.ts"; -import type { StackConfig } from "./StackBuilder.ts"; +import type { StackConfig } from "./StackConfig.ts"; import { UnixHttpClient, UnixHttpClientError } from "./UnixHttpClient.ts"; interface BunUnixRequestInit extends RequestInit { diff --git a/packages/stack/src/cleanup.ts b/packages/stack/src/cleanup.ts index 59ee0d2cae..7e43ecb6c6 100644 --- a/packages/stack/src/cleanup.ts +++ b/packages/stack/src/cleanup.ts @@ -2,7 +2,7 @@ import { execFileSync } from "node:child_process"; import { existsSync, rmSync } from "node:fs"; import { Duration, Effect } from "effect"; import type { CleanupTargets } from "./CleanupTargets.ts"; -import type { ResolvedStackConfig } from "./StackBuilder.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; /** * Force-remove Docker containers by name. Best-effort safety net — diff --git a/packages/stack/src/createStack.ts b/packages/stack/src/createStack.ts index f0b22c2f6d..9f0559a4fc 100644 --- a/packages/stack/src/createStack.ts +++ b/packages/stack/src/createStack.ts @@ -1,8 +1,5 @@ import type { LogEntry } from "@supabase/process-compose"; -import { readdir, readFile } from "node:fs/promises"; -import { mkdtempSync } from "node:fs"; -import { join } from "node:path"; -import { Duration, Effect, type Layer, ManagedRuntime, Schema, Stream } from "effect"; +import { Duration, Effect, type Layer, ManagedRuntime, Stream } from "effect"; import { FileSystem, Path } from "effect"; import { HttpServer } from "effect/unstable/http"; import { ChildProcessSpawner } from "effect/unstable/process"; @@ -10,76 +7,17 @@ import { cleanupAutoManagedPaths, dockerForceRemove } from "./cleanup.ts"; import type { CleanupTargets } from "./CleanupTargets.ts"; import { toStackError } from "./errors.ts"; import type { FunctionsConfig } from "./functions.ts"; -import { - defaultJwtSecret, - defaultPublishableKey, - defaultSecretKey, - generateJwt, -} from "./JwtGenerator.ts"; -import { - daemonLayer, - foregroundLayer, - type DaemonConfig, - type DaemonStartError, -} from "./layers.ts"; -import { - DEFAULT_MANAGED_STACK_NAME, - defaultCacheRoot, - defaultManagedProjectsRoot, - defaultManagedRuntimeRoot, - defaultManagedStackRoot, - shortTempPrefixRoot, -} from "./paths.ts"; -import { - allocatePorts, - DEFAULT_PORTS, - PORT_FIELDS, - reservePorts, - type AllocatedPorts, - type PortInput, - type PortAllocationError, - type PortLease, - type PortSelectionOptions, -} from "./PortAllocator.ts"; +import { daemonLayer, foregroundLayer, type DaemonStartError } from "./layers.ts"; +import { PORT_FIELDS, reservePorts, type PortLease } from "./PortAllocator.ts"; import { allocatedPortFieldsForConfig } from "./ServicePorts.ts"; -import { StackMetadataSchema } from "./StackMetadata.ts"; -import { InvalidStackStateError, StackAlreadyRunningError } from "./StateManager.ts"; import { Stack } from "./Stack.ts"; import type { EdgeRuntimeReloadConfig } from "./Stack.ts"; +import type { ReadyOptions, ResolvedStackConfig, StackConfig } from "./StackConfig.ts"; +import { resolveReadinessPolicy } from "./StackConfig.ts"; +import { resolveConfig } from "./StackConfigResolver.ts"; import type { StackServiceState } from "./StackServiceState.ts"; +import { InvalidStackStateError, StackAlreadyRunningError } from "./StateManager.ts"; import { UnixHttpClient } from "./UnixHttpClient.ts"; -import type { - AnalyticsConfig, - AuthConfig, - EdgeRuntimeConfig, - ImgproxyConfig, - MailpitConfig, - PgmetaConfig, - PoolerConfig, - PostgrestConfig, - RealtimeConfig, - ResolvedAnalyticsConfig, - ResolvedAuthConfig, - ResolvedEdgeRuntimeConfig, - ResolvedImgproxyConfig, - ResolvedMailpitConfig, - ResolvedPgmetaConfig, - ResolvedPoolerConfig, - ResolvedPostgrestConfig, - ResolvedRealtimeConfig, - ResolvedStackConfig, - ResolvedStorageConfig, - ResolvedStudioConfig, - ResolvedVectorConfig, - StackConfig, - StorageConfig, - StudioConfig, - VectorConfig, -} from "./StackBuilder.ts"; -import { DEFAULT_VERSIONS } from "./versions.ts"; - -const StackMetadataFileSchema = Schema.fromJsonString(StackMetadataSchema); -const decodeStackMetadataFile = Schema.decodeUnknownSync(StackMetadataFileSchema); export type PlatformServices = | FileSystem.FileSystem @@ -96,14 +34,6 @@ export interface PlatformFactoryOptions { export type PlatformFactory = (options: PlatformFactoryOptions) => PlatformLayer; -export interface ReadyOptions { - readonly timeout?: number; -} - -export function defaultManagedStackName(_cwd: string): string { - return DEFAULT_MANAGED_STACK_NAME; -} - export interface StackHandle extends AsyncDisposable { readonly url: string; readonly dbUrl: string; @@ -127,551 +57,6 @@ export interface StackHandle extends AsyncDisposable { logHistory(name: string, limit?: number): Promise>; } -interface ResolveConfigOptions { - readonly stackRoot?: string; - readonly runtimeRoot?: string; - readonly preferredPorts?: Partial; - readonly reservedPorts?: ReadonlySet; - readonly portAllocator?: ( - input: PortInput, - options: PortSelectionOptions, - ) => Effect.Effect; -} - -interface ResolvedRoots { - readonly cacheRoot: string; - readonly stackRoot: string; - readonly runtimeRoot: string; - readonly autoManagedPaths: ReadonlyArray; -} - -const makeTempRoot = (prefix: string) => mkdtempSync(join(shortTempPrefixRoot(), prefix)); - -const resolveRoots = (config: StackConfig, opts: ResolveConfigOptions): ResolvedRoots => { - const cacheRoot = config.cacheRoot ?? defaultCacheRoot(); - const autoManagedPaths: string[] = []; - - const stackRoot = - opts.stackRoot ?? - config.stackRoot ?? - (() => { - const dir = makeTempRoot("sb-stack-"); - autoManagedPaths.push(dir); - return dir; - })(); - - const runtimeRoot = - opts.runtimeRoot ?? - config.runtimeRoot ?? - (() => { - const dir = makeTempRoot("sb-run-"); - autoManagedPaths.push(dir); - return dir; - })(); - - return { - cacheRoot, - stackRoot, - runtimeRoot, - autoManagedPaths, - }; -}; - -const resolveDataDir = ( - explicitDir: string | undefined, - stackRoot: string, - suffix: string, -): string => explicitDir ?? join(stackRoot, "data", suffix); - -async function readStackMetadataFile(filePath: string) { - try { - const content = await readFile(filePath, "utf8"); - return decodeStackMetadataFile(content); - } catch { - return undefined; - } -} - -async function readOwnedPorts(stackRoot: string): Promise { - const metadata = await readStackMetadataFile(join(stackRoot, "stack.json")); - return metadata?.ports; -} - -async function readReservedPorts( - projectsRoot: string, - currentStackRoot: string, -): Promise> { - const reserved = new Set(); - - let projectEntries: Array<{ isDirectory(): boolean; name: string }>; - try { - projectEntries = await readdir(projectsRoot, { withFileTypes: true }); - } catch { - return reserved; - } - - await Promise.all( - projectEntries.map(async (projectEntry) => { - if (!projectEntry.isDirectory()) { - return; - } - - const stacksRoot = join(projectsRoot, projectEntry.name, "stacks"); - let stackEntries: Array<{ isDirectory(): boolean; name: string }>; - try { - stackEntries = await readdir(stacksRoot, { withFileTypes: true }); - } catch { - return; - } - - await Promise.all( - stackEntries.map(async (stackEntry) => { - if (!stackEntry.isDirectory()) { - return; - } - - const stackRoot = join(stacksRoot, stackEntry.name); - if (stackRoot === currentStackRoot) { - return; - } - - const ports = (await readStackMetadataFile(join(stackRoot, "stack.json")))?.ports; - if (ports === undefined) { - return; - } - - for (const field of PORT_FIELDS) { - reserved.add(ports[field]); - } - }), - ); - }), - ); - - return reserved; -} - -async function readReservedPortsInStacksRoot( - stacksRoot: string, - currentStackRoot: string, -): Promise> { - const reserved = new Set(); - - let stackEntries: Array<{ isDirectory(): boolean; name: string }>; - try { - stackEntries = await readdir(stacksRoot, { withFileTypes: true }); - } catch { - return reserved; - } - - await Promise.all( - stackEntries.map(async (stackEntry) => { - if (!stackEntry.isDirectory()) { - return; - } - - const stackRoot = join(stacksRoot, stackEntry.name); - if (stackRoot === currentStackRoot) { - return; - } - - const ports = (await readStackMetadataFile(join(stackRoot, "stack.json")))?.ports; - if (ports === undefined) { - return; - } - - for (const field of PORT_FIELDS) { - reserved.add(ports[field]); - } - }), - ); - - return reserved; -} - -function resolvePostgrestConfig( - input: PostgrestConfig | undefined, - raw: PostgrestConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedPostgrestConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.postgrestPort, - adminPort: ports.postgrestAdminPort, - schemas: cfg.schemas ?? ["public", "graphql_public"], - extraSearchPath: cfg.extraSearchPath ?? ["public", "extensions"], - maxRows: cfg.maxRows ?? 1000, - version: cfg.version ?? DEFAULT_VERSIONS.postgrest, - }; -} - -function resolveAuthConfig( - input: AuthConfig | undefined, - raw: AuthConfig | false | undefined, - ports: AllocatedPorts, - apiPort: number, -): ResolvedAuthConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.authPort, - siteUrl: cfg.siteUrl ?? "http://localhost:3000", - jwtExpiry: cfg.jwtExpiry ?? 3600, - externalUrl: cfg.externalUrl ?? `http://127.0.0.1:${apiPort}`, - version: cfg.version ?? DEFAULT_VERSIONS.auth, - }; -} - -function resolveRealtimeConfig( - input: RealtimeConfig | undefined, - raw: RealtimeConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedRealtimeConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.realtimePort, - version: cfg.version ?? DEFAULT_VERSIONS.realtime, - tenantId: cfg.tenantId ?? "realtime-dev", - encryptionKey: cfg.encryptionKey ?? "supabaserealtime", - secretKeyBase: - cfg.secretKeyBase ?? "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG", - maxHeaderLength: cfg.maxHeaderLength ?? 4096, - }; -} - -function resolveEdgeRuntimeConfig( - input: EdgeRuntimeConfig | undefined, - raw: EdgeRuntimeConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedEdgeRuntimeConfig | false { - if (raw === false || raw?.enabled === false) return false; - const cfg = input ?? {}; - return { - enabled: cfg.enabled ?? true, - port: ports.edgeRuntimePort, - inspectorPort: ports.edgeRuntimeInspectorPort, - policy: cfg.policy ?? "per_worker", - version: cfg.version ?? DEFAULT_VERSIONS["edge-runtime"], - env: cfg.env ?? {}, - }; -} - -function resolveFunctionsConfig(config: StackConfig) { - if (config.functions === false) return false; - return { - envFile: config.functions?.envFile, - noVerifyJwt: config.functions?.noVerifyJwt ?? false, - }; -} - -function resolveStorageConfig( - input: StorageConfig | undefined, - raw: StorageConfig | false | undefined, - ports: AllocatedPorts, - opts: ResolveConfigOptions, -): ResolvedStorageConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.storagePort, - version: cfg.version ?? DEFAULT_VERSIONS.storage, - dataDir: resolveDataDir(cfg.dataDir, opts.stackRoot!, "storage"), - fileSizeLimit: cfg.fileSizeLimit ?? "50MiB", - s3ProtocolEnabled: cfg.s3ProtocolEnabled ?? true, - }; -} - -function resolveImgproxyConfig( - input: ImgproxyConfig | undefined, - raw: ImgproxyConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedImgproxyConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.imgproxyPort, - version: cfg.version ?? DEFAULT_VERSIONS.imgproxy, - }; -} - -function resolveMailpitConfig( - input: MailpitConfig | undefined, - raw: MailpitConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedMailpitConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.mailpitPort, - smtpPort: ports.mailpitSmtpPort, - pop3Port: ports.mailpitPop3Port, - version: cfg.version ?? DEFAULT_VERSIONS.mailpit, - adminEmail: cfg.adminEmail ?? "admin@email.com", - senderName: cfg.senderName ?? "Admin", - }; -} - -function resolvePgmetaConfig( - input: PgmetaConfig | undefined, - raw: PgmetaConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedPgmetaConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.pgmetaPort, - version: cfg.version ?? DEFAULT_VERSIONS.pgmeta, - }; -} - -function resolveStudioConfig( - input: StudioConfig | undefined, - raw: StudioConfig | false | undefined, - ports: AllocatedPorts, - apiPort: number, -): ResolvedStudioConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.studioPort, - version: cfg.version ?? DEFAULT_VERSIONS.studio, - apiUrl: cfg.apiUrl ?? `http://127.0.0.1:${apiPort}`, - }; -} - -function resolveAnalyticsConfig( - input: AnalyticsConfig | undefined, - raw: AnalyticsConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedAnalyticsConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.analyticsPort, - version: cfg.version ?? DEFAULT_VERSIONS.analytics, - backend: cfg.backend ?? "postgres", - apiKey: cfg.apiKey ?? "api-key", - }; -} - -function resolveVectorConfig( - input: VectorConfig | undefined, - raw: VectorConfig | false | undefined, -): ResolvedVectorConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - version: cfg.version ?? DEFAULT_VERSIONS.vector, - }; -} - -function resolvePoolerConfig( - input: PoolerConfig | undefined, - raw: PoolerConfig | false | undefined, - ports: AllocatedPorts, -): ResolvedPoolerConfig | false { - if (raw === false) return false; - const cfg = input ?? {}; - return { - port: ports.poolerPort, - apiPort: ports.poolerApiPort, - mode: cfg.mode ?? "transaction", - version: cfg.version ?? DEFAULT_VERSIONS.pooler, - tenantId: cfg.tenantId ?? "pooler-dev", - encryptionKey: cfg.encryptionKey ?? "12345678901234567890123456789032", - secretKeyBase: - cfg.secretKeyBase ?? "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG", - defaultPoolSize: cfg.defaultPoolSize ?? 20, - maxClientConn: cfg.maxClientConn ?? 100, - }; -} - -export async function resolveConfig( - input?: StackConfig, - opts: ResolveConfigOptions = {}, -): Promise { - const config = input ?? {}; - const projectDir = config.projectDir ?? process.cwd(); - const resolvedMode = config.mode ?? "auto"; - const roots = resolveRoots(config, opts); - const postgresInput = config.postgres ?? {}; - const postgrestInput = config.postgrest !== false ? (config.postgrest ?? undefined) : undefined; - const authInput = config.auth !== false ? (config.auth ?? undefined) : undefined; - const edgeRuntimeEnabled = - !(resolvedMode === "native" && config.edgeRuntime === undefined) && - config.edgeRuntime !== false && - (config.edgeRuntime?.enabled ?? true) !== false; - const realtimeEnabled = config.realtime !== undefined && config.realtime !== false; - const storageEnabled = config.storage !== undefined && config.storage !== false; - const imgproxyEnabled = config.imgproxy !== undefined && config.imgproxy !== false; - const mailpitEnabled = config.mailpit !== undefined && config.mailpit !== false; - const pgmetaEnabled = config.pgmeta !== undefined && config.pgmeta !== false; - const studioEnabled = config.studio !== undefined && config.studio !== false; - const analyticsEnabled = config.analytics !== undefined && config.analytics !== false; - const vectorEnabled = config.vector !== undefined && config.vector !== false; - const poolerEnabled = config.pooler !== undefined && config.pooler !== false; - const edgeRuntimeInput = edgeRuntimeEnabled ? (config.edgeRuntime ?? undefined) : undefined; - const realtimeInput = realtimeEnabled ? (config.realtime ?? undefined) : undefined; - const storageInput = storageEnabled ? (config.storage ?? undefined) : undefined; - const imgproxyInput = imgproxyEnabled ? (config.imgproxy ?? undefined) : undefined; - const mailpitInput = mailpitEnabled ? (config.mailpit ?? undefined) : undefined; - const pgmetaInput = pgmetaEnabled ? (config.pgmeta ?? undefined) : undefined; - const studioInput = studioEnabled ? (config.studio ?? undefined) : undefined; - const analyticsInput = analyticsEnabled ? (config.analytics ?? undefined) : undefined; - const vectorInput = vectorEnabled ? (config.vector ?? undefined) : undefined; - const poolerInput = poolerEnabled ? (config.pooler ?? undefined) : undefined; - - const postgresDataDir = resolveDataDir(postgresInput.dataDir, roots.stackRoot, "postgres"); - - const ports = await Effect.runPromise( - (opts.portAllocator ?? allocatePorts)( - { - apiPort: config.port, - dbPort: postgresInput.port, - authPort: authInput?.port, - postgrestPort: undefined, - postgrestAdminPort: undefined, - edgeRuntimePort: edgeRuntimeInput?.port, - edgeRuntimeInspectorPort: edgeRuntimeInput?.inspectorPort, - realtimePort: realtimeInput?.port, - storagePort: storageInput?.port, - imgproxyPort: imgproxyInput?.port, - mailpitPort: mailpitInput?.port, - mailpitSmtpPort: mailpitInput?.smtpPort, - mailpitPop3Port: mailpitInput?.pop3Port, - pgmetaPort: pgmetaInput?.port, - studioPort: studioInput?.port, - analyticsPort: analyticsInput?.port, - poolerPort: poolerInput?.port, - poolerApiPort: poolerInput?.apiPort, - }, - { - preferred: opts.preferredPorts, - reserved: opts.reservedPorts, - }, - ), - ).catch((error: unknown) => { - throw toStackError(error); - }); - - const jwtSecret = config.jwtSecret ?? defaultJwtSecret; - const anonJwt = generateJwt(jwtSecret, "anon"); - const serviceRoleJwt = generateJwt(jwtSecret, "service_role"); - - return { - cacheRoot: roots.cacheRoot, - stackRoot: roots.stackRoot, - runtimeRoot: roots.runtimeRoot, - projectDir, - mode: resolvedMode, - startupMode: config.startupMode ?? "eager", - jwtSecret, - ports, - apiPort: ports.apiPort, - dbPort: ports.dbPort, - publishableKey: config.publishableKey ?? defaultPublishableKey, - secretKey: config.secretKey ?? defaultSecretKey, - functions: resolveFunctionsConfig(config), - autoManagedPaths: roots.autoManagedPaths, - anonJwt, - serviceRoleJwt, - postgres: { - port: ports.dbPort, - dataDir: postgresDataDir, - version: postgresInput.version ?? DEFAULT_VERSIONS.postgres, - autoExposeNewTables: postgresInput.autoExposeNewTables ?? true, - }, - postgrest: resolvePostgrestConfig(postgrestInput, config.postgrest, ports), - auth: resolveAuthConfig(authInput, config.auth, ports, ports.apiPort), - edgeRuntime: edgeRuntimeEnabled - ? resolveEdgeRuntimeConfig(edgeRuntimeInput, config.edgeRuntime, ports) - : false, - realtime: realtimeEnabled - ? resolveRealtimeConfig(realtimeInput, config.realtime, ports) - : false, - storage: storageEnabled - ? resolveStorageConfig(storageInput, config.storage, ports, { - ...opts, - stackRoot: roots.stackRoot, - }) - : false, - imgproxy: imgproxyEnabled - ? resolveImgproxyConfig(imgproxyInput, config.imgproxy, ports) - : false, - mailpit: mailpitEnabled ? resolveMailpitConfig(mailpitInput, config.mailpit, ports) : false, - pgmeta: pgmetaEnabled ? resolvePgmetaConfig(pgmetaInput, config.pgmeta, ports) : false, - studio: studioEnabled - ? resolveStudioConfig(studioInput, config.studio, ports, ports.apiPort) - : false, - analytics: analyticsEnabled - ? resolveAnalyticsConfig(analyticsInput, config.analytics, ports) - : false, - vector: vectorEnabled ? resolveVectorConfig(vectorInput, config.vector) : false, - pooler: poolerEnabled ? resolvePoolerConfig(poolerInput, config.pooler, ports) : false, - }; -} - -export type DaemonConfigInput = StackConfig & { - readonly cwd: string; - readonly name?: string; - readonly projectDir?: string; - readonly projectStateRoot?: string; -}; - -export async function resolveDaemonConfig( - input: DaemonConfigInput, - opts: Pick = {}, -): Promise { - const { cwd, name, projectDir, projectStateRoot, ...stackConfig } = input; - if (stackConfig.stackRoot !== undefined || stackConfig.runtimeRoot !== undefined) { - throw new Error("Managed daemon stacks derive stackRoot and runtimeRoot automatically"); - } - const effectiveProjectDir = projectDir ?? cwd; - const resolvedName = name ?? defaultManagedStackName(effectiveProjectDir); - const cacheRoot = stackConfig.cacheRoot ?? defaultCacheRoot(); - const stackRoot = - projectStateRoot !== undefined - ? join(projectStateRoot, "stacks", resolvedName) - : defaultManagedStackRoot(cacheRoot, effectiveProjectDir, resolvedName); - const runtimeRoot = defaultManagedRuntimeRoot(stackRoot); - const savedPorts = await readOwnedPorts(stackRoot); - const reservedPortSets = await Promise.all([ - readReservedPorts(defaultManagedProjectsRoot(cacheRoot), stackRoot), - projectStateRoot === undefined - ? Promise.resolve>(new Set()) - : readReservedPortsInStacksRoot(join(projectStateRoot, "stacks"), stackRoot), - ]); - const reservedPorts = new Set(); - for (const ports of reservedPortSets) { - for (const port of ports) { - reservedPorts.add(port); - } - } - const resolved = await resolveConfig( - { - ...stackConfig, - cacheRoot, - stackRoot, - runtimeRoot, - projectDir: effectiveProjectDir, - }, - { - stackRoot, - runtimeRoot, - preferredPorts: savedPorts ?? DEFAULT_PORTS, - reservedPorts, - portAllocator: opts.portAllocator, - }, - ); - return { - ...resolved, - name: resolvedName, - projectDir: effectiveProjectDir, - }; -} - export const projectDaemonLayer = (opts: { readonly cacheRoot: string; readonly cwd: string; @@ -784,16 +169,24 @@ export async function createStack( reloadFunctions: (opts) => run(localStack.reloadFunctions(opts)), reloadEdgeRuntime: (opts) => run(localStack.reloadEdgeRuntime(opts)), ready: (opts) => { + const policy = resolveReadinessPolicy({ + readyOptions: opts, + stackPolicy: resolved.readiness, + }); const effect = - opts?.timeout != null - ? localStack.waitAllReady().pipe(Effect.timeout(Duration.millis(opts.timeout))) + policy.mode === "finite" + ? localStack.waitAllReady().pipe(Effect.timeout(Duration.millis(policy.timeoutMs))) : localStack.waitAllReady(); return run(effect); }, serviceReady: (name, opts) => { + const policy = resolveReadinessPolicy({ + readyOptions: opts, + stackPolicy: resolved.readiness, + }); const effect = - opts?.timeout != null - ? localStack.waitReady(name).pipe(Effect.timeout(Duration.millis(opts.timeout))) + policy.mode === "finite" + ? localStack.waitReady(name).pipe(Effect.timeout(Duration.millis(policy.timeoutMs))) : localStack.waitReady(name); return run(effect); }, diff --git a/packages/stack/src/createStack.unit.test.ts b/packages/stack/src/createStack.unit.test.ts index 68caf179c1..747b16686b 100644 --- a/packages/stack/src/createStack.unit.test.ts +++ b/packages/stack/src/createStack.unit.test.ts @@ -2,12 +2,18 @@ import { describe, expect, it } from "vitest"; import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import type { ReadyOptions, StackHandle } from "./createStack.ts"; -import { resolveConfig, resolveDaemonConfig } from "./createStack.ts"; +import type { StackHandle } from "./createStack.ts"; import type { AllocatedPorts } from "./PortAllocator.ts"; import { DEFAULT_MANAGED_STACK_NAME, projectKeyForProjectDir } from "./paths.ts"; import { stackMetadata } from "./StackMetadata.ts"; -import type { AuthConfig, PostgresConfig, PostgrestConfig, StackConfig } from "./StackBuilder.ts"; +import type { + AuthConfig, + PostgresConfig, + PostgrestConfig, + ReadyOptions, + StackConfig, +} from "./StackConfig.ts"; +import { resolveConfig, resolveDaemonConfig } from "./StackConfigResolver.ts"; import { DEFAULT_VERSIONS } from "./versions.ts"; const DEFAULT_PORTS: AllocatedPorts = { @@ -251,3 +257,15 @@ describe("resolveConfig startup mode", () => { expect(config.startupMode).toBe("lazy"); }); }); + +describe("resolveConfig readiness policy", () => { + it("uses a finite package default", async () => { + const config = await resolveConfig(); + expect(config.readiness).toEqual({ mode: "finite", timeoutMs: 120_000 }); + }); + + it("preserves an explicit infinite policy", async () => { + const config = await resolveConfig({ readiness: { mode: "infinite" } }); + expect(config.readiness).toEqual({ mode: "infinite" }); + }); +}); diff --git a/packages/stack/src/daemon.ts b/packages/stack/src/daemon.ts index 1171d634c5..30361ecb92 100644 --- a/packages/stack/src/daemon.ts +++ b/packages/stack/src/daemon.ts @@ -1,16 +1,13 @@ import { Effect, Layer, ManagedRuntime } from "effect"; import { HttpServer } from "effect/unstable/http"; -import { - resolveDaemonConfig, - type DaemonConfigInput, - type PlatformFactory, -} from "./createStack.ts"; +import type { PlatformFactory } from "./createStack.ts"; import { DaemonServer } from "./DaemonServer.ts"; import { PORT_FIELDS, reservePorts, type PortLease } from "./PortAllocator.ts"; import { allocatedPortFieldsForConfig } from "./ServicePorts.ts"; import { runningServiceVersionsForConfig } from "./StackMetadata.ts"; import { foregroundDaemonLayer } from "./layers.ts"; import { Stack } from "./Stack.ts"; +import { resolveDaemonConfig, type DaemonConfigInput } from "./StackConfigResolver.ts"; import { StateManager, type StackState } from "./StateManager.ts"; /** Factory for creating the daemon's Unix socket HTTP server (platform-specific). */ diff --git a/packages/stack/src/discovery.ts b/packages/stack/src/discovery.ts index 18de19f60f..314ec2d5ba 100644 --- a/packages/stack/src/discovery.ts +++ b/packages/stack/src/discovery.ts @@ -1,7 +1,7 @@ import { Data, Duration, Effect } from "effect"; import { FileSystem, Path } from "effect"; import { dockerForceRemove } from "./cleanup.ts"; -import { defaultManagedStackName } from "./createStack.ts"; +import { defaultManagedStackName } from "./StackConfigResolver.ts"; import { InvalidStackMetadataError, InvalidStackStateError, diff --git a/packages/stack/src/effect.ts b/packages/stack/src/effect.ts index f3efd79c4e..215e6b6892 100644 --- a/packages/stack/src/effect.ts +++ b/packages/stack/src/effect.ts @@ -27,8 +27,7 @@ export { export type { BinarySpec } from "./BinaryResolver.ts"; export { BinaryResolver } from "./BinaryResolver.ts"; -export type { ServiceResolution } from "./resolve.ts"; -export { resolveService } from "./resolve.ts"; +export type { ServiceResolution } from "./StackPreparation.ts"; export type { PrefetchOptions, PrefetchResult } from "./prefetch.ts"; export { prefetch } from "./prefetch.ts"; @@ -84,11 +83,14 @@ export type { ResolvedStorageConfig, ResolvedStudioConfig, ResolvedVectorConfig, + ReadinessPolicy, + ReadyOptions, StackConfig, StorageConfig, StudioConfig, VectorConfig, -} from "./StackBuilder.ts"; +} from "./StackConfig.ts"; +export { DEFAULT_STACK_READINESS_POLICY, resolveReadinessPolicy } from "./StackConfig.ts"; export { StackBuilder } from "./StackBuilder.ts"; export type { EdgeRuntimeReloadConfig, StackInfo } from "./Stack.ts"; @@ -164,18 +166,16 @@ export type { PlatformFactoryOptions, PlatformLayer, PlatformServices, - ReadyOptions, StackHandle, } from "./createStack.ts"; +export { createStack, projectDaemonLayer } from "./createStack.ts"; +export type { ResolvedDaemonConfig } from "./StackConfig.ts"; export { - createStack, defaultManagedStackName, - projectDaemonLayer, resolveConfig, resolveDaemonConfig, -} from "./createStack.ts"; +} from "./StackConfigResolver.ts"; -export type { DaemonConfig } from "./layers.ts"; export { connectLayer, DaemonStartError, daemonLayer, foregroundLayer } from "./layers.ts"; export type { ManagedStack } from "./managed-stack.ts"; export { resolveManagedStack } from "./managed-stack.ts"; diff --git a/packages/stack/src/functions.ts b/packages/stack/src/functions.ts index 527a163b65..fd95607f93 100644 --- a/packages/stack/src/functions.ts +++ b/packages/stack/src/functions.ts @@ -8,7 +8,7 @@ import { type ResolvedFunctionConfig, } from "@supabase/config"; import { Effect, FileSystem, Path, Redacted } from "effect"; -import type { ResolvedStackConfig } from "./StackBuilder.ts"; +import type { ResolvedStackConfig } from "./StackConfig.ts"; export interface FunctionsConfig { readonly envFile?: string; diff --git a/packages/stack/src/functions.unit.test.ts b/packages/stack/src/functions.unit.test.ts index 56ead117fb..e537ea9a90 100644 --- a/packages/stack/src/functions.unit.test.ts +++ b/packages/stack/src/functions.unit.test.ts @@ -5,7 +5,7 @@ import { mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { Effect } from "effect"; -import { resolveConfig } from "./createStack.ts"; +import { resolveConfig } from "./StackConfigResolver.ts"; import { defaultJwtSecret, generateJwt } from "./JwtGenerator.ts"; import { configureFunctionsRuntime, diff --git a/packages/stack/src/index.ts b/packages/stack/src/index.ts index 087b40503e..8e990d0640 100644 --- a/packages/stack/src/index.ts +++ b/packages/stack/src/index.ts @@ -14,15 +14,17 @@ export type { PostgresConfig, PostgrestConfig, RealtimeConfig, + ReadinessPolicy, + ReadyOptions, StackConfig, StorageConfig, StudioConfig, VectorConfig, -} from "./StackBuilder.ts"; +} from "./StackConfig.ts"; export type { ServiceName, VersionManifest } from "./versions.ts"; -export type { ServiceResolution } from "./resolve.ts"; +export type { ServiceResolution } from "./StackPreparation.ts"; export type { PrefetchOptions, PrefetchResult } from "./prefetch.ts"; -export type { ReadyOptions, StackHandle } from "./createStack.ts"; +export type { StackHandle } from "./createStack.ts"; export type { FunctionsConfig, FunctionsRuntimeConfig } from "./functions.ts"; export { UnixHttpClient, UnixHttpClientError } from "./UnixHttpClient.ts"; diff --git a/packages/stack/src/layers.ts b/packages/stack/src/layers.ts index 34f3ea0413..30fb1c3eb4 100644 --- a/packages/stack/src/layers.ts +++ b/packages/stack/src/layers.ts @@ -6,7 +6,7 @@ import { FileSystem, Path } from "effect"; import { FetchHttpClient } from "effect/unstable/http"; import { ApiProxy, type ProxyConfig } from "./ApiProxy.ts"; import { BinaryResolver } from "./BinaryResolver.ts"; -import type { DaemonConfigInput, PlatformFactory } from "./createStack.ts"; +import type { PlatformFactory } from "./createStack.ts"; import type { DaemonMessage, DaemonStartMessage } from "./daemon.ts"; import { DaemonMessageSchema } from "./DaemonProtocol.ts"; import type { PortLease } from "./PortAllocator.ts"; @@ -23,7 +23,9 @@ import { StateManager, singleStackStateManagerPaths, } from "./StateManager.ts"; -import { StackBuilder, type ResolvedStackConfig } from "./StackBuilder.ts"; +import { StackBuilder } from "./StackBuilder.ts"; +import type { ResolvedDaemonConfig, ResolvedStackConfig } from "./StackConfig.ts"; +import type { DaemonConfigInput } from "./StackConfigResolver.ts"; import { UnixHttpClient } from "./UnixHttpClient.ts"; import { resolveManagedStack } from "./managed-stack.ts"; import { @@ -108,13 +110,8 @@ export class DaemonStartError extends Data.TaggedError("DaemonStartError")<{ // Daemon-backed mode // --------------------------------------------------------------------------- -export interface DaemonConfig extends ResolvedStackConfig { - readonly name: string; - readonly projectDir: string; -} - export const foregroundDaemonLayer = ( - config: DaemonConfig, + config: ResolvedDaemonConfig, platformFactory: PlatformFactory, portLease: PortLease, ): Layer.Layer => { diff --git a/packages/stack/src/node.ts b/packages/stack/src/node.ts index 8feb036db6..aedf7ab1f1 100644 --- a/packages/stack/src/node.ts +++ b/packages/stack/src/node.ts @@ -19,7 +19,7 @@ import { } from "./prefetch.ts"; import { defaultCacheRoot } from "./paths.ts"; import { StackPreparation } from "./StackPreparation.ts"; -import type { StackConfig } from "./StackBuilder.ts"; +import type { StackConfig } from "./StackConfig.ts"; import { UnixHttpClient, UnixHttpClientError } from "./UnixHttpClient.ts"; const mergeBodyHeaders = ( diff --git a/packages/stack/src/prefetch.ts b/packages/stack/src/prefetch.ts index f086bd5102..b68d4b5f4d 100644 --- a/packages/stack/src/prefetch.ts +++ b/packages/stack/src/prefetch.ts @@ -1,16 +1,20 @@ import { Effect } from "effect"; import type { ChecksumMismatchError } from "./errors.ts"; import type { DockerPullError } from "./errors.ts"; -import { type PreparedStackArtifacts, type StackPreparationInput } from "./StackPreparation.ts"; +import { + type PreparedStackArtifacts, + type ServiceResolution, + type StackPreparationInput, +} from "./StackPreparation.ts"; import { StackPreparation } from "./StackPreparation.ts"; -import type { ServiceResolution } from "./resolve.ts"; +import type { ServiceName } from "./ServiceName.ts"; export interface PrefetchOptions extends StackPreparationInput {} -export type PrefetchResult = Record; +export type PrefetchResult = Partial>; const toPrefetchResult = (artifacts: PreparedStackArtifacts): PrefetchResult => - artifacts.resolutions as PrefetchResult; + artifacts.resolutions; export const prefetch = ( options?: PrefetchOptions, diff --git a/packages/stack/src/resolve.ts b/packages/stack/src/resolve.ts deleted file mode 100644 index 813a7448f7..0000000000 --- a/packages/stack/src/resolve.ts +++ /dev/null @@ -1,35 +0,0 @@ -import { Effect } from "effect"; -import type { BinaryResolver } from "./BinaryResolver.ts"; -import type { ChecksumMismatchError } from "./errors.ts"; -import type { ServiceName } from "./versions.ts"; -import { dockerImageForService } from "./versions.ts"; - -export type ServiceResolution = - | { readonly type: "binary"; readonly path: string } - | { readonly type: "docker"; readonly image: string }; - -/** - * Resolve a service to either a native binary path or a Docker image. - * Tries BinaryResolver first; falls back to Docker on BinaryNotFoundError or DownloadError. - * ChecksumMismatchError is a real error and propagates. - */ -export const resolveService = ( - resolver: BinaryResolver["Service"], - service: ServiceName, - version: string, -): Effect.Effect => - resolver.resolve({ service, version }).pipe( - Effect.map((path): ServiceResolution => ({ type: "binary", path })), - Effect.catchTag("BinaryNotFoundError", () => - Effect.succeed({ - type: "docker", - image: dockerImageForService(service, version), - }), - ), - Effect.catchTag("DownloadError", () => - Effect.succeed({ - type: "docker", - image: dockerImageForService(service, version), - }), - ), - ); diff --git a/packages/stack/src/services/docker-cleanup.ts b/packages/stack/src/services/docker-cleanup.ts index b3673a1b61..c96c7af6b9 100644 --- a/packages/stack/src/services/docker-cleanup.ts +++ b/packages/stack/src/services/docker-cleanup.ts @@ -14,7 +14,14 @@ export const dockerServiceCleanup = (containerName: string): Effect.Effect export const dockerServiceOrphanCleanup = ( containerName: string, -): ReadonlyArray => [{ _tag: "DockerRemove", containerName }]; +): ReadonlyArray => [ + { + _tag: "RunCommand", + executable: "docker", + args: ["rm", "-f", containerName], + timeoutMs: 5_000, + }, +]; export const removePathOnOrphanCleanup = ( path: string, diff --git a/packages/stack/src/services/service-utils.ts b/packages/stack/src/services/service-utils.ts index eabb31cd5e..4259ed02f7 100644 --- a/packages/stack/src/services/service-utils.ts +++ b/packages/stack/src/services/service-utils.ts @@ -1,4 +1,4 @@ -import type { ServiceDef } from "@supabase/process-compose"; +import type { ExternalCleanupAction, ServiceDef } from "@supabase/process-compose"; import { dockerServiceCleanup, dockerServiceOrphanCleanup } from "./docker-cleanup.ts"; export interface ServiceDependency { @@ -20,7 +20,7 @@ interface DockerRunServiceOptions { readonly healthCheck?: ServiceDef["healthCheck"]; readonly restart?: ServiceDef["restart"]; readonly shutdown?: ServiceDef["shutdown"]; - readonly orphanCleanup?: ReadonlyArray; + readonly orphanCleanup?: ReadonlyArray; } const envArgs = (env: Record): ReadonlyArray => diff --git a/packages/stack/src/services/services.unit.test.ts b/packages/stack/src/services/services.unit.test.ts index c777062c83..7c4d50c1a1 100644 --- a/packages/stack/src/services/services.unit.test.ts +++ b/packages/stack/src/services/services.unit.test.ts @@ -200,7 +200,14 @@ describe("makePostgresServiceDocker", () => { expect(def.dependencies).toBeUndefined(); expect(def.restart).toBe("unless-stopped"); expect(def.supervision).toEqual({ - orphanCleanup: [{ _tag: "DockerRemove", containerName: `supabase-postgres-${API_PORT}` }], + orphanCleanup: [ + { + _tag: "RunCommand", + executable: "docker", + args: ["rm", "-f", `supabase-postgres-${API_PORT}`], + timeoutMs: 5_000, + }, + ], }); }); @@ -313,7 +320,14 @@ describe("makeAuthServiceDocker", () => { expect(def.args).toContain("9999:9999"); expect(def.dependencies).toEqual([{ service: "postgres", condition: "healthy" }]); expect(def.supervision).toEqual({ - orphanCleanup: [{ _tag: "DockerRemove", containerName: `supabase-auth-${API_PORT}` }], + orphanCleanup: [ + { + _tag: "RunCommand", + executable: "docker", + args: ["rm", "-f", `supabase-auth-${API_PORT}`], + timeoutMs: 5_000, + }, + ], }); }); }); diff --git a/packages/stack/src/versions.ts b/packages/stack/src/versions.ts index 6a9c4a1a9c..9e6295fac0 100644 --- a/packages/stack/src/versions.ts +++ b/packages/stack/src/versions.ts @@ -1,71 +1,16 @@ import { + DEFAULT_VERSIONS, + SERVICE_NAMES, dockerImageCandidatesForArtifact, dockerImageForArtifact, imageTagPrefixForService, -} from "./ServiceArtifacts.ts"; +} from "./ServiceCatalog.ts"; +import type { ServiceName } from "./ServiceName.ts"; -export type ServiceName = - | "postgres" - | "postgrest" - | "auth" - | "edge-runtime" - | "realtime" - | "storage" - | "imgproxy" - | "mailpit" - | "pgmeta" - | "studio" - | "analytics" - | "vector" - | "pooler"; +export { DEFAULT_VERSIONS, SERVICE_NAMES } from "./ServiceCatalog.ts"; +export type { ServiceName } from "./ServiceName.ts"; -export const SERVICE_NAMES = [ - "postgres", - "postgrest", - "auth", - "edge-runtime", - "realtime", - "storage", - "imgproxy", - "mailpit", - "pgmeta", - "studio", - "analytics", - "vector", - "pooler", -] as const satisfies ReadonlyArray; - -export interface VersionManifest { - readonly postgres: string; - readonly postgrest: string; - readonly auth: string; - readonly "edge-runtime": string; - readonly realtime: string; - readonly storage: string; - readonly imgproxy: string; - readonly mailpit: string; - readonly pgmeta: string; - readonly studio: string; - readonly analytics: string; - readonly vector: string; - readonly pooler: string; -} - -export const DEFAULT_VERSIONS: VersionManifest = { - postgres: "17.6.1.158", - postgrest: "14.16", - auth: "2.195.0", - "edge-runtime": "1.74.3", - realtime: "2.123.1", - storage: "1.68.1", - imgproxy: "v3.8.0", - mailpit: "v1.30.2", - pgmeta: "0.96.6", - studio: "2026.08.03-sha-022b374", - analytics: "1.49.2", - vector: "0.53.0-alpine", - pooler: "2.9.7", -} as const; +export type VersionManifest = Readonly>; export const IMAGE_TAG_PREFIX: Partial> = Object.fromEntries( SERVICE_NAMES.flatMap((service) => { diff --git a/packages/stack/src/versions.unit.test.ts b/packages/stack/src/versions.unit.test.ts index 4b31249dbd..bd5e584c3f 100644 --- a/packages/stack/src/versions.unit.test.ts +++ b/packages/stack/src/versions.unit.test.ts @@ -13,7 +13,7 @@ import { SERVICE_NAMES, type VersionManifest, } from "./versions.ts"; -import { SERVICE_ARTIFACTS } from "./ServiceArtifacts.ts"; +import { SERVICE_CATALOG } from "./ServiceCatalog.ts"; const sampleDockerfile = ` FROM supabase/postgres:17.0.0.1 AS pg @@ -35,44 +35,29 @@ FROM supabase/migra:3.0.1663481299 AS migra describe("syncDefaultVersionsSource", () => { it("rewrites the DEFAULT_VERSIONS block from Dockerfile versions", () => { - const source = `before -export const DEFAULT_VERSIONS: VersionManifest = { - postgres: "old", - postgrest: "old", - auth: "old", - "edge-runtime": "old", - realtime: "old", - storage: "old", - imgproxy: "old", - mailpit: "old", - pgmeta: "old", - studio: "old", - analytics: "old", - vector: "old", - pooler: "old", -} as const; -after`; - - expect(syncDefaultVersionsSource(source, readVersionManifestFromDockerfile(sampleDockerfile))) - .toMatchInlineSnapshot(` - "before - export const DEFAULT_VERSIONS: VersionManifest = { - postgres: "17.0.0.1", - postgrest: "14.0", - auth: "2.100.0", - "edge-runtime": "1.70.0", - realtime: "2.100.0", - storage: "1.50.0", - imgproxy: "v3.8.0", - mailpit: "v1.2.3", - pgmeta: "0.90.0", - studio: "2026.01.01-sha-abcdef0", - analytics: "1.40.0", - vector: "0.50.0-alpine", - pooler: "2.1.0", - } as const; - after" - `); + const source = SERVICE_NAMES.map( + (service) => ` ${JSON.stringify(service)}: { + name: ${JSON.stringify(service)}, + configKey: "example", + defaultVersion: "old", + },`, + ).join("\n"); + + const updated = syncDefaultVersionsSource( + source, + readVersionManifestFromDockerfile(sampleDockerfile), + ); + + expect(updated).toContain( + 'name: "postgres",\n configKey: "example",\n defaultVersion: "17.0.0.1"', + ); + expect(updated).toContain( + 'name: "edge-runtime",\n configKey: "example",\n defaultVersion: "1.70.0"', + ); + expect(updated).toContain( + 'name: "mailpit",\n configKey: "example",\n defaultVersion: "v1.2.3"', + ); + expect(updated).not.toContain('defaultVersion: "old"'); }); it("fails when a required Dockerfile image alias is missing", () => { @@ -92,7 +77,7 @@ after`; describe("dockerImageForService", () => { it("defines artifact capabilities for every stack service", () => { - expect(Object.keys(SERVICE_ARTIFACTS).sort()).toEqual([...SERVICE_NAMES].sort()); + expect(Object.keys(SERVICE_CATALOG).sort()).toEqual([...SERVICE_NAMES].sort()); }); it("returns correct image for postgres", () => { @@ -134,17 +119,17 @@ describe("dockerImageForService", () => { }); it("keeps non-managed services Docker-only", () => { - expect(SERVICE_ARTIFACTS.imgproxy).toMatchObject({ + expect(SERVICE_CATALOG.imgproxy).toMatchObject({ runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "darthsim/imgproxy" }, + artifact: { docker: { ownership: "upstream", repository: "darthsim/imgproxy" } }, }); - expect(SERVICE_ARTIFACTS.mailpit).toMatchObject({ + expect(SERVICE_CATALOG.mailpit).toMatchObject({ runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "axllent/mailpit" }, + artifact: { docker: { ownership: "upstream", repository: "axllent/mailpit" } }, }); - expect(SERVICE_ARTIFACTS.vector).toMatchObject({ + expect(SERVICE_CATALOG.vector).toMatchObject({ runtimeSupport: "docker-only", - docker: { ownership: "upstream", repository: "timberio/vector" }, + artifact: { docker: { ownership: "upstream", repository: "timberio/vector" } }, }); }); });