Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 27 additions & 22 deletions packages/stack/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Manages a local Supabase development stack — resolving native binaries, wiring
- [errors — typed error hierarchy](#errors--typed-error-hierarchy)
- [Platform — OS and architecture detection](#platform--os-and-architecture-detection)
- [BinaryResolver — download and cache binaries](#binaryresolver--download-and-cache-binaries)
- [resolveService — binary-first Docker fallback](#resolveservice--binary-first-docker-fallback)
- [resolveService — binary-first, mode-aware Docker fallback](#resolveservice--binary-first-mode-aware-docker-fallback)
- [JwtGenerator — JWT token generation and opaque keys](#jwtgenerator--jwt-token-generation-and-opaque-keys)
- [PortAllocator — dynamic port assignment](#portallocator--dynamic-port-assignment)
- [prefetch — pre-download binaries and images](#prefetch--pre-download-binaries-and-images)
Expand Down Expand Up @@ -326,11 +326,11 @@ The download is written to a temporary file (`_download.tar` or `_download.zip`)

---

### resolveService — binary-first Docker fallback
### resolveService — binary-first, mode-aware Docker fallback
Comment thread
7ttp marked this conversation as resolved.

**File:** `src/resolve.ts`

`resolveService` is a thin helper that wraps `BinaryResolver.resolve()` and implements the binary-first, Docker-fallback strategy used by `StackPreparation` and therefore shared by both `stack.start()` and `prefetch()`.
`resolveService` is a thin helper that wraps `BinaryResolver.resolve()` and implements the binary-first resolution strategy used by `StackPreparation` and therefore shared by both `stack.start()` and `prefetch()`. The Docker fallback is **mode-aware**: it only applies in `"auto"` mode — `"native"` mode propagates resolution failures instead of silently flipping the service onto Docker (supabase/cli#5787).

#### ServiceResolution type

Expand All @@ -344,34 +344,39 @@ This discriminated union is the canonical output of resolution: downstream code

#### Resolution logic

`resolveService(resolver, service, version)` calls `resolver.resolve({ service, version })` and maps the result:
`resolveService(resolver, service, version, mode = "auto")` calls `resolver.resolve({ service, version })` and maps the result:

- **Success** (binary found and extracted) → `{ type: "binary", path }`.
- **`BinaryNotFoundError`** (no native asset for this OS/arch) → `{ type: "docker", image }` using the default Docker image for the service and version.
- **`DownloadError`** (network or extraction failure) → `{ type: "docker", image }` — falls back to Docker rather than hard-failing.
- **`ChecksumMismatchError`** → propagates as a real error; a tampered or corrupted download is never silently replaced by Docker.
- **`BinaryNotFoundError`** (no native asset for this OS/arch) → in `"auto"` mode, `{ type: "docker", image }` using the default Docker image for the service and version; in `"native"` mode the error propagates.
- **`DownloadError`** (network or extraction failure) → in `"auto"` mode, `{ type: "docker", image }` rather than hard-failing; in `"native"` mode the error propagates.
- **`ChecksumMismatchError`** → propagates as a real error in every mode; a tampered or corrupted download is never silently replaced by Docker.

```ts
export const resolveService = (
resolver: BinaryResolver["Service"],
service: ServiceName,
version: string,
): Effect.Effect<ServiceResolution, ChecksumMismatchError> =>
resolver.resolve({ service, version }).pipe(
Effect.map((path): ServiceResolution => ({ type: "binary", path })),
Effect.catchTag("BinaryNotFoundError", () =>
Effect.succeed<ServiceResolution>({
type: "docker",
image: dockerImageForService(service, version),
}),
),
Effect.catchTag("DownloadError", () =>
Effect.succeed<ServiceResolution>({
type: "docker",
image: dockerImageForService(service, version),
}),
),
mode: "native" | "auto" = "auto",
): Effect.Effect<
ServiceResolution,
ChecksumMismatchError | BinaryNotFoundError | DownloadError
> => {
const nativeBinary = resolver
.resolve({ service, version })
.pipe(Effect.map((path): ServiceResolution => ({ type: "binary", path })));
if (mode === "native") {
return nativeBinary;
}
const dockerFallback = () =>
Effect.succeed<ServiceResolution>({
type: "docker",
image: dockerImageForService(service, version),
});
return nativeBinary.pipe(
Effect.catchTag("BinaryNotFoundError", dockerFallback),
Effect.catchTag("DownloadError", dockerFallback),
);
};
```

---
Expand Down
4 changes: 3 additions & 1 deletion packages/stack/src/StackLifecycleCoordinator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,9 @@ export class StackLifecycleCoordinator extends Context.Service<
Stream.mapError(
(cause) =>
new StackBuildError({
detail: "Failed to prepare stack assets",
// Carry the underlying failure text (e.g. which native binary
// is missing) — `.cause` alone never reaches CLI output.
detail: `Failed to prepare stack assets: ${cause.message}`,
cause,
}),
),
Expand Down
79 changes: 44 additions & 35 deletions packages/stack/src/StackPreparation.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
import { Cause, Data, Effect, Exit, Layer, Queue, Context, Stream } from "effect";
import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process";
import { BinaryResolver } from "./BinaryResolver.ts";
import type { ChecksumMismatchError } from "./errors.ts";
import type { BinaryNotFoundError, ChecksumMismatchError, DownloadError } from "./errors.ts";
import { DockerPullError } from "./errors.ts";
import type { ServiceResolution } from "./resolve.ts";
import {
DEFAULT_VERSIONS,
SERVICE_NAMES,
dockerImageCandidatesForService,
dockerImageForService,
type ServiceName,
type VersionManifest,
} from "./versions.ts";
Expand All @@ -22,6 +23,13 @@ export interface StackPreparationInput {
readonly mode?: "native" | "auto" | "docker";
}

/** `BinaryNotFoundError`/`DownloadError` surface only in `mode: "native"` (no Docker fallback there). */
export type StackPreparationError =
| DockerPullError
| ChecksumMismatchError
| BinaryNotFoundError
| DownloadError;

export class ServiceDownloadStarted extends Data.TaggedClass("ServiceDownloadStarted")<{
readonly service: ServiceName;
}> {}
Expand Down Expand Up @@ -86,17 +94,15 @@ export const prepareAssetsWithDependencies = (
spawner: ChildProcessSpawner.ChildProcessSpawner["Service"],
input?: StackPreparationInput,
publishEvent?: (event: StackPreparationEvent) => Effect.Effect<void>,
): Effect.Effect<PreparedStackArtifacts, DockerPullError | ChecksumMismatchError> =>
): Effect.Effect<PreparedStackArtifacts, StackPreparationError> =>
Effect.gen(function* () {
const versions = { ...DEFAULT_VERSIONS, ...input?.versions };
const services: ReadonlyArray<ServiceName> = input?.services ?? SERVICE_NAMES;
const mode = input?.mode ?? "auto";

type Entry = readonly [ServiceName, ServiceResolution];

const resolveService = (
service: ServiceName,
): Effect.Effect<Entry, DockerPullError | ChecksumMismatchError> => {
const resolveService = (service: ServiceName): Effect.Effect<Entry, StackPreparationError> => {
let isDownloading = false;
const markDownloadStart = () =>
Effect.sync(() => {
Expand All @@ -120,7 +126,18 @@ export const prepareAssetsWithDependencies = (
);
}

// Docker-only services cannot run natively. The native+docker-only CONFIG
// contract is enforced one layer up (`StackBuilder.validateResolvedConfig`);
// this branch is only reachable in "native" mode via `prefetch()` (which
// defaults to ALL services), where warming native assets must not require
// a Docker daemon — resolve to the canonical image name without pulling.
if (dockerOnlyServices.has(service)) {
if (mode === "native") {
return Effect.succeed<Entry>([
service,
{ type: "docker", image: dockerImageForService(service, versions[service]) },
]);
}
return resolveDockerImageForService(spawner, service, versions[service], {
onDownloadStart: markDownloadStart(),
Comment thread
7ttp marked this conversation as resolved.
}).pipe(
Expand All @@ -135,6 +152,7 @@ export const prepareAssetsWithDependencies = (
service,
versions[service],
markDownloadStart(),
mode,
).pipe(
Effect.map((resolution): Entry => [service, resolution]),
Effect.ensuring(markDownloadFinished()),
Expand All @@ -157,10 +175,10 @@ export class StackPreparation extends Context.Service<
{
readonly prepare: (
input?: StackPreparationInput,
) => Effect.Effect<PreparedStackArtifacts, DockerPullError | ChecksumMismatchError>;
) => Effect.Effect<PreparedStackArtifacts, StackPreparationError>;
readonly prepareEvents: (
input?: StackPreparationInput,
) => Stream.Stream<StackPreparationEvent, DockerPullError | ChecksumMismatchError>;
) => Stream.Stream<StackPreparationEvent, StackPreparationError>;
}
>()("stack/StackPreparation") {
static layer: Layer.Layer<
Expand All @@ -177,7 +195,7 @@ export class StackPreparation extends Context.Service<
prepare: (input?: StackPreparationInput) =>
prepareAssetsWithDependencies(resolver, spawner, input),
prepareEvents: (input?: StackPreparationInput) =>
Stream.callback<StackPreparationEvent, DockerPullError | ChecksumMismatchError>((queue) =>
Stream.callback<StackPreparationEvent, StackPreparationError>((queue) =>
prepareAssetsWithDependencies(resolver, spawner, input, (event) =>
Queue.offer(queue, event),
).pipe(
Expand Down Expand Up @@ -268,34 +286,25 @@ const resolveServiceWithMetadata = (
service: ServiceName,
version: string,
onDownloadStart: Effect.Effect<void>,
): Effect.Effect<ServiceResolution, DockerPullError | ChecksumMismatchError> =>
resolver.resolveWithMetadata({ service, version }, { onDownloadStart }).pipe(
Effect.map(({ path }): ServiceResolution => ({ type: "binary", path })),
Effect.catchTag("BinaryNotFoundError", () =>
resolveDockerImageForService(spawner, service, version, {
onDownloadStart,
}).pipe(
Effect.map(
(image): ServiceResolution => ({
type: "docker",
image,
}),
),
),
),
Effect.catchTag("DownloadError", () =>
resolveDockerImageForService(spawner, service, version, {
onDownloadStart,
}).pipe(
Effect.map(
(image): ServiceResolution => ({
type: "docker",
image,
}),
),
),
),
mode: "native" | "auto",
): Effect.Effect<ServiceResolution, StackPreparationError> => {
const nativeBinary = resolver
.resolveWithMetadata({ service, version }, { onDownloadStart })
.pipe(Effect.map(({ path }): ServiceResolution => ({ type: "binary", path })));
// `mode: "native"` requires native binaries (README): resolution failures
// propagate instead of silently flipping the service onto Docker.
if (mode === "native") {
return nativeBinary;
}
const dockerFallback = () =>
resolveDockerImageForService(spawner, service, version, {
onDownloadStart,
}).pipe(Effect.map((image): ServiceResolution => ({ type: "docker", image })));
return nativeBinary.pipe(
Effect.catchTag("BinaryNotFoundError", dockerFallback),
Effect.catchTag("DownloadError", dockerFallback),
);
};

const runPullCommand = (
spawner: ChildProcessSpawner.ChildProcessSpawner["Service"],
Expand Down
1 change: 1 addition & 0 deletions packages/stack/src/effect.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ export { BinaryResolver } from "./BinaryResolver.ts";

export type { ServiceResolution } from "./resolve.ts";
export { resolveService } from "./resolve.ts";
export type { StackPreparationError } from "./StackPreparation.ts";

export type { PrefetchOptions, PrefetchResult } from "./prefetch.ts";
export { prefetch } from "./prefetch.ts";
Expand Down
25 changes: 21 additions & 4 deletions packages/stack/src/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,41 @@ import { Data } from "effect";
export class BinaryNotFoundError extends Data.TaggedError("BinaryNotFoundError")<{
readonly service: string;
readonly platform: string;
}> {}
}> {
override get message() {
return `No native ${this.service} binary is available for ${this.platform}. Native mode requires a native binary — use mode "auto" or "docker" to run this service with Docker.`;
}
}

export class DownloadError extends Data.TaggedError("DownloadError")<{
readonly url: string;
readonly cause: unknown;
}> {}
}> {
override get message() {
const cause = this.cause instanceof Error ? `: ${this.cause.message}` : "";
return `Failed to download ${this.url}${cause}`;
}
}

export class ChecksumMismatchError extends Data.TaggedError("ChecksumMismatchError")<{
readonly url: string;
readonly expected: string;
readonly actual: string;
}> {}
}> {
override get message() {
return `Checksum mismatch for ${this.url} (expected ${this.expected}, got ${this.actual})`;
}
}

export class DockerPullError extends Data.TaggedError("DockerPullError")<{
readonly image: string;
readonly detail: string;
readonly cause: unknown;
}> {}
}> {
override get message() {
return this.detail;
}
}

export class StackBuildError extends Data.TaggedError("StackBuildError")<{
readonly detail: string;
Expand Down
10 changes: 6 additions & 4 deletions packages/stack/src/prefetch.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import { Effect } from "effect";
import type { ChecksumMismatchError } from "./errors.ts";
import type { DockerPullError } from "./errors.ts";
import { type PreparedStackArtifacts, type StackPreparationInput } from "./StackPreparation.ts";
import {
type PreparedStackArtifacts,
type StackPreparationError,
type StackPreparationInput,
} from "./StackPreparation.ts";
import { StackPreparation } from "./StackPreparation.ts";
import type { ServiceResolution } from "./resolve.ts";

Expand All @@ -14,7 +16,7 @@ const toPrefetchResult = (artifacts: PreparedStackArtifacts): PrefetchResult =>

export const prefetch = (
options?: PrefetchOptions,
): Effect.Effect<PrefetchResult, DockerPullError | ChecksumMismatchError, StackPreparation> =>
): Effect.Effect<PrefetchResult, StackPreparationError, StackPreparation> =>
Effect.gen(function* () {
const preparation = yield* StackPreparation;
return yield* preparation.prepare(options).pipe(Effect.map(toPrefetchResult));
Expand Down
60 changes: 60 additions & 0 deletions packages/stack/src/prefetch.unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,66 @@ describe("prefetch", () => {
expect(events.at(-1)).toBe("PreparationCompleted");
});

test("native mode fails on a missing binary instead of silently falling back to Docker", async () => {
const resolver = mockBinaryResolver({ failServices: ["auth"] });
const spawner = mockSequenceSpawner([]);

const layer = StackPreparation.layer.pipe(
Layer.provide(resolver.layer),
Layer.provide(spawner.layer),
);

const error = await Effect.runPromise(
prefetch({ mode: "native", services: ["auth"] }).pipe(Effect.provide(layer), Effect.flip),
);

expect(error._tag).toBe("BinaryNotFoundError");
// The silent substrate flip is the bug: no Docker resolution may be attempted.
expect(spawner.spawned).toEqual([]);
});

test("native mode fails on a download error instead of silently falling back to Docker", async () => {
const resolver = mockBinaryResolver({ downloadErrorServices: ["postgres"] });
const spawner = mockSequenceSpawner([]);

const layer = StackPreparation.layer.pipe(
Layer.provide(resolver.layer),
Layer.provide(spawner.layer),
);

const error = await Effect.runPromise(
prefetch({ mode: "native", services: ["postgres"] }).pipe(Effect.provide(layer), Effect.flip),
);

expect(error._tag).toBe("DownloadError");
expect(spawner.spawned).toEqual([]);
});

test("native mode resolves docker-only services to an image name without touching Docker", async () => {
// `prefetch({ mode: "native" })` may run on a machine with no Docker daemon
// to warm native assets — docker-only services must not trigger any pull.
const resolver = mockBinaryResolver();
const spawner = mockSequenceSpawner([]);

const layer = StackPreparation.layer.pipe(
Layer.provide(resolver.layer),
Layer.provide(spawner.layer),
);

const result = await Effect.runPromise(
prefetch({ mode: "native", services: ["postgres", "edge-runtime"] }).pipe(
Effect.provide(layer),
),
);

expect(result["edge-runtime"]).toEqual({
type: "docker",
image: `public.ecr.aws/supabase/edge-runtime:v${DEFAULT_VERSIONS["edge-runtime"]}`,
});
expect(result["postgres"]?.type).toBe("binary");
expect(spawner.spawned).toEqual([]);
});

test("uses docker for edge-runtime in auto mode even when a native binary exists", async () => {
const resolver = mockBinaryResolver();
const spawner = mockSequenceSpawner([{ exitCode: 0 }]);
Expand Down
Loading
Loading