y-cluster initial workflows - #2
Merged
Merged
Conversation
Net repo-level changes since the v0.2.0 scope freeze: - Promote root main.go to cmd/kustomize-traverse/ alongside the growing cmd/y-cluster CLI; legacy root e2e_test.go drops. - README.md advertises the binary's full subcommand surface and points at the dedicated specs repo. - .github/workflows/ci.yaml is the consolidated build matrix (build once, multi-arch image via contain, raw release assets, lint + e2e jobs); mirror-k3s.yaml runs the upstream-to-ghcr k3s mirror nightly. - test.sh is the single-command "everything" runner: unit tests, e2e tagged variants, gated kvm/docker buckets where the host supports them. No Go-source changes in this commit -- those land in the feature commits that follow (yconverge / utils / cluster+image lifecycle / serve / provision / envoy gateway). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates yconverge off kubectl shellouts and onto client-go's
typed APIs for the two operations that drove almost every
out-of-band shell-out in the original implementation:
- pkg/k8sapply replaces 'kubectl apply --server-side --force-conflicts
--field-manager=y-cluster -k <dir>'. Same wire format (server-side
apply Patch with PatchType=ApplyYAMLPatchType, Force=true), built
on the dynamic client + RESTMapper. CRDs apply first then trigger
a discovery cache invalidation so subsequent CRs of those types
resolve.
- pkg/k8swait replaces 'kubectl wait' and 'kubectl rollout status'.
Wait compiles a typed predicate (condition= or jsonpath=);
RolloutStatus implements the per-kind progression rules kubectl
uses for Deployment / StatefulSet / DaemonSet.
Both packages return categorisable errors (apierrors.IsConflict,
IsForbidden, IsServerTimeout, IsNotFound) so callers can act on
specific failure modes -- something the previous "exit 1, see
stderr" path made impossible.
pkg/kubeconfig switches from 'kubectl config' shellouts to
clientcmd merge / cleanup so the cross-provisioner context
management code (used by qemu and later docker) builds and
inspects rules through the same library kubectl itself uses.
pkg/yconverge picks up the new primitives:
- checks.go / cue.go / deps.go / yconverge.go updated to thread
errors with paths, surface kubectl output structurally, and
propagate --checks-only across the dep graph.
- cue.go reads the module name from cue.mod/module.cue so the
hardcoded 'yolean.se/ystack' prefix is no longer assumed.
e2e/cluster/{cluster,kwok,fixtures}.go is the shared kwok-in-docker
harness extracted from yconverge's original e2e tests; subsequent
feature commits reuse the same fixture rather than each spinning
up their own apiserver.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five small packages used by the rest of the codebase:
- pkg/configfile: generic Load[T] with Validator / Defaulter /
DirAware optional interfaces. Strict YAML decode, abs-path
resolution, and per-error path wrapping. The unified loading
primitive every CLI subcommand that takes -c <dir> uses.
- pkg/envsubst: opt-in shell-style ${VAR} and ${VAR:-default}
expansion on string fields tagged envsubst:"true". Forward-compat
scan rejects ${...} on untagged scalars and on map keys -- the
schema, not a text-level pre-pass, is the authoritative list of
substitution points. Wired into pkg/configfile.Load so every
config gets the same policy.
- pkg/cache: resolves the on-disk root y-cluster uses for
downloaded artefacts (k3s airgap bundles, OCI image layouts).
Resolution order: --cache-dir flag, $Y_CLUSTER_CACHE_DIR,
$XDG_CACHE_HOME/y-cluster, $HOME/.cache/y-cluster.
- pkg/sshexec: replaces ssh / scp / ssh-keygen shell-outs with
golang.org/x/crypto/ssh + pkg/sftp. SSH targets are typed,
sessions wired through context.Context, errors carry the host
the operation hit.
- pkg/dockerexec: replaces docker run / exec / rm / logs with
the moby/client API + containerd/errdefs typed errors so
callers can branch on IsNotFound, IsConflict, etc. without
parsing stderr.
No CLI surface changes in this commit -- the subcommands that
consume these libraries land in the cluster + image lifecycle,
serve, and provision commits that follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the bash that ystack used to ship for local-cluster inspection and image staging: - pkg/cluster.Lookup detects which provisioner serves a kubeconfig context (qemu vs k3s-in-docker) by inspecting the cluster's endpoint and matching it against known shapes; pkg/cluster.Exec routes commands to the right backend (SSH for qemu, docker exec for k3s-in-docker) so a single 'ctr' or 'crictl' invocation works against either. - pkg/images.ListYAML extracts every container image referenced by a Kubernetes YAML stream (PodSpec walker covering Deployment / StatefulSet / DaemonSet / Job / CronJob / ReplicaSet / Pod); pkg/images.Cache pulls a ref into a content-addressable OCI layout under the shared cache; pkg/images.Load streams an OCI archive into the cluster node's containerd via the routed exec. CLI surface added in cmd/y-cluster: detect print the provisioner backend serving --context ctr <args...> run ctr on the cluster node, routed by detect crictl <args...> run crictl on the cluster node, routed by detect images list print refs from a YAML stream / kustomize build images cache pull a ref into the shared OCI cache images load stream an OCI archive into the node's containerd cache info resolve and print the cache root and its size cache purge remove cached artefacts (selective or all) The e2e harness gains a registry helper (e2e/cluster/registry.go) plus per-feature tests: e2e/cluster_features_test.go runs detect / ctr / crictl against every real-cluster provisioner; e2e/images_cache_test.go covers the list + cache + load matrix including the airgap-pod proof for k3s+containerd. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e DX
Builds out the y-cluster serve subcommand from the v0.2.0 stub
into the shape ystack's gateway-v4 surface needs:
- y-kustomize-local backend: kustomize-builds source directories
on disk and serves the rendered YAML over HTTP, instead of
re-flattening file paths. Catches and rejects file-renaming
generators (configMapGenerator, secretGenerator) that would
conflict between sibling sources. Source layout switched from
the old y-kustomize-bases/ subdir convention to a flat sources/
layout with each source rooted at its own kustomization.yaml.
- y-kustomize-incluster backend: watches a configured set of
Kubernetes objects and serves their content over HTTP. Renamed
from the prior 'incluster' backend; readiness now signalled
via /health so e2e tests can wait for watch propagation
reliably.
- typed Ensure result with Action enum (Started / Restarted /
Noop). Success status lines go to stdout (so scripts can
capture them); daemon errors stay on stderr and surface
inline rather than waiting for the spawned process to flush.
- pkg/serve picks up the configfile.Load wiring so
y-cluster-serve.yaml gets the same validation, defaulting,
and ${VAR} substitution policy the other configs use.
Generated schema (pkg/serve/schema/y-cluster-serve.schema.json)
and the testdata fixtures move with the package changes;
testdata/serve-ykustomize-local/ now reflects the flat layout
and includes the duplicate-source detection fixtures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the v0.2.0 ad-hoc qemu provisioner with a typed,
provider-discriminated configuration surface and adds a parallel
docker provisioner so the same y-cluster-provision.yaml can drive
either backend.
Configuration:
- pkg/provision/config defines CommonConfig (provider discriminator,
name, context, memory, cpus, k3s, portForwards, registries) and
per-provider QEMUConfig / DockerConfig that embed it. schemagen
in cmd/internal/schemagen emits one JSON Schema per provider
plus common.schema.json, with a collision check that prevents
two providers from declaring the same own yaml key.
- LoadProvision dispatches on the provider field and falls back to
runtime discovery (linux+/dev/kvm+qemu-system -> qemu;
docker info OK -> docker) when omitted.
- portForwards lifts to CommonConfig (defaults 6443/80/443 with a
required guest:6443 entry). The docker provider uses the same
shape as qemu instead of a single apiPort field.
- registries: section maps to the k3s registries.yaml shape;
pkg/provision/registries marshals it and both providers stage it
on the node before k3s starts (qemu via SSH+tee, docker via
CopyToContainer between create and start). Auth credentials are
envsubst:"true" so ${GCP_ACCESS_TOKEN}-style references resolve
through pkg/envsubst's strict policy.
qemu provisioner:
- pkg/provision/qemu installs k3s on the VM via the script or
airgap method (the airgap path caches the binary + tarball
under pkg/cache.K3s so re-provision skips the GitHub round
trip).
- TeardownConfig sends SIGTERM, waits, escalates to SIGKILL,
then deletes the disk -- order matters so a stuck VM doesn't
leave host-port forwards bound while teardown reports success.
- pkg/provision/qemu/process.go replaces kill / kill -0 / curl
shellouts with stdlib equivalents that surface typed errors.
docker provisioner:
- pkg/provision/docker runs k3s in a privileged container.
ResolveImage probes the y-cluster mirror (ghcr.io/yolean/k3s)
for the configured version and falls back to upstream
rancher/k3s when the mirror has no manifest yet -- and treats
401/403 the same as 404 so an anonymous probe of a missing or
private GHCR repo lands on the upstream fallback rather than
erroring out.
CLI changes in cmd/y-cluster/main.go: provisionCmd / teardownCmd /
exportCmd / importCmd switch from the qemu-only flat-flags shape
to -c <dir> + the typed config dispatcher; --version surfaces the
short git ref + an optional -dirty marker via runtime/debug.
E2E coverage:
- testdata/provision-qemu/ holds a minimal y-cluster-provision.yaml
fixture used by the e2e tests.
- e2e/qemu_test.go exercises the full provision -> teardown loop
(with pid-gone post-teardown assertion).
- e2e/docker_test.go covers the docker provider including a
non-API portForwards entry to prove the binding wiring.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces k3s's default Traefik with Envoy Gateway, completing the
spec's "Envoy Gateway during provision" item. Both providers gain
a single ingress-controller story so consumer kustomize bases can
target gatewayClassName: eg without a separate setup step.
What lands during provision:
1. k3s starts with --disable=traefik (qemu install scripts and
docker container Cmd both flipped).
2. envoygateway.Ensure resolves install.yaml from the per-version
cache, downloading from the upstream release on first need
via an atomic .tmp+rename. Cache layout under
<root>/envoygateway/<version>/ matches pkg/cache.K3s in style
so a 'cache purge --envoygateway-version=v1.7.2' can be one
recursive delete.
3. envoygateway.Install applies install.yaml (CRDs first via
k8sapply's CRD-aware ordering), waits for the controller
Deployment, then applies the embedded default `eg` GatewayClass.
4. envoygateway.CacheImages pre-pulls the install manifest's image
refs into <version>/images/<digest>/ so airgap loading is a
future commit away.
Package layout:
- pkg/cache.EnvoyGateway / EnvoyGatewayVersion: the new cache
helpers parallel to K3s.
- pkg/provision/envoygateway/version.go: pinned constant.
- assets/gatewayclass.yaml: y-cluster-owned default GatewayClass,
embedded since it's tiny and identical across releases.
- ensure.go: download + atomic write, idempotent.
- install.go: orchestrates Ensure + apply + wait + GatewayClass.
- images.go: list + pre-cache.
k8sapply gains ApplyYAML alongside Apply so vendored multi-doc
manifests apply through the same SSA + CRD-first path without a
temp-dir + kustomization.yaml dance.
Self-contained coverage:
- pkg/provision/envoygateway/{version,images,ensure}_test.go:
format guard, image enumeration + dedup contract, httptest-
driven download with second-call cache-hit and failed-download
leaves no stale file.
- e2e/envoygateway_test.go: against the kwok harness with
ReadyTimeout=-1 (kwok doesn't run the real controller). Asserts
CRDs, namespace, Deployment, Service, GatewayClass land via
kubectl. Covers default install and SkipGatewayClass opt-out.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
solsson
force-pushed
the
y-cluster-initial-workflows
branch
from
April 27, 2026 10:41
fa4a79b to
f97e18a
Compare
- pkg/cluster/exec.go, pkg/cluster/lookup.go, and
pkg/provision/docker/docker.go each had a `defer cli.Close()`
on a *moby/client.Client whose Close returns error. Wrap with
`defer func() { _ = cli.Close() }()` to satisfy errcheck (the
pattern matches what pkg/sshexec already uses for ssh / sftp
clients).
- pkg/dockerexec.New drops client.WithAPIVersionNegotiation;
the moby client deprecated it as a no-op since negotiation
is the default.
scripts/e2e-serve-against-binary.sh: the second `serve ensure`
call (the no-op path) used to print "already running" but the
typed Ensure result the rest of the codebase moved to emits
"y-cluster serve noop on :<port>". Update the grep target.
The test.sh runner now does what CI does, locally:
- golangci-lint run (when installed; otherwise prints "skipped: not
installed; CI runs it" and continues so dev machines without the
binary aren't blocked).
- y-cluster binary build + scripts/e2e-serve-against-binary.sh
against it, which was previously only exercised by the release
pipeline.
Running both surfaced four pre-existing failures latent in the
v0.2.0..spec-fulfillment tree that no path was hitting:
1. pkg/provision/docker/docker.go: ContainerCreate set both
top-level Image and Config.Image -- the moby client rejects
that with "either Image or config.Image should be set" since
the deprecation of the top-level shortcut. Drop the top-level
field; Config.Image stays since it's where Cmd lives anyway.
2. e2e/cluster/fixtures.go: the synthetic OCI image had no
linux/amd64 platform stamped, so containerd 2.x's `ctr image
import` filtered it out as platform-less and the load was a
silent no-op. Set Architecture+OS via mutate.ConfigFile so the
fixture is importable everywhere ctr-import is.
3. pkg/cluster/lookup.go: qemuRunning hardcoded
~/.cache/y-cluster-qemu/<name>.pid, but the e2e qemu test puts
its cache under t.TempDir(). detect/ctr/crictl invoked through
the test then failed "no running qemu cluster matches".
qemuRunning honors $Y_CLUSTER_QEMU_CACHE_DIR; the e2e test
t.Setenv's it to cfg.CacheDir.
4. e2e/cluster_features_test.go: an `images load` failure
discarded the binary's output (assigned to `_`), so today's CI
surfaced "exit status 1" with no detail. Forward the binary
output into t.Fatalf -- the diagnostic that pinpointed (2)
above.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
solsson
force-pushed
the
y-cluster-initial-workflows
branch
from
April 27, 2026 12:02
83ac846 to
0fee164
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.