Orchestrator Extraction — Remove Orchestrator Code, Add Plugin Loader - #819
Merged
Conversation
…ule profiles, caching, LFS, hooks Add generic enterprise-grade features to the orchestrator, enabling Unity projects with complex CI/CD pipelines to adopt game-ci/unity-builder with built-in support for: - CLI provider protocol: JSON-over-stdin/stdout bridge enabling providers in any language (Go, Python, Rust, shell) via the `providerExecutable` input - Submodule profiles: YAML-based selective submodule initialization with glob patterns and variant overlays (`submoduleProfilePath`, `submoduleVariantPath`) - Local build caching: Filesystem-based Library and LFS caching for local builds without external cache actions (`localCacheEnabled`, `localCacheRoot`) - Custom LFS transfer agents: Register external transfer agents like elastic-git-storage (`lfsTransferAgent`, `lfsTransferAgentArgs`, `lfsStoragePaths`) - Git hooks support: Detect and install lefthook/husky with configurable skip lists (`gitHooksEnabled`, `gitHooksSkipList`) Also removes all `orchestrator-develop` branch references, replacing with `main`. 13 new action inputs, 13 new files, 14 new CLI provider tests, 17 submodule tests, plus cache/LFS/hooks unit tests. All 452 tests pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…iders Add two new cloud provider implementations for the orchestrator, both marked as experimental: - **GCP Cloud Run Jobs** (`providerStrategy: gcp-cloud-run`): Executes Unity builds as Cloud Run Jobs with GCS FUSE for large artifact storage. Supports configurable machine types, service accounts, and VPC connectors. 7 new inputs (gcpProject, gcpRegion, gcpBucket, gcpMachineType, gcpDiskSizeGb, gcpServiceAccount, gcpVpcConnector). - **Azure Container Instances** (`providerStrategy: azure-aci`): Executes Unity builds as ACI containers with Azure File Shares (Premium FileStorage) for large artifact storage up to 100 TiB. Supports configurable CPU/memory, VNet integration, and subscription targeting. 9 new inputs (azureResourceGroup, azureLocation, azureStorageAccount, azureFileShareName, azureSubscriptionId, azureCpu, azureMemoryGb, azureDiskSizeGb, azureSubnetId). Both providers use their respective CLIs (gcloud, az) for infrastructure management and support garbage collection of old build resources. No tests included as these require real cloud infrastructure to validate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both providers now support four storage backends via gcpStorageType / azureStorageType: GCP Cloud Run: - gcs-fuse: Mount GCS bucket as POSIX filesystem (unlimited, best for large sequential I/O) - gcs-copy: Copy artifacts in/out via gsutil (simpler, no FUSE overhead) - nfs: Filestore NFS mount (true POSIX, good random I/O, up to 100 TiB) - in-memory: tmpfs (fastest, volatile, up to 32 GiB) Azure ACI: - azure-files: SMB file share mount (up to 100 TiB, premium throughput) - blob-copy: Copy artifacts in/out via az storage blob (no mount overhead) - azure-files-nfs: NFS 4.1 file share mount (true POSIX, no SMB lock overhead) - in-memory: emptyDir tmpfs (fastest, volatile, limited by container memory) New inputs: gcpStorageType, gcpFilestoreIp, gcpFilestoreShare, azureStorageType, azureBlobContainer. Constructor validates storage config and warns on missing prerequisites (e.g. NFS requires VPC connector/subnet). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ity check Adds built-in load balancing: check GitHub runner availability before builds start, auto-route to a fallback provider when runners are busy or offline. Eliminates the need for a separate check-runner job. New inputs: fallbackProviderStrategy, runnerCheckEnabled, runnerCheckLabels, runnerCheckMinAvailable. Outputs providerFallbackUsed and providerFallbackReason for workflow visibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds retryOnFallback (retry failed builds on alternate provider) and providerInitTimeout (swap provider if init takes too long). Refactors run() into run()/runWithProvider() to support retry loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds tests for cache hit restore (picks latest tar), LFS cache restore/save, garbage collection age filtering, and edge cases like permission errors and empty directories. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers: no token skip, no runners fallback, busy/offline runners, label filtering (case-insensitive), minAvailable threshold, fail-open on API error, mixed runner states. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds 64 new mock-based unit tests covering orchestrator services that previously had zero test coverage: - TaskParameterSerializer: env var format conversion, round-trip, uniqBy deduplication, blocked params, default secrets - FollowLogStreamService: build output message parsing — end of transmission, build success/failure detection, error accumulation, Library rebuild detection - OrchestratorNamespace (guid): GUID generation format, platform name normalization, nanoid uniqueness - OrchestratorFolders: path computation for all folder getters, ToLinuxFolder conversion, repo URL generation, purge flag detection All tests are pure mock-based and run without any external infrastructure (no LocalStack, K8s, Docker, or AWS). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a fast-fail unit test step at the top of orchestrator-integrity, right after yarn install and before any infrastructure setup (k3d, LocalStack). Runs 113 mock-based orchestrator tests in ~5 seconds. If serialization, path computation, log parsing, or provider loading is broken, the workflow fails immediately instead of spending 30+ minutes setting up LocalStack and k3d clusters. Tests included: orchestrator-guid, orchestrator-folders, task-parameter-serializer, follow-log-stream-service, runner-availability-service, provider-url-parser, provider-loader, provider-git-manager, orchestrator-image, orchestrator-hooks, orchestrator-github-checks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive tests for CLI provider (cleanupWorkflow, garbageCollect, listWorkflow, watchWorkflow, stderr forwarding, timeout handling), local cache service (saveLfsCache full path and error handling), git hooks service (husky install, failure logging, edge cases), and LFS agent service (empty storagePaths, validate logging). 73 tests across 4 test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace token-in-URL pattern with http.extraHeader for git clone and LFS operations. The token no longer appears in clone URLs, git remote config, or process command lines. Add gitAuthMode input (default: 'header', legacy: 'url') so users can fall back to the old behavior if needed. Closes #785 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SecretSourceService with premade secret source integrations:
- aws-secrets-manager (with --query SecretString for direct value)
- aws-parameter-store (with --with-decryption)
- gcp-secret-manager (latest version)
- azure-key-vault (via $AZURE_VAULT_NAME env var)
- env (environment variables, no shell command needed)
- Custom commands (any string with {0} placeholder)
- YAML file definitions for custom sources
Add secretSource input that takes precedence over inputPullCommand.
Backward compatible — existing inputPullCommand behavior unchanged.
Closes #776
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds three Vault entries: hashicorp-vault (KV v2), hashicorp-vault-kv1 (KV v1), and vault (short alias). Uses VAULT_ADDR for server address and VAULT_MOUNT env var for configurable mount path (defaults to 'secret'). Refs #776 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
First-class support for elastic-git-storage as a custom LFS transfer agent. When lfsTransferAgent is set to "elastic-git-storage" (or "elastic-git-storage@v1.0.0" for a specific version), the service automatically finds or installs the agent from GitHub releases, then configures it via git config. Supports version pinning via @Version suffix in the agent value, eliminating the need for a separate version parameter. Platform and architecture detection handles linux/darwin/windows on amd64/arm64. 37 unit tests covering detection, PATH lookup, installation, version parsing, and configuration delegation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Built-in support for Unity Git Hooks (com.frostebite.unitygithooks): - Auto-detect UPM package in Packages/manifest.json - Run init-unity-lefthook.js before hook installation - Set CI-friendly env vars (disable background project mode) New gitHooksRunBeforeBuild input runs specific lefthook groups before the Unity build, allowing CI to trigger pre-commit or pre-push checks that normally only fire on git events. 35 unit tests covering detection, init, CI env, group execution, and failure handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Initial scaffold for the test workflow engine service directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Initial scaffold for the runner registration and hot editor provider module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, and collection service Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ut, and storage-backed sync Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add scheduled workflow that validates community Unity packages compile and build correctly using unity-builder. Runs weekly on Sunday. Includes: - YAML plugin registry (community-plugins.yml) for package listings - Matrix expansion across plugins and platforms - Automatic failure reporting via GitHub issues - Manual trigger with plugin filter and Unity version override Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… Actions, GitLab CI, Ansible Add four new providers that delegate builds to external CI platforms: - remote-powershell: Execute on remote machines via WinRM/SSH - github-actions: Dispatch workflow_dispatch on target repository - gitlab-ci: Trigger pipeline via GitLab API - ansible: Run playbooks against managed inventory Each follows the CI-as-a-provider pattern: trigger remote job, pass build parameters, stream logs, report status. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ved filename cleanup, archival Add three optional reliability features for hardening CI pipelines: - Git corruption detection & recovery (fsck, stale lock cleanup, submodule backing store validation, auto-recovery) - Reserved filename cleanup (removes Windows device names that cause Unity asset importer infinite loops) - Build output archival with configurable retention policy All features are opt-in and fail gracefully with warnings only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rity, reserved filename cleanup, and build archival Adds BuildReliabilityService with the following capabilities: - checkGitIntegrity(): runs git fsck --no-dangling and parses output for corruption - cleanStaleLockFiles(): removes stale .lock files older than 10 minutes - validateSubmoduleBackingStores(): validates .git files point to valid backing stores - recoverCorruptedRepo(): orchestrates fsck, lock cleanup, re-fetch, retry fsck - cleanReservedFilenames(): removes Windows reserved filenames (con, prn, aux, nul, com1-9, lpt1-9) - archiveBuildOutput(): creates tar.gz archive of build output - enforceRetention(): deletes archives older than retention period - configureGitEnvironment(): sets GIT_TERMINAL_PROMPT=0, http.postBuffer, core.longpaths Wired into action.yml as opt-in inputs, with pre-build integrity checks and post-build archival in the main entry point. Includes 29 unit tests covering success and failure cases for all methods. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Lab CI, PowerShell, and Ansible providers (#806) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… monitoring, and job dispatch (#791) Adds persistent Unity editor instance support to reduce build iteration time by eliminating cold-start overhead. Includes: - HotRunnerTypes: interfaces for config, status, job request/result, transport - HotRunnerRegistry: in-memory runner management with file-based persistence - HotRunnerHealthMonitor: periodic health checks, idle recycling, job-count recycling - HotRunnerDispatcher: job routing with wait-for-runner, timeout, and output streaming - HotRunnerService: high-level API integrating registry, health, and dispatch - 34 unit tests covering registration, filtering, health, dispatch, timeout, fallback - action.yml inputs for hot runner configuration (7 new inputs) - Input/BuildParameters integration for hot runner settings - index.ts wiring with cold-build fallback when hot runner unavailable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s, tests, and action integration (#798) - Add ArtifactUploadHandler with support for github-artifacts, storage (rclone), and local copy upload targets, including large file chunking for GitHub Artifacts - Add 44 unit tests covering OutputTypeRegistry, OutputService, and ArtifactUploadHandler (config parsing, upload coordination, file collection) - Add 6 new action.yml inputs for artifact configuration - Add artifactManifestPath action output - Wire artifact collection and upload into index.ts post-build flow Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…omy filtering, and structured results (#790) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…te management, and tests (#799) - Add storage-pull strategy: rclone-based sync from remote storage with overlay and clean modes, URI parsing (storage://remote:bucket/path), transfer parallelism, and automatic rclone availability checking - Add SyncStateManager: persistent state load/save with configurable paths, workspace hash calculation via SHA-256 of key project files, and drift detection for external modification awareness - Add action.yml inputs: syncStrategy, syncInputRef, syncStorageRemote, syncRevertAfter, syncStatePath with sensible defaults - Wire sync into Input (5 getters), BuildParameters (5 fields), index.ts (local build path), and RemoteClient (orchestrator path) with post-job overlay revert when syncRevertAfter is true - Add 42 unit tests covering all strategies, URI parsing, state management, hash calculation, drift detection, error handling, and edge cases (missing rclone, invalid URIs, absent state, empty diffs) - Add root:true to eslintrc to prevent plugin resolution conflicts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add workflow_call trigger to validate-orchestrator-integration.yml so other workflows can invoke the exhaustive test suite - Add orchestrator-integration job to integrity-check.yml that runs on pushes to main (skipped on PRs to avoid 1-2h CI time) - Daily cron + manual dispatch remain as fallback triggers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
localstack:latest (v4.14+) returns JSON responses for some S3 operations, but @aws-sdk/client-s3 v3.779+ uses AwsRestXmlProtocol which expects XML. This breaks all SharedWorkspaceLocking tests (locking, e2e caching, retaining). Pin to v3.8.1 (last v3 release) where the S3 provider returns proper XML responses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The S3 deserialization issue was caused by @aws-sdk/client-s3 v3.1005 (schema-based AwsRestXmlProtocol), not LocalStack's version. The SDK is now pinned to ~3.779.0 in the orchestrator repo, so localstack:latest works correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move mandatory tests (caching, locking-core, locking-get-locked) before continue-on-error e2e tests. The e2e tests can corrupt the workspace (delete package.json), which was causing subsequent mandatory tests to fail with "Couldn't find a package.json". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Member
Author
There is a CLI provided by orchestrator now it's standalone. It's a pretty good entrypoint for a CLI generally for Game-CI. This change specifically doesn't add it or require it, because this just implements orchestrator as a standalone plugin. |
# Conflicts: # .github/workflows/orchestrator-integrity.yml # dist/index.js # dist/index.js.map # src/model/orchestrator/providers/aws/aws-task-runner.ts # src/model/orchestrator/providers/aws/index.ts
Replace hardcoded orchestrator params with a lifecycle-based plugin interface. The orchestrator reads its own config from env vars — unity-builder just calls 6 hooks (initialize, canHandleBuild, handleBuild, beforeLocalBuild, afterLocalBuild, handlePostBuild). Removes ~2900 lines from unity-builder (93 BuildParameters fields, 346 Input getters, 70 action.yml inputs, 400 lines of service orchestration in index.ts). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The validate-orchestrator workflows referenced loadOrchestrator and loadPluginServices which don't exist — the source exports loadOrchestratorPlugin. Updated all CI steps to use the correct function name and test the actual OrchestratorPlugin lifecycle interface. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The validate-orchestrator workflow was always checking out the main branch of game-ci/orchestrator. When both repos have changes on a feature branch (e.g. refactor/orchestrator-extraction), the CI needs to use the matching branch. Falls back to main if the branch doesn't exist in the orchestrator repo. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PRs labeled `run-integration` now run the full orchestrator integration suite (K8s, AWS, local-docker, rclone via LocalStack + k3d). Without the label, integration tests only run on push to main and the daily cron. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Try the matching branch name (e.g. refactor/orchestrator-extraction) from game-ci/orchestrator first, falling back to main. This allows testing cross-repo changes before merging to orchestrator main. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LocalStack community edition was discontinued (2026.03.0+) and now requires a paid license for ECS, CloudFormation, Kinesis, and other services used in integration tests. Switch to MiniStack (MIT, free, ministackorg/ministack) which provides all 40+ AWS services on the same port 4566 with backward-compatible health endpoints. ~10x smaller image, ~2s startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Manually-triggered workflow that copies secrets (Unity credentials, AWS/GCP tokens, Codecov) from unity-builder to orchestrator or cli repos. Supports dry-run mode. Folded from PR #825. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 tasks
…ntain permissions' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MiniStack doesn't require an auth token. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
frostebite
added a commit
to game-ci/documentation
that referenced
this pull request
May 3, 2026
* Add custom providers documentation for the plugin system
Document how to use custom providers via GitHub repos, NPM packages,
or local paths. Covers the ProviderInterface, supported source formats,
caching behavior, and a full example implementation. Also updates the
API reference to mention custom providers under providerStrategy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Comprehensive orchestrator documentation overhaul
- Restructure providers into dedicated section with overview, custom
providers, community providers (with GitHub edit link), and
GitHub/GitLab integration pages
- Rewrite API reference with proper tables, all missing parameters
(orchestratorRepoName, githubOwner, allowDirtyBuild, postBuildSteps,
preBuildSteps, customHookFiles, customCommandHooks, useCleanupCron),
and environment variables (AWS_FORCE_PROVIDER, PURGE_REMOTE_BUILDER_CACHE,
ORCHESTRATOR_AWS_STACK_WAIT_TIME, GIT_PRIVATE_TOKEN)
- Document premade rclone hooks and Steam deployment hooks
- Add S3/rclone workspace locking documentation
- Tighten language across all pages for clarity
- Add ASCII diagrams to introduction, caching, logging, and config override
- Add tasteful emoji to section headers
- Rename "Game-CI vs Orchestrator" to "Standard Game-CI vs Orchestrator Mode"
- Remove outdated Deno section from command line docs
- Improve examples with proper tables, workflow snippets, and cross-links
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Remove old standalone GitLab pages from versioned docs
Content already merged into the providers section at
07-providers/05-gitlab-integration.mdx
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Restructure Orchestrator docs: promote Providers to top-level, improve cross-linking
- Promote Providers from Advanced Topics to top-level section (05-providers/)
with dedicated pages for AWS, Kubernetes, Local Docker, Local, Custom, Community,
GitHub Integration, and GitLab Integration
- Move Secrets out of Advanced Topics to top-level (06-secrets.mdx)
- Rename custom-hooks to hooks throughout
- Remove all WIP/preview/release-status notices (project is stable)
- Fix floating {/* */} comment symbols in community-providers (use code block template)
- Update ASCII diagram in Game-CI vs Orchestrator to show CLI/any CI dispatch
- Add sidebar_label frontmatter for Game-CI vs Orchestrator page
- Add comprehensive cross-linking across all orchestrator docs:
- Introduction links to providers, hooks, getting started, platforms
- API Reference links to caching, hooks, providers, configuration override
- Provider pages link to caching, hooks, API Reference sections
- Getting Started links to provider setup guides and secrets
- GitHub Integration links to API Reference for parameters and modes
- Advanced Topics pages cross-reference each other and API Reference
- Fix all broken links from old directory structure
- Delete old directories (examples/github-examples, advanced-topics/providers)
- Run Prettier on all files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Merge Configuration Override into Secrets page, rename to Pull Secrets
- Merge configuration-override.mdx content into secrets.mdx as a section
- Delete standalone configuration-override page
- Rename "Configuration Override" to "Pull Secrets" in API reference
- Update all cross-links (command-line, GitLab integration, API reference)
- Fix logging: "Orchestrator job (Fargate task)" instead of "Fargate tasks"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix broken link: hooks directory has no index page
Link to container-hooks page instead of the hooks directory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add comprehensive GitHub Actions examples page
Complete workflow examples for every provider and common patterns:
- Minimal workflow, AWS Fargate, Kubernetes, Local Docker
- Async mode with GitHub Checks
- Scheduled garbage collection
- Multi-platform matrix builds
- Retained workspaces for faster rebuilds
- Container hooks (S3 upload + Steam deploy)
- Required secrets tables and cross-links to all relevant docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix ASCII diagram alignment in Game-CI vs Orchestrator
Equalize box widths and arrow spacing for consistent rendering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add ASCII diagrams to custom providers, GitHub integration, and retained workspaces
- Custom Providers: plugin loading flow (source → fetch → ProviderInterface)
- GitHub Integration: async mode lifecycle (dispatch → return → Check updates)
- Retained Workspaces: workspace locking across concurrent builds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add ASCII diagrams to container hooks, garbage collection, AWS, and providers overview
- Container Hooks: build pipeline with pre/post hook execution points
- Garbage Collection: resource lifecycle (normal cleanup vs stale → GC)
- AWS: CloudFormation resource stack (ECS, S3, CloudWatch, Kinesis)
- Providers Overview: decision flowchart for choosing a provider
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Replace provider decision tree with simple 4-across comparison
Shows each provider side-by-side with its key trait instead of
a decision flowchart.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix sidebar ordering: Secrets before Advanced Topics
Set Advanced Topics position to 7.0 so it renders after
Secrets (position 6 from filename).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename Premade Container Hooks to Built-In Hooks
Update title and all cross-references across container hooks,
command hooks, and GitHub Actions examples.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename Built-In Hooks file, move Custom Job out of Hooks, fix alignment
- Rename premade-container-jobs.mdx to built-in-hooks.mdx (fixes URL slug)
- Update all links from premade-container-jobs to built-in-hooks
- Rename "Pre-built Hooks" section header to "Built-In Hooks"
- Move Custom Job from hooks/ to advanced-topics/ (it's not a hook)
- Rename "Custom Jobs" to "Custom Job" (singular)
- Update API reference link to advanced-topics/custom-job
- Fix numbering conflicts in advanced-topics
- Fix retained workspace diagram alignment (remove emoji, align box walls)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix alignment of all ASCII diagrams across orchestrator docs
Remove emoji characters from diagrams (variable width across platforms
makes alignment impossible). Fix box wall alignment, arrow connections,
and consistent spacing in all 11 diagrams:
- Introduction (architecture overview)
- Caching (standard vs retained)
- Providers overview (4-across comparison)
- Container hooks (build pipeline)
- GitHub integration (async mode lifecycle)
- AWS (CloudFormation resource stack)
- Secrets (pull flow)
- Logging (log pipeline)
- Garbage collection (resource lifecycle)
- Custom providers (plugin loading)
- Retained workspaces (already fixed)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add Load Balancing documentation page
Documents how to route builds across multiple providers using
GitHub Actions scripting: platform-based routing, branch-based
routing, runner availability fallback, weighted distribution,
and async mode integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add Storage, Architecture pages and Build Caching section
- Storage page: documents project files, build output, caches,
S3 and rclone backends, LZ4 compression, workspace locking,
large packages, and container file system layout
- Architecture page: describes build lifecycle, core components,
provider system, workflow composition, hook system, configuration
resolution, remote client, CLI modes, and source code map
- Caching page: add Build Caching section explaining automatic
build output caching based on cache key
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix introduction diagram to list all supported CI platforms
The box previously said "GitHub Actions" which contradicted the
"Your Machine / CI" header. Now lists GitHub Actions, GitLab CI,
CLI, etc. to reflect that Orchestrator works from any entry point.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix ASCII diagram alignment in load balancing page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(orchestrator): build services — submodule profiles, caching, LFS, hooks
Adds a new advanced topics page documenting orchestrator build services:
- Submodule profiles (YAML, glob patterns, variant overlays)
- Local build caching (Library + LFS filesystem cache)
- Custom LFS transfer agents (elastic-git-storage, etc.)
- Git hooks (lefthook/husky detection, skip lists)
Related: game-ci/unity-builder#777
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(providers): add GCP Cloud Run and Azure ACI provider documentation
Covers storage type comparison tables, inputs, examples, and cross-links
to related providers. Both marked as experimental.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(orchestrator): CLI provider protocol documentation
Adds a new page documenting the CLI provider protocol that lets users
write orchestrator providers in any language (Go, Python, Rust, shell).
Covers: invocation model, JSON stdin/stdout protocol, streaming output,
subcommands with timeouts, shell example, CLI vs TypeScript comparison.
Related: game-ci/unity-builder#777
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(load-balancing): add built-in automatic fallback API section
Documents the new fallbackProviderStrategy, runnerCheckEnabled,
runnerCheckLabels, and runnerCheckMinAvailable inputs. Adds comparison
table for built-in vs manual fallback approaches.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(load-balancing): rewrite as comprehensive load balancing guide
Reframes page around intelligent provider routing with built-in API.
Adds retry-on-alternate, provider init timeout, async mode integration,
and decision table. Restructures manual scripting as secondary option.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(load-balancing): add workflow dispatch and reusable workflow routing examples
Add two new script-based routing patterns: dispatching to an alternate
workflow when self-hosted runners are busy, and using reusable workflows
for shared build config with dynamic provider routing. Updated the
comparison table with the new patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(secrets): comprehensive secret sources documentation
Expand the Secrets page with premade source documentation (AWS Secrets
Manager, AWS Parameter Store, GCP Secret Manager, Azure Key Vault, env),
custom commands, YAML definitions, and migration from legacy
inputPullCommand. Covers all five cloud providers and the env source.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(secrets): add HashiCorp Vault as premade secret source
Documents hashicorp-vault (KV v2), hashicorp-vault-kv1 (KV v1), and
vault (shorthand alias). Covers VAULT_ADDR, VAULT_TOKEN, and VAULT_MOUNT
configuration with examples for both KV versions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add Orchestrator Jobs page and Custom LFS Agents page
New Jobs page explains the build lifecycle, job types (build, test,
custom editor method, custom job, async), pre/post build phases, and
execution by provider. Gives users a conceptual overview before diving
into advanced topics.
New LFS Agents page documents elastic-git-storage built-in support with
auto-install, version pinning, multiple storage backends, and custom
agent configuration.
Renamed api-reference from 04 to 05 to accommodate the new Jobs page.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: test workflow engine and hot runner protocol
Add documentation for two new orchestrator features:
- Test Workflow Engine: YAML-based test suite definitions, taxonomy filters, structured results
- Hot Runner Protocol: extensible runner registration, persistent editor providers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add structured build output system page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: separate incremental sync protocol, update hot runner focus
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add massive projects and monorepo support advanced topics
Add two new documentation pages to the orchestrator advanced topics:
- 15-massive-projects.mdx: Two-level workspaces, move-centric caching,
custom LFS agents, and performance tips for 100GB+ projects
- 16-monorepo-support.mdx: Submodule profiles, variant overlays,
multi-product CI matrix, and framework configuration patterns
Closes game-ci/unity-builder#802
Closes game-ci/unity-builder#803
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add build reliability advanced topics page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add CI dispatch and infrastructure automation provider pages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(docs): add missing sidebar_position frontmatter to advanced topics pages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(orchestrator): Fix typos and formatting in documentation
- Change "first class" to "first-class" for proper hyphenation
- Update "cost saving" to "cost-saving" for consistency
- Add period after "e.g" to "e.g." in multiple locations
- Fix "effecient" to "efficient" spelling
- Fix "syncronization" to "synchronization" spelling
- Fix "signficantly" to "significantly" spelling
- Change "typescript" to "TypeScript" for proper capitalization
- Remove markdown code fence markers from status tables
- Reformat long command line example with line breaks for readability
- Fix "3Configuration" typo to "Configuration"
- Add missing comma in sentence for proper grammar
* Apply suggestion from @GabLeRoux
* docs(cli): add game-ci CLI documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cli): correct flag names, defaults, and coverage to match implementation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add integration branch update scripts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update for standalone @game-ci/orchestrator package
- Add standalone package callout to introduction
- Update external links to point to orchestrator repo
- Add standalone package note to getting started
- Update CLI docs to reference orchestrator package for installation
- Update version output and update command references
- Remove temporary delete-me scripts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add dedicated AWS and Kubernetes example pages
Restores dedicated example pages for AWS and Kubernetes that were
removed during the docs restructure. These complement the provider
reference pages with copy-paste workflow examples.
Related: game-ci/unity-builder#819, #541
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: restore versioned_docs that were accidentally deleted
These frozen versioned docs (version-2, version-3) should not be
modified by the orchestrator documentation update.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: resolve broken Docusaurus links
- Rename 06b-cli-provider-protocol.mdx to 12-cli-provider-protocol.mdx
(Docusaurus didn't recognize '06b' as a valid numeric prefix)
- Fix orchestrate-command link: ../providers → ../providers/overview
- Fix jobs link: advanced-topics/submodule-profiles → advanced-topics/build-services
- Fix build-services link: use relative path for cli-provider-protocol
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: replace npm install with GitHub Releases install scripts
Remove npm/npx installation references. Add PowerShell install script
for Windows. Fix install.sh URL to point to unity-builder repo where
the scripts live. Add environment variable options and manual download
section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update install URLs to point to orchestrator repo
Install scripts and releases live in game-ci/orchestrator, not
unity-builder. Updated all install URLs accordingly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: format orchestrator docs with prettier
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: remove em-dashes and convert ASCII diagram to mermaid
Replace all 336 em-dash characters with regular dashes across 37 docs
files. Convert remote-powershell ASCII box diagram to mermaid sequence
diagram.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: run prettier on documentation files
Fix formatting issues after em-dash removal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update CLI examples to use standalone orchestrator binary
Replace old "git clone unity-builder / yarn run cli" instructions with
the proper game-ci CLI install and usage from the orchestrator package.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add mermaid support and convert ASCII diagrams across orchestrator docs
- Enable @docusaurus/theme-mermaid for native diagram rendering
- Convert all ASCII box-drawing diagrams to mermaid flowcharts (22 files)
- Fix broken admonition syntax (:::info/:::caution blocks)
- Rewrite getting-started page with clear GitHub Actions and CLI sections
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: refine introduction and comparison pages for orchestrator
- Reposition orchestrator as advanced layer on top of unity-builder
- Emphasize benefits for projects of any size, not just large ones
- Add self-hosted runner complementarity (failover, load balancing)
- Expand "What Orchestrator Handles" with full lifecycle details
- Add "Choosing Your Setup" decision matrix to comparison page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: resolve cytoscape webpack export error and broken doc link
- Add webpack alias to redirect cytoscape UMD import to CJS bundle
- Fix broken markdown link to unity-builder (use GitHub URL)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove npm install step, add missing providers to overview
- Remove bogus npm install step from getting-started (orchestrator is
built into unity-builder, no separate install needed)
- Add dispatch, experimental, and additional providers to overview page
- Clarify orchestrator is built-in and activates via providerStrategy
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: improve test workflow engine taxonomy framing
Replace "Adolescent" maturity label with "Stable" for clearer
terminology. Rename "Built-in Dimensions" to "Example Dimensions"
and add extensibility note to emphasize the taxonomy is a starting
point that projects can fully customize.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document engine plugin system for game-engine agnostic orchestration
Add documentation for the new EnginePlugin interface that allows the
orchestrator to support non-Unity engines (Godot, Unreal, custom).
- New page: Advanced Topics > Engine Plugins — full guide covering the
interface, plugin sources (npm, CLI, Docker), and authoring plugins
- Updated introduction to mention engine agnosticism
- Updated caching page to reference engine-aware cache folders
- Added engine/enginePlugin to API reference parameters
- Added --engine and --engine-plugin to CLI build command docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: reframe orchestrator as hardware-agnostic, not cloud-first
Update introduction, getting-started, and comparison pages to describe
the orchestrator as taking whatever hardware you give it, rather than
framing it as three distinct types (cloud, self-hosted, local).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: replace "hardware" with "machines" for beginner friendliness
Update all instances across docs and versioned docs to use
"machines" instead of "hardware" for clearer, more approachable
language.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: replace em dashes with hyphens and fix remaining "hardware"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: revert circleci em dash changes, fix remaining hardware refs
Revert em dash changes outside orchestrator subfolder. Fix remaining
"hardware" references in orchestrator docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document AWS ECS 8192-byte containerOverrides limit and secret pulling workaround
Add troubleshooting entry for the Container Overrides 8192-byte limit
that AWS ECS/Fargate users can hit with complex workflows. Document the
connection between using secretSource/pullInputList and reducing the
override payload size. Cross-link from AWS provider docs and secrets docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: rename AWS section from Limitations to Troubleshooting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add reset button for maxed-out failed builds on versions page
Show failure count (e.g., "15/15") next to failed build status icons.
Add a reset button (admin-only) that calls the new resetFailedBuilds
backend endpoint to clear inflated failure counts so the Ingeminator
can retry them automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use destructured meta variable to satisfy lint rule
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: move troubleshooting entry and reset button UI to dedicated PR #548
The 8192 troubleshooting entry and build reset button UI changes are
now in a separate PR (#548) targeting main.
The orchestrator-specific docs (AWS provider troubleshooting section
and secrets tip callout) remain here since those files only exist on
this branch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: restore 8192 troubleshooting entry for orchestrator LTS 2.0.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update docs/03-github-orchestrator/01-introduction.mdx
Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>
* Update docs/03-github-orchestrator/01-introduction.mdx
Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>
* Update docs/03-github-orchestrator/02-getting-started.mdx
Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>
* docs: address Gabe's review — provider tabs, OIDC auth, SSO note, GC clarification
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: improve garbage-collect description with safety warning
- Note that the base "game-ci" stack is preserved
- Remove mention of --garbageMaxAge (not actually wired in AWS provider)
- Add caution admonition about no dry-run/confirmation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "docs: improve garbage-collect description with safety warning"
This reverts commit 66a738d.
* docs: add Unity Accelerator integration guide
Covers two approaches:
- Sidecar (per-build): start/stop accelerator via container hooks,
persist cache to S3 between builds
- Persistent (shared): always-on EC2/ECS instance in same VPC
Includes full hook YAML examples, workflow config, troubleshooting,
and guidance on combining with Library caching.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add cache checkpointing and survival guide
Documents three new caching features:
- cacheCheckpointInterval: periodic Library saves during build
- cacheSaveOnFailure: trap-based partial save on OOM/crash
- cacheRetentionDays: auto-purge old S3 entries
Includes decision tables, Mermaid diagrams, and guidance on
combining with Unity Accelerator for maximum resilience.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add cache pre-warming examples to checkpointing guide
Shows three approaches: local tar + S3 upload, CLI cache-push,
and one-time local-docker build to seed the cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: align mermaid dependencies with lockfile
* fix: avoid mermaid policy-blocked dependencies
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>
This was referenced May 5, 2026
This was referenced May 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Orchestrator Extraction — Remove Orchestrator Code, Add Plugin Loader
Extracts the orchestrator subsystem from unity-builder into the standalone
game-ci/orchestratorrepository. Unity-builder becomes a lean GitHub Action; all orchestrator, CLI, and cloud provider features live in the standalone repo.Why
src/model/orchestrator/BuildParametersis a god object mixing both concernsWhat Changes for Users
The orchestrator is now a separate package that lives in its own repository (
game-ci/orchestrator). What this means depends on your build strategy:Local builds (
providerStrategy: local) — No change requiredIf you use the default local Docker builds, nothing changes. Unity-builder works standalone with no extra install step:
Cloud builds (
providerStrategy: aws,k8s, etc.) — Install the orchestrator firstFor cloud provider strategies, you now need to install the orchestrator before running unity-builder. The install adds the
game-ciCLI and the@game-ci/orchestratorpackage so that unity-builder's plugin loader can detect it automatically.GitHub Actions example:
That is all that changes: one extra step to install the orchestrator before the build step. Unity-builder automatically detects the orchestrator via its plugin loader and enables cloud providers and all services.
Install commands by platform
curl -fsSL https://raw.githubusercontent.com/game-ci/orchestrator/main/install.sh | shirm https://raw.githubusercontent.com/game-ci/orchestrator/main/install.ps1 | iexStandalone CLI usage (optional)
The orchestrator can also be used directly from the command line without GitHub Actions:
What This PR Does
118 files changed, 3,741 insertions, 15,404 deletions
Deleted
src/model/orchestrator/directory (148.tsfiles)src/cli.tsandsrc/cli/(CLI moved to orchestrator repo)src/integration/orchestrator-github-checks-integration-test.tssrc/test-utils/orchestrator-test-helpers.ts.github/workflows/orchestrator-async-checks.yml.github/workflows/orchestrator-integrity.yml.github/workflows/release-cli.yml@aws-sdk/*(5),@kubernetes/client-node,async-wait-until,aws-sdk,base-64,kubernetes-client,shell-quote,uuidyargs,@types/yargs,pkginstall.sh,install.ps1) — moved to orchestrator repoAdded
src/model/orchestrator-plugin.ts): Dynamicimport('@game-ci/orchestrator')with graceful degradationsrc/types/game-ci-orchestrator.d.ts): Ambient module for optional@game-ci/orchestratorpackagesrc/model/orchestrator-plugin.test.ts): 15 tests covering both installed and not-installed paths.github/workflows/validate-orchestrator.yml): Per-PR health checks — builds both repos, tests plugin loader, verifies type declarations.github/workflows/validate-orchestrator-integration.yml): Nightly exhaustive suite — k8s (k3d), AWS (LocalStack), local-docker, rcloneChanged
build-parameters.ts: Replaced 44OrchestratorOptions.*withInput.getInput(), inlined constantsgithub.ts: Stripped to minimal classinput.ts: RemovedOrchestratorQueryOverrideinput-readers/: ReplacedOrchestratorSystem.Runwithchild_process.execindex.ts: Plugin loader pattern for servicesHow the Plugin Loader Works
Test Results
dist/bundle rebuilt (~4MB vs ~15MB before)Orchestrator Repo Status
Checklist