[Spike] Pipeline generation: Core abstractions and GitHub Actions scheduling#15643
Draft
mitchdenny wants to merge 15 commits intomainfrom
Draft
[Spike] Pipeline generation: Core abstractions and GitHub Actions scheduling#15643mitchdenny wants to merge 15 commits intomainfrom
mitchdenny wants to merge 15 commits intomainfrom
Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15643Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15643" |
Contributor
|
Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
|
Contributor
|
Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
|
…uling Add core pipeline environment abstractions (IPipelineEnvironment, IPipelineStepTarget, PipelineEnvironmentCheckAnnotation) to Aspire.Hosting, along with a new Aspire.Hosting.Pipelines.GitHubActions package implementing workflow resource model and scheduling resolver. Key changes: - IPipelineEnvironment marker interface for CI/CD environments - IPipelineStepTarget for scheduling steps onto workflow jobs - PipelineStep.ScheduledBy property for step-to-job assignment - GetEnvironmentAsync() with annotation-based environment resolution - GitHubActionsWorkflowResource and GitHubActionsJob types - SchedulingResolver projecting step DAG onto job dependency graph - 29 unit tests covering environment resolution and scheduling Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add WorkflowYaml model types (WorkflowYaml, JobYaml, StepYaml, etc.) - Add hand-rolled WorkflowYamlSerializer (no external dependencies) - Add WorkflowYamlGenerator: scheduling result → complete workflow YAML - Boilerplate steps: checkout, setup-dotnet, install Aspire CLI - State artifact upload/download between dependent jobs - Per-job aspire do --continue --job <jobId> execution - Add TryRestoreStepAsync on PipelineStep for CI/CD state restore - Executor calls restore callback before Action - If restore returns true, step is skipped (already completed) - Add 9 YAML generator unit tests - Add 4 Verify snapshot tests with complete YAML output - Add 5 step state restore integration tests - Update spec doc with YAML generation and state restore design Total: 47 tests passing (12 hosting + 35 GitHub Actions) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds ScheduleStep(string stepName, IPipelineStepTarget target) to IDistributedApplicationPipeline. This allows consumers to schedule built-in steps (e.g., WellKnownPipelineSteps.Build) onto CI/CD jobs without having to create them — useful when integrations register steps and the AppHost just needs to assign them to workflow jobs. - Interface and implementation in Aspire.Hosting - 7 unit tests: basic scheduling, override, built-in steps, null guards, error on missing step Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert IDistributedApplicationPipeline to its original signature to preserve binary compatibility. The interface method AddStep retains its original 4-parameter signature. New members (ScheduleStep, GetEnvironmentAsync, and the 5-param AddStep overload with scheduledBy) are on DistributedApplicationPipeline only. Adding an optional parameter to an interface method changes the IL signature, which is a binary breaking change even though it's source compatible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mitchdenny
force-pushed
the
spike/pipeline-generation
branch
from
d1185d9 to
3a360ae
Compare
…urce
- GitHubActionsJob → GitHubActionsJobResource (extends Resource)
- New GitHubActionsStageResource (extends Resource) for logical job grouping
- workflow.AddStage("build-stage") → stage.AddJob("build")
- workflow.AddJob("build") still works for direct job creation
- Jobs created via stages are also registered on the workflow
- 6 new tests for stage/resource APIs
- Total: 60 tests passing
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…jobs
- GitHubActionsWorkflowResource and GitHubActionsStageResource now
implement IPipelineStepTarget, so steps can be scheduled onto them
- When a step targets a workflow, a default stage + default job are
auto-created to host it
- When a step targets a stage, a default job is auto-created within
that stage (named '{stage}-default')
- SchedulingResolver.ResolveJobForStep uses pattern matching to
resolve workflow/stage/job targets down to concrete jobs
- GetOrAddDefaultStage() on workflow, GetOrAddDefaultJob() on stage
- DefaultJob in SchedulingResult is now nullable (not created when
all steps are explicitly scheduled)
- 6 new resolver tests covering workflow/stage scheduling targets
- Total: 66 tests passing (19 hosting + 47 GH Actions)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- PipelineCommand: group command (like 'mcp') at 'aspire pipeline' - PipelineInitCommand: subcommand extending PipelineCommandBase - Passes --operation publish --step pipeline-init to AppHost - Follows same pattern as deploy/publish commands - Added WellKnownPipelineSteps.PipelineInit constant - Registered in DI (Program.cs) and RootCommand - Resource strings (.resx + .Designer.cs) with localization xlf files - HelpGroup.Deployment for discoverability Usage: aspire pipeline init [--apphost <path>] [--output-path <path>] Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix DI resolution failure: 'Unable to resolve service for type PipelineCommand while attempting to activate RootCommand'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Register a built-in 'pipeline-init' step in DistributedApplicationPipeline that discovers IPipelineEnvironment resources and calls their PipelineWorkflowGeneratorAnnotation to generate CI/CD workflow files. - Add PipelineWorkflowGeneratorAnnotation and PipelineWorkflowGenerationContext - Register generator annotation in AddGitHubActionsWorkflow extension - Pipeline-init step iterates environments and invokes generators Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
🎬 CLI E2E Test Recordings — 52 recordings uploaded (commit View recordings
📹 Recordings uploaded automatically from CI run #23682876756 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Spike/proof-of-concept for pipeline generation — the ability to generate CI/CD workflow YAML files (e.g., GitHub Actions) from the Aspire application model.
What's Included
Core Abstractions (
Aspire.Hosting)IPipelineEnvironment— Marker interface extendingIResourcefor CI/CD environmentsIPipelineStepTarget— Interface for scheduling targets (workflow jobs implement this)PipelineEnvironmentCheckAnnotation— Annotation-based environment relevance checkLocalPipelineEnvironment— Internal fallback for local executionPipelineStep.ScheduledBy— Property linking steps to workflow jobsGetEnvironmentAsync()— Resolution of active pipeline environment from modelGitHub Actions Package (
Aspire.Hosting.Pipelines.GitHubActions)GitHubActionsWorkflowResource—Resource + IPipelineEnvironmentrepresenting a workflowGitHubActionsJob— Job within a workflow, implementsIPipelineStepTargetGitHubActionsWorkflowExtensions—builder.AddGitHubActionsWorkflow(name)extensionSchedulingResolver— Projects step DAG onto job dependency graph with cycle detectionTests (29 total)
Spec Document
docs/specs/pipeline-generation.md— Full architecture, API primitives, and future roadmapExample Usage
Not Yet Implemented
aspire pipeline init)aspire do --continue --job <jobId>CLI supportRelated
See
docs/specs/pipeline-generation.mdfor full design details and open questions.