feat(provision): provider-agnostic provision validation checks#9019
Merged
Conversation
Introduce a new provider-agnostic "provision" validation check type that is dispatched from provisioning.Manager.Deploy/Preview before any provider runs. This lets extension-provided provisioning providers (e.g. microsoft.foundry) participate in client-side validation, not just the Bicep provider's existing "local-preflight" checks. - Add ValidationCheckTypeProvision / ValidationCheckTypeLocalPreflight constants plus a lean provision context (env_name, subscription_id, resource_group, target_scope) and accessors in pkg/azdext. - Add Manager.runProvisionValidation dispatched from Deploy and Preview, with abort/error semantics (ErrProvisionValidationAborted) and reuse of the existing preflight telemetry and config gate (provision.preflight). - Translate ErrProvisionValidationAborted -> ErrAbortedByUser in wrapProvisionError. - Update the demo extension to register BOTH check types: the Bicep-only "local-preflight" (demo_warning) and the new agnostic "provision" (demo_provision_warning). - Add unit tests and update design/extension-framework docs + CHANGELOG. Resolves #9016 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…9019) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a provider-agnostic provision validation check type so that extension-registered client-side checks can run before any provisioning provider executes — not just the built-in Bicep provider. Previously DispatchChecks was only called from BicepProvider.validatePreflight, so checks registered by extension-provided providers (microsoft.foundry, demo) or the core Terraform provider were effectively dead code. The new dispatch is inserted into the provider-agnostic provisioning.Manager.Deploy/Preview, immediately before the selected provider runs, and reuses the existing preflight report, severity/abort semantics, telemetry event, and provision.preflight config gate. This unblocks issue #9016 (the follow-up AI-extension check is tracked separately).
Changes:
- New
runProvisionValidationinManager.Deploy/Previewthat dispatches the"provision"check type with a lean, ARM-free context (env name, subscription, location, resource group, target scope), plusErrProvisionValidationAbortedmapped tointernal.ErrAbortedByUser. - New
azdextconstants (ValidationCheckTypeProvision,ValidationCheckTypeLocalPreflight, provision context keys) and typed accessors onValidationContext. - Demo extension registers both check types as a reference, with new checks/tests and updated design + extension-framework docs and changelog.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
cli/azd/pkg/infra/provisioning/provision_validation.go |
Core provider-agnostic dispatch, lean context builder, config gate, telemetry, abort sentinel. |
cli/azd/pkg/infra/provisioning/manager.go |
Invokes runProvisionValidation at the start of Deploy and Preview. |
cli/azd/pkg/infra/provisioning/provision_validation_test.go |
Unit tests for pass/warning/declined/error/no-dispatcher/dispatch-error outcomes. |
cli/azd/pkg/azdext/validation_provider.go |
New check-type constants, provision context-key constants, and accessors. |
cli/azd/internal/cmd/provision_graph.go |
Maps ErrProvisionValidationAborted to the aborted-by-user path in wrapProvisionError. |
cli/azd/internal/cmd/errors_test.go |
Records ErrProvisionValidationAborted in the package-level error mapping. |
cli/azd/extensions/microsoft.azd.demo/.../demo_provision_validation_check.go (+test) |
Sample "provision" check reading the lean context. |
cli/azd/extensions/microsoft.azd.demo/.../demo_validation_check_test.go |
Switches literals to the new ValidationCheckTypeLocalPreflight constant. |
cli/azd/extensions/microsoft.azd.demo/internal/cmd/listen.go |
Registers both local-preflight and provision demo checks. |
cli/azd/docs/extensions/extension-framework.md, cli/azd/docs/design/local-preflight-validation.md |
Document the new provision check type and shared behavior. |
cli/azd/CHANGELOG.md |
Feature entry for the new provider-agnostic validation check type. |
One consideration flagged inline: the reused PreflightValidationEvent now fires from two sites for Bicep provisions when a validation-provider extension is loaded, which can double-count telemetry outcomes without an attribute distinguishing the dispatch sites. Otherwise the abort contract correctly reuses the existing PreflightAbortedSkipped handling that all Deploy callers already honor, and the change is well-tested and documented.
…teration 1) Address Copilot review feedback: the provider-agnostic "provision" validation and the Bicep "local-preflight" validation both emit PreflightValidationEvent, so a Bicep provision with a validation-provider extension loaded emitted the event twice with no way to tell the two apart (double-counting downstream). Add a new SystemMetadata/FeatureInsight field `validation.preflight.check_type` (fixed enum: "local-preflight" | "provision") set at both dispatch sites, and document it across the telemetry reference and the metrics-audit schema/matrix (including a query gotcha). Also replace the "local-preflight" magic string in the Bicep dispatch with the azdext constant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…teration 2) Address Copilot review feedback: when extension dispatch fails or times out and returns no results, runProvisionValidation recorded the preflight outcome as "passed", conflating a dispatch failure with a successful validation. Record provisionValidationOutcomeError when dispatchErr is non-nil so telemetry reflects that the checks did not actually run. Provisioning still proceeds (dispatch errors remain non-fatal). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address Copilot review feedback: the "Shared Behavior" section named the Bicep-only extensionValidationTimeout constant when describing the provision path, which is misleading. Reference provisionValidationTimeout (the constant the provider-agnostic path actually uses) while noting it mirrors the Bicep bound. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
ext-agents
…teration 4)
Address Copilot review feedback: runProvisionValidation lived at the top of
Manager.Deploy, but multi-layer provisioning (infra.layers[]) runs each layer's
Deploy through its own Manager concurrently. That dispatched the provider-
agnostic "provision" checks once per layer with an identical env-scoped context
and, for a warning, prompted the user once per layer.
Move the dispatch out of Manager.Deploy/Preview and into the command layer so it
runs exactly once per `azd provision` / `azd up`, before the layer graph:
- Export Manager.RunProvisionValidation (was runProvisionValidation).
- Remove the per-Deploy / per-Preview dispatch from Manager.
- Call it once in provisionLayersGraph (covers preview + single/multi-layer) and
once in the up graph (guarded on len(layers) > 0), surfacing aborts through
the shared wrapProvisionError ("Provisioning was cancelled.").
- Update unit tests to exercise RunProvisionValidation directly and refresh the
design doc's dispatch diagram/description.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address Copilot review feedback:
- ValidationContext.ResourceGroup() doc claimed it returns ("", false) for
subscription-scoped deployments, but the "provision" dispatch always includes
the resource group key (empty value for subscription scope), so ok is always
true. Corrected the comment and pointed authors at TargetScope() to
distinguish scopes.
- Added TestManagerDeployProvisionValidation_SubscriptionScoped covering the
no-AZURE_RESOURCE_GROUP path: asserts the resource group key is present but
empty and target_scope == "subscription".
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📋 Prioritization NoteThanks for the contribution! The linked issue isn't in the current milestone yet. |
JeffreyCA
reviewed
Jul 8, 2026
jongio
approved these changes
Jul 8, 2026
hemarina
approved these changes
Jul 8, 2026
hemarina
left a comment
hemarina
left a comment
Contributor
There was a problem hiding this comment.
Nicely executed and thoroughly documented. Hoisting dispatch to the command layer so the provision checks (and any warning prompt) fire once per command rather than once per concurrently-provisioned layer is the right call, and the reasoning is captured well in the code comments. The validation.preflight.check_type field cleanly resolves the shared-event double-count, and the docs/tests are in good shape.
Things I verified that are solid:
- No double-dispatch —
provisionLayersGraphis only theprovisionentry point;upcallsRunProvisionValidationexactly once. - Both dispatch sites are guarded on non-empty layers (up via
len(layers) > 0, provision via thelen(layers)==0early return). - Severity classification matches Bicep's existing logic, and
ErrProvisionValidationAbortedis correctly mapped toErrAbortedByUser(exit 0) and covered byTest_PackageLevelErrorsMapped.
One thing to resolve before merge — +1 to @JeffreyCA's cold-run context concern: because dispatch runs before the provider resolves subscription/location/resource group, a first-time provision/up ships empty subscription_id/env_location/resource_group, and TargetScope() (inferred only from AZURE_RESOURCE_GROUP) can be wrong on cold runs. Either resolve the deployment context before dispatch, or document the provision context as best-effort/ambient and soften the TargetScope()/ResourceGroup() doc comments so they don't read as authoritative.
Otherwise this looks good.
richardpark-msft
left a comment
richardpark-msft
left a comment
Member
There was a problem hiding this comment.
I don't have enough domain knowledge to approve this, but I had some comments.
…lues Address review feedback (JeffreyCA): the provider-agnostic "provision" dispatch runs before the provider resolves/prompts for subscription, location, and resource group, so on a cold first-time run those values can be empty and the inferred target scope is not authoritative. Document the context and its accessors (SubscriptionID, EnvLocation, ResourceGroup, TargetScope) as best-effort ambient env values so extension check authors tolerate empty / not-yet-authoritative values instead of misfiring. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…err) Address review feedback (richardpark-msft): the (abort bool, err error) pair was redundant — both callers already treated abort as "return ErrProvisionValidationAborted through wrapProvisionError". Collapse to a single error return: - ErrProvisionValidationAborted for an error-severity finding or a declined warning (wrapProvisionError maps it to internal.ErrAbortedByUser, exit 0); - the wrapped prompt error for a confirmation-prompt failure; - nil for pass / skip / warnings-accepted. Both callers now funnel every non-nil error through wrapProvisionError, centralizing the abort-to-cancelled mapping. Updated unit tests and the design doc accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JeffreyCA
approved these changes
Jul 8, 2026
…shape Address review feedback (richardpark-msft): the "lean provision context" was only described informally in the demo check. Document on the ValidationContext type itself how Data contents depend on CheckType — listing the ARM-rich accessors for "local-preflight" and the lean, best-effort accessors for "provision" — and trim the demo comment to reference the type as the source of truth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address review feedback (richardpark-msft): "carries no ARM template" implied only the template was missing. Spell out that the lean context omits all ARM-derived data (template, resolved parameters, resources snapshot, predicted resources) that "local-preflight" carries — because non-ARM providers don't produce it — and carries only ambient environment values. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
enabled auto-merge (squash)
July 9, 2026 01:09
richardpark-msft
approved these changes
Jul 9, 2026
richardpark-msft
left a comment
richardpark-msft
left a comment
Member
There was a problem hiding this comment.
Looks good! I know there's more work to do, but no reason this can't go out.
Contributor
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
vhvb1989
added a commit
that referenced
this pull request
Jul 9, 2026
The resource-group location-mismatch check was registered under the Bicep-only 'local-preflight' validation type, which is never dispatched when the azure.ai.agents extension provisions through its own microsoft.foundry provider, so the check was dead code. Switch it to the new provider-agnostic 'provision' validation check type (ValidationCheckTypeProvision, added in #9019), which core dispatches once before provisioning regardless of the provisioning provider, and declare the 'validation-provider' capability so registration is accepted. The check now prefers the lean provision context values (subscription, resource group) with robust fallbacks to the azd environment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 9, 2026
The resource-group location-mismatch check was registered under the Bicep-only 'local-preflight' validation type, which is never dispatched when the azure.ai.agents extension provisions through its own microsoft.foundry provider, so the check was dead code. Switch it to the new provider-agnostic 'provision' validation check type (ValidationCheckTypeProvision, added in #9019), which core dispatches once before provisioning regardless of the provisioning provider, and declare the 'validation-provider' capability so registration is accepted. The check now prefers the lean provision context values (subscription, resource group) with robust fallbacks to the azd environment. Because the 'provision' check type is dispatched for every project whenever the extension is installed, guard the check so it only runs when the current project actually provisions through microsoft.foundry (azure.yaml infra.provider). For any other provider an existing resource group in a different region is valid, so running the check there would be a false positive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 9, 2026
- Bump requiredAzdVersion to >=1.27.1. The provider-agnostic provision validation dispatch this check relies on (#9019) shipped in azd 1.27.1; it is absent from 1.27.0, where the check would silently no-op. - Skip the resource-group location check for brownfield (endpoint:) Foundry projects. When the Foundry service sets endpoint:, the microsoft.foundry provider connects to an existing project and provisions nothing, deriving its target from AZURE_AI_PROJECT_ID and ignoring AZURE_RESOURCE_GROUP. Running the check there could compare AZURE_LOCATION against an unrelated same-named resource group and wrongly block provisioning. The new isBrownfieldFoundryProject guard reuses findFoundryProjectService + foundryServiceEndpoint so it stays in sync with the provider. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 9, 2026
The resource-group location-mismatch check was registered under the Bicep-only 'local-preflight' validation type, which is never dispatched when the azure.ai.agents extension provisions through its own microsoft.foundry provider, so the check was dead code. Switch it to the new provider-agnostic 'provision' validation check type (ValidationCheckTypeProvision, added in #9019), which core dispatches once before provisioning regardless of the provisioning provider, and declare the 'validation-provider' capability so registration is accepted. The check now prefers the lean provision context values (subscription, resource group) with robust fallbacks to the azd environment. Because the 'provision' check type is dispatched for every project whenever the extension is installed, guard the check so it only runs when the current project actually provisions through microsoft.foundry (azure.yaml infra.provider). For any other provider an existing resource group in a different region is valid, so running the check there would be a false positive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 9, 2026
- Bump requiredAzdVersion to >=1.27.1. The provider-agnostic provision validation dispatch this check relies on (#9019) shipped in azd 1.27.1; it is absent from 1.27.0, where the check would silently no-op. - Skip the resource-group location check for brownfield (endpoint:) Foundry projects. When the Foundry service sets endpoint:, the microsoft.foundry provider connects to an existing project and provisions nothing, deriving its target from AZURE_AI_PROJECT_ID and ignoring AZURE_RESOURCE_GROUP. Running the check there could compare AZURE_LOCATION against an unrelated same-named resource group and wrongly block provisioning. The new isBrownfieldFoundryProject guard reuses findFoundryProjectService + foundryServiceEndpoint so it stays in sync with the provider. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 13, 2026
…e.ai.agents) (#9007) * Add resource group location mismatch preflight check (azure.ai.agents) Adds a local-preflight validation check to the azure.ai.agents extension that detects an immutable resource-group region conflict before provisioning. When the target resource group already exists in a different region than AZURE_LOCATION, the subscription-scoped Bicep deployment fails with the ARM error "InvalidResourceGroupLocation" (the #1 production error). This check surfaces the conflict during azd's validation phase with a clear message and remediation guidance instead of a slow deploy-time failure. Registered via the extension WithValidationCheck capability (RuleID resource_group_location_mismatch, severity Error). The check is best-effort: it skips silently on missing env values, auth failures, a 404 (resource group not created yet), or any API error. Fixes #9006 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Register RG location check under provider-agnostic provision type The resource-group location-mismatch check was registered under the Bicep-only 'local-preflight' validation type, which is never dispatched when the azure.ai.agents extension provisions through its own microsoft.foundry provider, so the check was dead code. Switch it to the new provider-agnostic 'provision' validation check type (ValidationCheckTypeProvision, added in #9019), which core dispatches once before provisioning regardless of the provisioning provider, and declare the 'validation-provider' capability so registration is accepted. The check now prefers the lean provision context values (subscription, resource group) with robust fallbacks to the azd environment. Because the 'provision' check type is dispatched for every project whenever the extension is installed, guard the check so it only runs when the current project actually provisions through microsoft.foundry (azure.yaml infra.provider). For any other provider an existing resource group in a different region is valid, so running the check there would be a false positive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: reuse defaultResourceGroupName and trim env values - Use defaultResourceGroupName(envName) for the AZURE_RESOURCE_GROUP default instead of an inline "rg-<env>" literal, keeping the check in sync with the foundry provisioning path. - Rename the package-level envValue helper to envValueOrEmpty and trim the returned value (strings.TrimSpace), matching FoundryProvisioningProvider.envValue. This avoids a false-positive InvalidResourceGroupLocation block when AZURE_LOCATION or AZURE_RESOURCE_GROUP has stray leading/trailing whitespace (the region comparison uses strings.EqualFold, which does not trim). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: bump requiredAzdVersion and skip brownfield projects - Bump requiredAzdVersion to >=1.27.1. The provider-agnostic provision validation dispatch this check relies on (#9019) shipped in azd 1.27.1; it is absent from 1.27.0, where the check would silently no-op. - Skip the resource-group location check for brownfield (endpoint:) Foundry projects. When the Foundry service sets endpoint:, the microsoft.foundry provider connects to an existing project and provisions nothing, deriving its target from AZURE_AI_PROJECT_ID and ignoring AZURE_RESOURCE_GROUP. Running the check there could compare AZURE_LOCATION against an unrelated same-named resource group and wrongly block provisioning. The new isBrownfieldFoundryProject guard reuses findFoundryProjectService + foundryServiceEndpoint so it stays in sync with the provider. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: trim context values, namespace rule ID, quote RG, add Validate tests - Trim the provision-context values (SubscriptionID, EnvLocation, ResourceGroup) before the emptiness/fallback checks. The context accessors return values verbatim, so a persisted value with surrounding whitespace previously bypassed the trimmed envValueOrEmpty fallback — causing a false region mismatch or silently skipping the check while the Foundry provider (which trims) would provision into the real group. - Namespace the rule ID as azure.ai.agents.resource_group_location_mismatch. The provision validation rule namespace is global (core enforces uniqueness on check_type + rule_id across all extensions), so prefixing with the extension id keeps the rule mappable to its owner for future suppressions / rule sets. Core follow-up tracked in #9060. - Shell-quote the resource-group name (%q) in the 'az group delete' remediation command so a name containing shell metacharacters (parentheses are valid in Azure RG names) stays copy-pasteable. Added a regression test. - Add provider-level Validate tests: refactor the ARM resource-group lookup into an injectable field and cover the non-Foundry gate, brownfield (endpoint) gate, missing-values skip, mismatch -> error, match -> no result, best-effort no-op on lookup failure / RG-not-found, context trimming, env fallback, and default RG name, using in-process gRPC stubs for the project/environment services. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 14, 2026
…ion" (#7113) Rename the client-side, pre-deployment check feature from "local preflight" to "provision validation" to avoid confusion with the Azure ARM Preflight API. User-facing changes: - Split config gates: new `validation.provision` key gates the client-side (local) checks; existing `provision.preflight` now gates ONLY the server-side ARM preflight call (previously a single key disabled both). - Rename the Bicep-only extension check type `local-preflight` -> `arm-provision`; the provider-agnostic `provision` check type (added in #9019) is unchanged. - Telemetry event `validation.preflight` -> `validation.provision`, fields `validation.preflight.*` -> `validation.provision.*`. - Outcome vocabulary `aborted_by_*` -> `canceled_by_*`; "abort" UI language replaced with "canceled". - `DeploymentStateSkipped` value corrected to lowercase "deployment state". Internal renames: PreflightReport -> ProvisionValidationReport, local_preflight.go -> provision_validation.go, PreflightCheck* -> ValidationCheck*, newLocalArmPreflight -> newProvisionValidator, PreflightValidationEvent -> ProvisionValidationEvent, Preflight*Key -> ProvisionValidation*Key, ErrProvisionValidationAborted -> ErrProvisionValidationCanceled, and related test/recording/snapshot renames. Server-side ARM preflight (`ValidatePreflight`), `mage preflight` dev checks, and generic third-party "preflight" usages are intentionally left unchanged. Closes #7113 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 16, 2026
…ion" (#7113) Rename the client-side, pre-deployment check feature from "local preflight" to "provision validation" to avoid confusion with the Azure ARM Preflight API. User-facing changes: - Split config gates: new `validation.provision` key gates the client-side (local) checks; existing `provision.preflight` now gates ONLY the server-side ARM preflight call (previously a single key disabled both). - Rename the Bicep-only extension check type `local-preflight` -> `arm-provision`; the provider-agnostic `provision` check type (added in #9019) is unchanged. - Telemetry event `validation.preflight` -> `validation.provision`, fields `validation.preflight.*` -> `validation.provision.*`. - Outcome vocabulary `aborted_by_*` -> `canceled_by_*`; "abort" UI language replaced with "canceled". - `DeploymentStateSkipped` value corrected to lowercase "deployment state". Internal renames: PreflightReport -> ProvisionValidationReport, local_preflight.go -> provision_validation.go, PreflightCheck* -> ValidationCheck*, newLocalArmPreflight -> newProvisionValidator, PreflightValidationEvent -> ProvisionValidationEvent, Preflight*Key -> ProvisionValidation*Key, ErrProvisionValidationAborted -> ErrProvisionValidationCanceled, and related test/recording/snapshot renames. Server-side ARM preflight (`ValidatePreflight`), `mage preflight` dev checks, and generic third-party "preflight" usages are intentionally left unchanged. Closes #7113 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vhvb1989
added a commit
that referenced
this pull request
Jul 16, 2026
* feat: rename client-side "preflight" validation to "provision validation" (#7113) Rename the client-side, pre-deployment check feature from "local preflight" to "provision validation" to avoid confusion with the Azure ARM Preflight API. User-facing changes: - Split config gates: new `validation.provision` key gates the client-side (local) checks; existing `provision.preflight` now gates ONLY the server-side ARM preflight call (previously a single key disabled both). - Rename the Bicep-only extension check type `local-preflight` -> `arm-provision`; the provider-agnostic `provision` check type (added in #9019) is unchanged. - Telemetry event `validation.preflight` -> `validation.provision`, fields `validation.preflight.*` -> `validation.provision.*`. - Outcome vocabulary `aborted_by_*` -> `canceled_by_*`; "abort" UI language replaced with "canceled". - `DeploymentStateSkipped` value corrected to lowercase "deployment state". Internal renames: PreflightReport -> ProvisionValidationReport, local_preflight.go -> provision_validation.go, PreflightCheck* -> ValidationCheck*, newLocalArmPreflight -> newProvisionValidator, PreflightValidationEvent -> ProvisionValidationEvent, Preflight*Key -> ProvisionValidation*Key, ErrProvisionValidationAborted -> ErrProvisionValidationCanceled, and related test/recording/snapshot renames. Server-side ARM preflight (`ValidatePreflight`), `mage preflight` dev checks, and generic third-party "preflight" usages are intentionally left unchanged. Closes #7113 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR review feedback on provision-validation rename - errors.go: give ErrAbortedByUser a distinct message ("operation declined by user") so it no longer collides with ErrOperationCancelled in logs/output (jongio). - provision_graph.go / up_graph.go: standardize user-facing message and comments on American spelling "canceled" (was "cancelled"). - provision_validation.go: emit the empty/zero validation.provision.* field shape (diagnostics + warning/error counts) in the no-results branch so telemetry is consistent with the Bicep "arm-provision" dispatch. - bicep_provider_test.go: rename table-test cases "aborted by *" -> "canceled by *" to match the canceled_by_* outcomes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: cover provision-validation gates + telemetry; emit empty agnostic rules - Add table-driven test for the config-gate split (validation.provision vs provision.preflight) and ARM-preflight invocation gating in the Bicep provider. - Add agnostic disabled-path coverage: validation.provision=off short-circuits dispatch; provision.preflight=off does not disable the client-side path. - Add ProvisionValidationFields subtest to the telemetry field-constants contract. - Emit an explicit empty validation.provision.rules slice on the agnostic path for consistent telemetry field shape. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e14c6401-fc80-4458-91e5-6bbcd0e045d0 * test: use t.Context() in provision validation gate test Address PR review feedback: tie context cancellation/cleanup to the test per the repo's modern Go test conventions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e14c6401-fc80-4458-91e5-6bbcd0e045d0 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #9016.
Adds a new provider-agnostic
provisionvalidation check type that is dispatched once perazd provision/azd up, from the command layer, before the layer graph runs — independently of which provisioning provider is selected. Previously, client-side validation checks (local-preflight) were dispatched only by the built-in Bicep provider, so extension-provided provisioning providers (e.g.microsoft.foundry, and future AI-extension checks) could never participate in validation.This unblocks adding a validation check type in the AI extension (the follow-up in #9016 is tracked separately).
What changed
ValidationCheckTypeProvision(and namedValidationCheckTypeLocalPreflight) inpkg/azdext, plus a lean provision context (env_name,subscription_id,resource_group,target_scope, reusingenv_location) with accessorsEnvName(),SubscriptionID(),ResourceGroup(),TargetScope(). No ARM template is assumed (Terraform/foundry have none).Manager.RunProvisionValidationis invoked once fromprovisionLayersGraph(covers--preview+ single/multi-layer) and once from the up graph (guarded onlen(layers) > 0), before the layer execution graph. It is not called fromManager.Deploy/Preview— multi-layer provisioning runs each layer'sDeploythrough its ownManagerconcurrently, so dispatching per-Deploywould fire the checks (and any warning prompt) once per layer. Bicep still also runs its ownlocal-preflightchecks internally — different check type, no conflict.ErrProvisionValidationAborted; on abort (error-severity finding or a declined warning)RunProvisionValidationreturnsabort=true, and the action surfaces the sentinel throughwrapProvisionError, which maps it tointernal.ErrAbortedByUser(exit code 0). Bicep's own provider-level preflight abort (PreflightAbortedSkipped) is a separate, still-handled path.validation.preflightevent and its outcome/count/rule fields, and the existingprovision.preflight = offconfig gate. Because theprovisionand Biceplocal-preflightdispatches share the same event (and both fire on a Bicep provision), this PR adds one new field,validation.preflight.check_type(local-preflight|provision), emitted at both dispatch sites so downstream consumers can distinguish them and avoid double-counting. The new field is documented acrossdocs/reference/telemetry-data.md,docs/specs/metrics-audit/telemetry-schema.md, anddocs/specs/metrics-audit/feature-telemetry-matrix.md. The field isSystemMetadata/FeatureInsightand not user-derived, so it is not hashed and needs no privacy-hashing-table entry.local-preflight(demo_warning, gracefully skipped when no Bicep snapshot exists — not dead code) and the new agnosticprovision(demo_provision_warning).Testing
provision_validation_test.gocallingManager.RunProvisionValidationdirectly (dispatch outcomes: pass / warnings-accepted / aborted / dispatch-error non-fatal / no-results proceeds / subscription-scoped context) and demodemo_provision_validation_check_test.go.go build ./..., targetedgo testfor changed packages, gofmt, golangci-lint, cspell, copyright — all pass.infra.provider: demo) →demo_provision_warningfires before provisioning.todo-nodejs-mongo-acawithhost: demo) → bothdemo_provision_warning(provision) anddemo_warning(local-preflight) fire.Notes for reviewers
provisioncheck runs for every project whenever an extension declaringvalidation-provideris installed (extensions with that capability are started globally, and dispatch fans out to all registered checks). Extensions that only apply to specific project types must self-condition inside theirValidatehandler (via the lean context or by calling back into azd forazure.yaml) and return a passing result otherwise.provisionvalidation now runs before preprovision hooks (and, inup, before packaging). This is intentional fail-fast behavior; the lean context is env-scoped and does not depend on hooks or packaging.azure.ai.agentsextension to consume the new check type is out of scope here and tracked separately in Enable provider-agnostic client-side validation ('azd validate') for extension-provided provisioning providers #9016.