Add support for Azure DevOps in azd pipeline config by hattan · Pull Request #693 · Azure/azure-dev

hattan · 2022-09-17T04:36:07Z

This PR fixes #101, #708, #699 and #702 and adds support for Azure DevOps an SCM and CI provider.

This new workflow enables the following:

Pipeline named providers for GitHub github and Azure DevOps azdo
The provider can be specified in two ways:
- via azure.yaml by defining the following section:
```
pipelines:
   - provider: azdo
```
  This providers a permanent mechanism for switcing to Azure DevOps as a pipeline provider.
- Via the --provider flag
```
 azd pipeline config --provider azdo
```
  This method updates the provider for only this command invocation. Subsequent pipeline config commands will default to github, unless --provider is passed in.
Detection of provider based on .azdo or .github in the template augments the above behavior.
- If only a .github folder exists, it is assumed the provider is github if azdo is set as a provider value, this will result in an error
- If only a .azdo folder exists, it is assume the provider is azdo. If github is set as the provider value, this will result in an error.
- If both .azdo and .github folder exists, the Azure Developer CLI will not make a determination and rely on the user to provide a pipeline provider.
AzDo provider supports using an existing project.
AzDo provider supports creating a new project.
AzDo provider supports using an existing repo.
AzDo provider supports creating a new repo in a project.
PAT authentication is used to push to the new repo, and the tool checks for credential helper and offers to set it to store to make subsequent pushes easier.
If a PAT is not provided, the tool will ask the user for a PAT and ask the user if they would like to store it.
If an organization name is not provided via an Environment Variable, the user will be prompted for an organization name and it will be stored in the environment file.
AzDo provider creates a build policy to ensure pushes to default branch are disabled and require a successful pipeline invocation.
Deployment Pipeline is created
Service Connection is created to allow the pipeline to deploy to azure.
Initial pipeline deployment is queued to follow the github behavior.

This was tested with a full deployment of todo-python-mongo using an Azure DevOps Pipeline.

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

…o add-azdo-provider

… and test based on linting results.

* Refactor to introduce new azdo package * refactor azdo code to multiple files in azdo package * Added azdo connection test and refactored provider * Added azdo provider initial tests * added more azdo provider tests * Adding test to verify save env config in azdo provider * Updating displayed urls to aka.ms

* Adding service connection name to pipeline variables * Checking for service connection exists and also removing do push condition * Check for pipeline exists and update the name to include repo if exists. * adding provider arg and setting variables only for terraform (#46) * Set default pipeline provider to github * Adjust project creation wait time. * prompt for pat and org name if not in system env vars. * Azure DevOps name standardization * Update Azdo readme. * refactor to fix failing test Co-authored-by: Victor Vazquez <victor.vazquez@microsoft.com>

jongio · 2022-09-21T16:51:48Z

 	local.StringVar(&p.manager.PipelineServicePrincipalName, "principal-name", "", "The name of the service principal to use to grant access to Azure resources as part of the pipeline.")
 	local.StringVar(&p.manager.PipelineRemoteName, "remote-name", "origin", "The name of the git remote to configure the pipeline to run on.")
 	local.StringVar(&p.manager.PipelineRoleName, "principal-role", "Contributor", "The role to assign to the service principal.")
+	local.StringVar(&p.manager.PipelineProvider, "provider", "github", "The pipeline provider to use (GitHub and Azdo supported).")


Suggested change

local.StringVar(&p.manager.PipelineProvider, "provider", "github", "The pipeline provider to use (GitHub and Azdo supported).")

local.StringVar(&p.manager.PipelineProvider, "provider", "github", "The pipeline provider to use (github and azdo supported).")

jongio · 2022-09-21T16:53:58Z

+)
+
+var (
+	AzDoHostName                 = "dev.azure.com"                                          // hostname of the AzDo PaaS service.


Is this overridable for self-host situations.

Yes, these variable are exported as part of the AzDo package and overridable. We can add helper method to make it easier, but they were built specifically with the idea that we could override to use a self host AzDo server.

After this PR I will set up a self hosted AzDo and test against that.

jongio · 2022-09-21T16:54:10Z

+	AzurePipelineName            = "Azure Dev Deploy"                                       // name of the azure pipeline that will be created
+	AzurePipelineYamlPath        = ".azdo/pipelines/azure-dev.yml"                          // path to the azure pipeline yaml
+	CloudEnvironment             = "AzureCloud"                                             // target Azure Cloud
+	DefaultBranch                = "master"                                                 // default branch for pipeline and branch policy


I think this should be main right?

Yes, we kept it as master because the original GitHub flow created a master branch. We can switch the default branch to main for both.

I added an issue to track this #725

Are you sure? I believe it defaults to what the developer machine has as the default and we want that default to be main.

jongio · 2022-09-21T16:54:34Z

+		require.NoError(t, err)
+		require.NotNil(t, policyType)
+		require.EqualValues(t, *policyType.DisplayName, "Build")
+


Please scribe for extra lines

No problem, I will remove them.

jongio · 2022-09-21T16:55:18Z

+		"versioncontrol": {
+			"sourceControlType": "git",
+		},
+		"processTemplate": {
+			"templateTypeId": processTemplateId,
+		},


One is allower, the other is camel...consistent?

We can make the adjustment, unfortunately this is an underlying api property. In a future PR we could create a struct with the json mapping comments instead of relying on the map object. This could be a contribution back to the go sdk for azure devops and the capabilities property.

jongio · 2022-09-21T16:55:57Z

+			console.Message(ctx, fmt.Sprintf("error: the project name '%s' is already in use\n", name))
+			continue // try again
+		} else if strings.Contains(message, "The following name is not valid") {
+			console.Message(ctx, fmt.Sprintf("error: the project name '%s' is not a valid Azure DevOps project Name. See https://aka.ms/azure-dev/azdo-project-naming\n", name))


Make sure all akas are created and have adpteam as co-owner

Will do, I added Rajesh as co-owner but will add adpteam.

jongio · 2022-09-21T16:56:31Z

+) (serviceendpoint.CreateServiceEndpointArgs, error) {
+	endpointType := "azurerm"
+	endpointOwner := "library"
+	endpointUrl := "https://management.azure.com/"


Is this overridable? Does it need to be?

Yes it should be . There's a few spots in the az cli package that is hard coded to this endpoint, but to support sovereign clouds, it should be overridable. I'll add an issue so we can address this.

Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>

azure-sdk · 2022-09-21T23:43:39Z

azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/693

View Changes | Compare Changes

azure-sdk · 2022-09-22T00:58:41Z

Azure Dev CLI Install Instructions

Install scripts

MacOS/Linux

May elevate using sudo on some platforms and configurations

bash:

curl -fsSL https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/uninstall-azd.sh | bash;
curl -fsSL https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/install-azd.sh | bash -s -- --base-url https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693 --version '' --verbose

pwsh:

Invoke-RestMethod 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/uninstall-azd.ps1' -OutFile uninstall-azd.ps1; ./uninstall-azd.ps1
Invoke-RestMethod 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/install-azd.ps1' -OutFile install-azd.ps1; ./install-azd.ps1 -BaseUrl 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693' -Version '' -Verbose

Windows

powershell -c "Set-ExecutionPolicy Bypass Process; irm 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/uninstall-azd.ps1' > uninstall-azd.ps1; ./uninstall-azd.ps1;"
powershell -c "Set-ExecutionPolicy Bypass Process; irm 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693/install-azd.ps1' > install-azd.ps1; ./install-azd.ps1 -BaseUrl 'https://azuresdkreleasepreview.blob.core.windows.net/azd/standalone/pr/693' -Version '' -Verbose;"

Standalone Binary

Container

docker run -it azdevcliextacr.azurecr.io/azure-dev:pr-693

vhvb1989 · 2022-09-23T22:01:43Z

Closed in favor of: #743

hattan and others added 19 commits September 13, 2022 07:45

Adding changes to support debugging pipeline provider

bdc3359

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

Removing local debug values

32bc049

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

add support for project selection

a4467f5

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

Adding Azdo remote validate.

4d9501f

Configure git remote for azdo and select Repo

351502a

Added changes to support git push with PAT and credential helper

0bbd031

adding support for create project & azdo details in .env

e7d6ec9

Cleanup and speed up azdo project creation

ffe676f

Utilizing default repo when creating a new project

86b15d1

Cleanup + support for create new repos on existing projects.

47e7d1b

Adding Pipeline creation for Azure DevOps

33a3e91

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

Merge branch 'main' of github.com:azure/azure-dev into add-azdo-provider

20ba445

build

c5d4778

adding create pipeline, service connection and build policy

ac62489

Co-authored-by: Hadwa Gaber <hadwaa@microsoft.com>

Merge branch 'add-azdo-provider' of github.com:kaizentm/azure-dev int…

8a78c3d

…o add-azdo-provider

Merging in pipeline detector code.

73a9ee3

Added comments to azdo provider functions

b0dcd30

added further comments to azdo.go

69449ee

Removing azdcontext from DetectProviders

f2324a9

hattan changed the title ~~Add azdo provider~~ Add support for Azure DevOps in azd pipeline config Sep 17, 2022

hattan added 7 commits September 17, 2022 08:21

Adding azdo provider api literal property names to cpell global list.

d1b2a5b

Adding missing error checks.

77dcf1f

Removing unused variable

ef8c377

Fixing golint error

7e0faac

Removing unused development test.

92addbf

Adding versioncontrol string literal to cspell. Used by the azdo api.

e484a5d

Modifying pipeline test check. Punctuation removed from error in code…

3752407

… and test based on linting results.

rajeshkamal5050 assigned hattan Sep 19, 2022

rajeshkamal5050 added this to the Release 0.3 milestone Sep 19, 2022

rajeshkamal5050 added the azdo label Sep 19, 2022

hattan marked this pull request as ready for review September 19, 2022 07:49

hattan requested review from ellismg, hemarina, jongio, vhvb1989, wbreza and weikanglim as code owners September 19, 2022 07:49

hattan requested a review from HadwaAbdelhalem September 19, 2022 07:49

hattan added 2 commits September 19, 2022 00:53

Adding build policy tests

c5acdee

Added copyright and license comments to azdo package.

4c18f41

jongio reviewed Sep 19, 2022

View reviewed changes

Comment thread cli/azd/pkg/azdo/azdo.go Outdated

Comment thread cli/azd/pkg/azdo/pipeline.go Outdated

Comment thread cli/azd/pkg/commands/pipeline/pipeline_manager.go Outdated

Modifying azd project description and existing sp env variable name.

1e1e899

jongio reviewed Sep 19, 2022

View reviewed changes

Comment thread cli/azd/pkg/commands/pipeline/pipeline_manager.go Outdated

Modifying existing sp env var to AZURE_CREDENTIALS

3f3927e

hattan assigned vhvb1989 Sep 20, 2022

hattan and others added 3 commits September 20, 2022 18:26

save pipeline provider election to env

8d93985

add err check to savePipelineProviderToEnv

22da58e

jongio suggested changes Sep 21, 2022

View reviewed changes

Apply suggestions from code review

861a68d

Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com>

vhvb1989 approved these changes Sep 21, 2022

View reviewed changes

puicchan mentioned this pull request Sep 23, 2022

azdo support - prompt about storing PAT in .env should be updated #737

Closed

vhvb1989 closed this Sep 23, 2022

vhvb1989 deleted the add-azdo-provider branch September 23, 2022 21:58

rajeshkamal5050 added the area/pipeline CI/CD pipeline config (GH Actions, AzDO) label Apr 1, 2026

	local.StringVar(&p.manager.PipelineProvider, "provider", "github", "The pipeline provider to use (GitHub and Azdo supported).")
	local.StringVar(&p.manager.PipelineProvider, "provider", "github", "The pipeline provider to use (github and azdo supported).")

Uh oh!

Conversation

hattan commented Sep 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

azure-sdk commented Sep 21, 2022

Repoman Generation Results

Project: todo-csharp-cosmos-sql

Remote: azure-samples-staging

Branch: pr/693

Project: todo-csharp-sql

Remote: azure-samples-staging

Branch: pr/693

Project: todo-nodejs-mongo-aca

Remote: azure-samples-staging

Branch: pr/693

Project: todo-nodejs-mongo-swa-func

Remote: azure-samples-staging

Branch: pr/693

Project: todo-nodejs-mongo

Remote: azure-samples-staging

Branch: pr/693

Project: todo-nodejs-mongo-terraform

Remote: azure-samples-staging

Branch: pr/693

Project: todo-python-mongo-aca

Remote: azure-samples-staging

Branch: pr/693

Project: todo-python-mongo-swa-func

Remote: azure-samples-staging

Branch: pr/693

Project: todo-python-mongo

Remote: azure-samples-staging

Branch: pr/693

Project: todo-python-mongo-terraform

Remote: azure-samples-staging

Branch: pr/693

Uh oh!

azure-sdk commented Sep 22, 2022

Azure Dev CLI Install Instructions

Install scripts

MacOS/Linux

Windows

Standalone Binary

hattan commented Sep 17, 2022 •

edited

Loading