Skip to content

LTomer/vault-reader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

37 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

vault-reader

The purpose of this task is to read secrets from HashiCorp Vault server in order to use it during the pipeline or the release process.


The icons for that task was taken from https://icons8.com/

Current Version: 4.0.0 | GitHub Repository | License: MIT

Runtime Requirements

  • Azure DevOps agent with Node16 or Node20 support (minimumAgentVersion: 2.206.1)

Version 4.*

The task supports HashiCorp Vault KV secrets engine in both v1 and v2 without requiring path changes in your instructions.

KV path differences:
V1: kv_name/level-1/../level-n
V2: kv_name/data/level-1/../level-n

The vault-reader handles this automatically: it first attempts the v2 path (with data injected), and falls back to the v1 path if that fails. This keeps existing pipelines working regardless of the KV engine version.

Task Configuration

How to create a service connection:

  • Open Azure DevOps and select the relevant project.
  • go to project settings.
  • Select service connections under Pipelines section.
  • Create a new service by clicking on the "New service connection" button.
  • Select "Vault Reader" type and press Next.

  • Fill all details:

    • Server URL - the URL of the HashiCorp server. i.e. https://myvault.com:8200
    • Auth Methods - select the method from the list:
      • LDAP - authenticate with username and password via LDAP.
      • Token - authenticate using a Vault token.
      • Userpass - authenticate with a Vault username and password.
    • Username - enter username.
    • Password/Token - enter password or token for that user.
    • Disable strict SSL - select this option if you get the error: unable to verify the first certificate.
  • Enter service name and description, click Save.


How to work with the task during a build/release process

Open build/release and add the task from the list.

Fields:

  • Vault Service - select the Vault Connection Service from the list.
  • Source Type:
    • Inline - define instructions (Variables & Actions) in multiline box.
    • File Path - read the instructions from file (there is also multiline box to define variables).

Instructions:

The signs => or <= are part of the task instructions.

General Instructions:

  • Comment - add a comment to your instruction by using # at the beginning of the line. the task will ignore that line.
  • Message - print a message during the Build/Release process by using @ at the beginning of the line.

Variables:
You can define a variable and use it later with the Action commands.

Format: Variable-Name <= Value

Variable-Name can contain Letters (upper/lower), numbers and underline ('_'). Must start with a letter.

Example:
projectPath <= /project/serviceA

In this example we will create a variable named projectPath that sets the value /project/serviceA

Result variable - for some Action type you can use the Azure-DevOps-Variable as a special variable for the next lines, See comment on the table below.

Define Actions:

General Format: ActionType => Path => Field => Azure-DevOps-Variable

Task Variables can affect the Path and Field of the action
Azure-DevOps-Variable can contain Letters (upper/lower), numbers, period and underline. Must start with a letter.

Action Description Azure DevOps Variable 1
var reads value from Path & Field assigns value to Azure DevOps variable 2
pre reads object from Path
Field will contain a list of keys (separated by a comma) or * for all keys (not recommended)
assigns multiple values to multiple variables
in a single command 2 3
raw reads value from Path & Field and store the value into a file (as is) assigns file location into a variable 4 5
base64 reads value from Path & Field, decodes the value from BASE64 and stores the result into a file assigns file location into a variable 4 5
b64var reads value from Path & Field, decodes the value from BASE64 and assigns the decoded string as a variable assigns value to Azure DevOps variable 2
json reads json object from Path and stores it into a file as json
Field will contain location of the file schema. If the data and the scheme aren't equal it would fail
use * to skip the compare process (not recommended)
assigns file location into a variable 5 6
yaml reads json object from Path and stores it into a file as yaml
Field will contain location of the json file schema. If the data and the scheme aren't equal it would fail
use * to skip the compare process (not recommended)
assigns file location into a variable 5 6
rep reads file from Field and replaces the string __[key]__ with a value that reads from Path and stores it into a file assigns file location into a variable 5 6
exp export JSON object into file. The line define in the Field as base64, %%KEY%% and %%VALUE%% defined place holder for key & value from the JSON object. assigns file location into a variable 5


Examples:

How to use var and pre actions

Read values from project-A using var action

# This line is a comment.
# Read environment value from vault into env variable 
var => DemoProjects/project-A => environment => env

# Read username & password from vault into usr & pass variables
var => DemoProjects/project-A => username => usr
var => DemoProjects/project-A => password => pss

Read username & password in one line using pre action and store them in login_username & login_password.

# Read username & password with pre action
pre => DemoProjects/project-A => username,password => login

How to use raw and base64 actions

Read value and save it into a file. The variable will be set with the file location.

@ This line will be printed on the build/release console
@ Read rawData and save it into a file. File path will be stored at variable $(file1)
raw =>  DemoProjects/project-B/service-A => rawData => file1

# Read mySecret from vault, decode it (base64) and save it into a file.
# The location will be stored at the variable secret.
base64 =>  DemoProjects/project-B/service-A => mySecret => secret

How to use b64var action

Similar to base64, but instead of writing the decoded content to a file, the decoded string is stored directly as a secret pipeline variable.

# Read mySecret from vault, decode it (base64) and store the decoded string in the variable secretValue.
b64var => DemoProjects/project-B/service-A => mySecret => secretValue

The variable $(secretValue) will contain the decoded (plain text) value of mySecret.

How to work with VaultReader variables

This section refers to the previous image.

# Define a variable that named servicePath
servicePath <= DemoProjects/project-B/service-A

# Read value using a variable
raw => {servicePath} => rawData => file1
base64 => {servicePath} => mySecret => secret

You can also use a variable as a part of Path/Field.
For example: {servicePath}/test (equal to DemoProjects/project-B/service-A/test)

How to use json action

We can create a json file that contains a template (value will be empty). This way we can compare the scheme between the objects from the vault server and the json file. The json file can be stored on the source control and will be pulled during the build.

File: service-b-template.json
The json file will be stored under config folder in the git repository.

{
  "service": {
    "name": "",
    "url": ""
  },
  "sql": {
    "ConnectionString": "__sql__"
  }
}
# Define a path
servicePath <= DemoProjects/project-B

# Read value using json action
json => {servicePath}/service-B => config/service-b-template.json => configFile

How to use yaml action

This section is based on the data that presented in the JSON explanation.

# Define a path
servicePath <= DemoProjects/project-B

# Read value using json action
yaml => {servicePath}/service-B => config/service-b-template.json => configFile

How to use replace action

When we have a file that we want to inject secrets during the build/release process to it. For example, a secret.yaml file on the working directory that defines Secret for k8s cluster:

apiVersion: v1
kind: Secret
metadata:
  name: secret-basic-auth
type: kubernetes.io/basic-auth
stringData:
  username: __username__
  password: __password__

On the task instructions:

# Inject secrets into a yaml file
rep => DemoProjects/project-A => secret.yaml => secretFile

The variable $(secretFile) will contain the location of the updated file.
To update kubernetes cluster you can run the following command:

kubectl apply -f $(secretFile)


How to use export action

Export the keys and values from the JSON object (1 level) into a file, every key & value will be placed in deferent line by the format in the Field. The format is set as base64, the %%KEY%% and %%VALUE%% will replace with the key & value of each item.

For example, we would like to convert JSON object to tfvars file (This type of file is used by terraform).

Line format: %%KEY%% = "%%VALUE%%"
Line format (Base64): JSVLRVklJSA9ICIlJVZBTFVFJSUi

The line we should add to the task:

exp => DemoProjects/project-A => JSVLRVklJSA9ICIlJVZBTFVFJSUi => tfvar_file

The file that will create after the task is complete, the path to the file will set to tfvar_file.

environment = "dev"
password = "p@assw0rd"
username = "someone"

How to work with VaultReader special variables

Task variables defined with <= can be referenced in Path and Field using {varName} syntax.
In addition, the output of file-producing actions (json, yaml, rep) can be used as input to subsequent actions by wrapping the Azure DevOps variable name in double curly braces: {{varName}}.

# Define a path
servicePath <= DemoProjects/project-B

# Read value using json action
json/yaml => {servicePath}/service-B => config/service-b-template.json => configFile

# Use the output file of the previous action as the template for this action
rep => database/sql => {{configFile}} => configFile2

In this example, {{configFile}} passes the file path produced by the json/yaml action as the template input to the rep action.


Development

Prerequisites

Tool Version
Node.js v25.2.1
npm 11.6.2
TypeScript (global) 6.0.2
tfx-cli (global) v0.23.1

Install global tools:

npm install -g typescript
npm install -g tfx-cli

Build

Compile TypeScript to JavaScript — two options:

Option A – local (recommended):

cd Task
npm install
npm run build

Option B – global TypeScript (run from project root):

tsc -p tsconfig.json

To compile and run a single file for quick testing:

tsc <file>.ts && node <file>.js

Deployment to Azure DevOps

There are two ways to deploy: uploading the task directly (faster for development) or publishing a full marketplace extension.

Option 1 – Azure DevOps Extension for Azure CLI

The Azure DevOps Extension for Azure CLI (az devops) is the official Microsoft CLI. It is useful for general Azure DevOps management but does not support uploading custom build tasks directly — use tfx-cli for that.

Option 2 – tfx-cli (uploading tasks directly)

Best for development iterations — uploads the task straight to the organization without packaging an extension.

Login:

tfx login -u https://dev.azure.com/<your-org> -t <personal-access-token>

Upload task:

cd Task
tsc; tfx build tasks upload --task-path .

List all uploaded tasks:

tfx build tasks list --no-color

Delete a task:

tfx build tasks delete --task-id <task-guid>

Task GUID for this project: 34d19f40-9306-11e8-80f8-e3fce4d54d70 (from task.json)

Option 3 – Package and publish as VSIX extension

Packages all task files into a .vsix extension file that can be published to the Visual Studio Marketplace or uploaded to a private Azure DevOps organization.

# From the project root
tfx extension create

This reads vss-extension.json and produces a .vsix file. To publish it:

tfx extension publish --vsix <file>.vsix

License

MIT © LTomer

Footnotes

  1. Azure DevOps Variable - the result of the action will be stored at the variable and can be used in the next tasks as $(variableName)

  2. The type of the variable is "secret" and therefore it can't be printed in the build & release console.
    2 3

  3. Variable name: Azure-DevOps-Variable_Field

  4. On Linux OS set permissions to Read only.
    2

  5. The file that contains the secrets will be deleted at the end of the build/release process (the file is stored under _temp folder).
    2 3 4 5 6

  6. The Azure-DevOps-Variable can used as a special variable surrounded with {{ }}, with this option output from one action can be the input for the others. 2 3

About

Read secrets from hahicorp vault during azure devops pipeline

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors