|
| 1 | +# AWS Configuration Methods |
| 2 | + |
| 3 | +Managing configuration is divided into two processes, [Configuration Management](#configuration-management) which is the process modifying and storing configuration in a cloud storage service, and [Configuration Consumption](#configuration-consumption) which is the process of ingesting the configuration from a cloud storage service into a Fargate container or Lambda function. |
| 4 | + |
| 5 | +## Configuration Management |
| 6 | + |
| 7 | +When using *Stash*, management is consistent accross supported cloud storage services. |
| 8 | + |
| 9 | +```bash |
| 10 | +$ stash sync config/dev/.env |
| 11 | +``` |
| 12 | +```bash |
| 13 | +$ stash edit -t dev |
| 14 | +``` |
| 15 | + |
| 16 | +## Configuration Consumption |
| 17 | + |
| 18 | +Consumption has many methods with different benefits. The pupose of this document is to discuss AWS methods for Fargate containers and Lambda functions, but some of the following methods may also apply to other technologies. |
| 19 | + |
| 20 | +#### User and Role Policies |
| 21 | +AWS methods require developers and the container execution role or application role to have access to the appropriate cloud storage service and KMS keys. |
| 22 | + |
| 23 | +The following *Stash* command generates Terraform scripts specific to the `dev` configuration. This makes it easier to manage the AWS access policies through Terraform. |
| 24 | + |
| 25 | +```bash |
| 26 | +$ stash get -t dev -o terraform |
| 27 | +``` |
| 28 | + |
| 29 | +### Method 1: Download Configuration File (Stash CLI) |
| 30 | + |
| 31 | +**Supports**: ECS Fargate Containers |
| 32 | + |
| 33 | +On start up containers can call a storage service directly using the *Stash* CLI and create a configuration file inside the container allowing the application to load it into memory. |
| 34 | + |
| 35 | +**Requirement**: The stash.yml file used to sync the configuration must included in the Docker image where *Stash* can find it. |
| 36 | + |
| 37 | +Dockerfile (build) |
| 38 | +```bash |
| 39 | +FROM golang:1.13 |
| 40 | + |
| 41 | +ARG version=0.0.0-unknown.0 |
| 42 | + |
| 43 | +# install stash |
| 44 | +RUN curl -L -o /usr/local/bin/stash https://github.com/dabblebox/stash/releases/download/v0.1.0-rc/stash_linux_386 && chmod +x /usr/local/bin/stash |
| 45 | + |
| 46 | + |
| 47 | +COPY . /app/ |
| 48 | + |
| 49 | +WORKDIR /app |
| 50 | + |
| 51 | +RUN go build -ldflags '-X main.version='$version -o app |
| 52 | + |
| 53 | +ENTRYPOINT [ "./docker-entrypoint.bash" ] |
| 54 | +``` |
| 55 | + |
| 56 | +docker-entrypoint.bash (get) |
| 57 | +```bash |
| 58 | +#!/bin/bash |
| 59 | + |
| 60 | +echo "Getting $CONFIG_ENV configuration." |
| 61 | + |
| 62 | +stash get -l -t $CONFIG_ENV 1> .env |
| 63 | + |
| 64 | +exec ./app |
| 65 | +``` |
| 66 | + |
| 67 | +### Method 2: File Injection (Stash CLI) |
| 68 | + |
| 69 | +On start up containers can call a storage service directly using the *Stash* CLI and inject secrets into a configuration file inside the container allowing the application to load the file containing the secrets into memory. Secret tokens can be added to a configuration file that is checked into a repository. |
| 70 | + |
| 71 | +The tokens are AWS Secret Manager secret names. Use double colons, `::`, to specify any field in the secret's json object. |
| 72 | + |
| 73 | +.env (config) |
| 74 | +``` |
| 75 | +USER=${app/dev/db::user} |
| 76 | +PASSWORD=${app/dev/db::password} |
| 77 | +``` |
| 78 | + |
| 79 | +Dockerfile (build) |
| 80 | +```bash |
| 81 | +FROM golang:1.13 |
| 82 | + |
| 83 | +ARG version=0.0.0-unknown.0 |
| 84 | + |
| 85 | +# install stash |
| 86 | +RUN curl -L -o /usr/local/bin/stash https://github.com/dabblebox/stash/releases/download/v0.1.0-rc/stash_linux_386 && chmod +x /usr/local/bin/stash |
| 87 | + |
| 88 | +COPY . /app/ |
| 89 | + |
| 90 | +WORKDIR /app |
| 91 | + |
| 92 | +RUN go build -ldflags '-X main.version='$version -o app |
| 93 | + |
| 94 | +ENTRYPOINT [ "./docker-entrypoint.bash" ] |
| 95 | +``` |
| 96 | + |
| 97 | +docker-entrypoint.bash (inject) |
| 98 | +```bash |
| 99 | +#!/bin/bash |
| 100 | + |
| 101 | +echo "Injecting $CONFIG_ENV configuration." |
| 102 | + |
| 103 | +stash inject $CONFIG_ENV/.env -l -s secrets-manager 1> secrets.env |
| 104 | + |
| 105 | +exec ./app |
| 106 | +``` |
| 107 | + |
| 108 | +### Method 3: Environment Injection (AWS) |
| 109 | + |
| 110 | +**Supports**: ECS Fargate Containers |
| 111 | + |
| 112 | +Configuration and secret references to cloud services like Secrets Manager, Parameter Store, and S3 can be listed in ECS task definitions. On container start, AWS injects the configuration and/or secrets from cloud service references into the containers environment variables. |
| 113 | + |
| 114 | +*Stash* provides commands that print the task definition JSON for stashed configuration making it easier to add to a task definition. [Secrets](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) from AWS Secrets Manager and SSM Parameter Store along with S3 [environment files](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/taskdef-envfiles.html) are supported. |
| 115 | + |
| 116 | +```bash |
| 117 | +$ stash get -t dev -o ecs-task-inject-json |
| 118 | +``` |
| 119 | + |
| 120 | +```bash |
| 121 | +$ stash get -t dev -o ecs-task-inject-env |
| 122 | +``` |
| 123 | + |
| 124 | +### Method 4: Direct Ingest (Stash Library) |
| 125 | + |
| 126 | +**Supports**: ECS Fargate Containers / Lambda Functions |
| 127 | + |
| 128 | +Code can use the *Stash* Go integration library to load configuration directly into memory. |
| 129 | + |
| 130 | +**Requirement**: The stash.yml file used to sync the configuration must be included in the Docker image or Lambda function where *Stash* can find it. |
| 131 | + |
| 132 | +```go |
| 133 | +package main |
| 134 | + |
| 135 | +import ( |
| 136 | + "log" |
| 137 | + |
| 138 | + "github.com/dabblebox/stash" |
| 139 | + "github.com/dabblebox/stash/component/output" |
| 140 | +) |
| 141 | + |
| 142 | +config, err := stash.GetMap(stash.GetOptions{}) |
| 143 | +if err != nil { |
| 144 | + log.Fatal(err) |
| 145 | +} |
| 146 | + |
| 147 | +for k, v := range config { |
| 148 | + log.Printf("%s=%s\n", k, v) |
| 149 | +} |
| 150 | +``` |
0 commit comments