Skip to content

terramate-io/terramate-quickstart-aws

Repository files navigation

Terramate

Terramate AWS Quickstart Guide

✨ https://terramate.io ✨


Discord Server

πŸ“– Terramate Docs | πŸš€ Getting Started | πŸ’» Playground | πŸ™Œ Join Us



Introduction

This template provides a pre-configured Terramate project for Terraform (or OpenTofu) on AWS using best practices. It includes GitOps workflows in GitHub Actions, DRY code generation for stacks, and support for multiple environments. Use it to orchestrate Terraform, clone or create stacks, and automate plans and applies in Pull Requests.

Features

  • GitOps for Terraform with GitHub Actions: Pre-configured GitHub Action GitOps workflows using merge-and-apply strategy with drift detection and reconciliation.
  • Recommended Project Structure: Best practice project structure with environment-based organization (stg, prd).
  • Change Preview in Pull Requests: Preview and approval of plans in Pull Requests to review and approve changes before deploying.
  • DRY Terraform Stacks: Generate Terraform provider and backend configuration in stacks via mixins and generators.
  • OpenID Connect (OIDC): Allows GitHub Actions workflows to access AWS resources without storing long-lived GitHub secrets.
  • Terraform S3 Remote State Backend: Terraform Remote State Storage and State Locking with AWS S3 and DynamoDB.
  • Terramate Cloud Integration: Pushes data to Terramate Cloud for observability, asset management, drift management, and Slack notifications.
  • Multi-Environment Support: Built-in support for staging and production environments with separate AWS accounts.
  • Drift Detection and Reconciliation: Automated drift detection and reconciliation workflows to maintain infrastructure consistency.

AWS Architecture

This template creates a complete AWS infrastructure with the following components:

  1. VPC (Virtual Private Cloud)

    • A dedicated network space for your infrastructure
    • Public and private subnets across multiple availability zones
    • NAT Gateway for outbound internet access from private subnets
    • Internet Gateway for inbound internet access to public subnets
  2. EKS (Elastic Kubernetes Service)

    • A managed Kubernetes cluster in auto-mode
    • Node groups for running your containerized applications
    • Auto-scaling capabilities based on workload demand
    • Integration with AWS IAM for authentication and authorization
  3. Sample Applications

    • Two sample web applications deployed in the EKS cluster
    • Applications are accessible via LoadBalancer services
    • Demonstrates best practices for Kubernetes deployments

The infrastructure is designed to be highly available across multiple availability zones, secure with proper network isolation, and scalable with clear separation of concerns.


Repository Structure

The project is organized as follows:

terramate-quickstart-aws/
β”œβ”€β”€ config.tm.hcl              # Root: Terraform version, backend, providers, OIDC
β”œβ”€β”€ imports.tm.hcl             # Imports mixins and generators
β”œβ”€β”€ terramate.tm.hcl          # Project config: cloud, git, experiments
β”œβ”€β”€ workflows.tm.hcl          # Root-level Terramate scripts
β”œβ”€β”€ imports/
β”‚   β”œβ”€β”€ mixins/               # Shared code-gen: backend, providers, Kubernetes auth
β”‚   β”‚   β”œβ”€β”€ backend.tm.hcl
β”‚   β”‚   β”œβ”€β”€ terraform.tm.hcl
β”‚   β”‚   └── kubernetes.tm.hcl
β”‚   └── generators/           # Versioned code-gen templates
β”‚       └── v1/               # Current generator version
β”‚           β”œβ”€β”€ generate_vpc.tm.hcl
β”‚           β”œβ”€β”€ generate_eks.tm.hcl
β”‚           β”œβ”€β”€ generate_apps_namespace.tm.hcl
β”‚           └── generate_app.tm.hcl
β”œβ”€β”€ stacks/
β”‚   β”œβ”€β”€ terraform/
β”‚   β”‚   β”œβ”€β”€ workflows.tm.hcl  # init, preview, deploy, drift scripts
β”‚   β”‚   └── envs/
β”‚   β”‚       β”œβ”€β”€ stg/         # Staging: config.tm.hcl, vpc/, eks/, eks/apps/
β”‚   β”‚       └── prd/         # Production (same structure)
β”‚   β”œβ”€β”€ opentofu/            # OpenTofu example stacks
β”‚   └── ...
└── _bootstrap/              # One-time: state bucket, OIDC provider
    β”œβ”€β”€ terraform-state-bucket/
    └── oidc-aws-github/

Key directories:

  • config.tm.hcl (root) β€” Global defaults: generator version, Terraform version, S3 backend, AWS/Kubernetes providers, OIDC GitHub repos.
  • imports.tm.hcl β€” Declares imports for imports/mixins/*.tm.hcl and the active generator version (e.g. imports/generators/v1/*.tm.hcl).
  • imports/mixins/ β€” Generate backend.tf, terraform.tf, and (for stacks with kubernetes tag) kubernetes.tf in every stack.
  • imports/generators/v1/ β€” Versioned code-gen templates for VPC, EKS, namespace, and apps. Each generator uses a condition on global.generators.version so multiple versions can coexist (see How Code Generation Works and Upgrading Generator Templates).
  • stacks/terraform/envs/{stg,prd}/ β€” Environment-specific stacks. Each env has a config.tm.hcl (env name, VPC CIDR, EKS cluster name, etc.) and sub-stacks: vpc/, eks/, eks/apps/, eks/apps/app1/, eks/apps/app2/.
  • stacks/opentofu/ β€” Example OpenTofu stacks with their own version and backend key in stacks/opentofu/config.tm.hcl.
  • _bootstrap/ β€” Bootstrap stacks for the S3 state bucket and GitHub OIDC; use local state initially, then migrate to S3.
  • workflows.tm.hcl (root) and stacks/terraform/workflows.tm.hcl β€” Terramate script definitions.

Stacks are deployed in a fixed order: VPC β†’ EKS cluster β†’ apps namespace β†’ application stacks. EKS stacks use after = ["tag:vpc"] so the VPC is applied first.

flowchart LR
  VPC --> EKS --> Namespace --> App1
  Namespace --> App2
Loading

How Code Generation Works

This repo keeps stacks DRY by generating Terraform files from Terramate config and globals. You define stack metadata and globals; Terramate generates the actual .tf (and some _*.tf) files.

Two mechanisms:

  1. Generators (imports/generators/v1/*.tm.hcl) β€” Use generate_hcl with stack_filter.project_paths to emit specific files only into stacks whose path matches. Each generator also has a condition = global.generators.version == "v1" so that multiple generator versions can coexist (see Upgrading Generator Templates). The current generators are:

    • generate_vpc.tm.hcl β€” Targets **/envs/*/vpc, generates main.tf (VPC module) from global.vpc.*.
    • generate_eks.tm.hcl β€” Targets **/envs/*/eks, generates main.tf and data.tf from global.eks.*.
    • generate_apps_namespace.tm.hcl β€” Targets **/envs/*/eks/apps, generates namespace.tf.
    • generate_app.tm.hcl β€” Targets **/envs/*/eks/apps/*, generates main.tf (Deployment + LoadBalancer Service) from global.app.* (e.g. image, container_name, container_port).
  2. Mixins (imports/mixins/*.tm.hcl) β€” Generate shared plumbing in (almost) all stacks: backend.tm.hcl β†’ backend.tf, terraform.tm.hcl β†’ terraform.tf, and kubernetes.tm.hcl β†’ kubernetes.tf only for stacks with the kubernetes tag.

Workflow:

  • Run terramate generate to (re)generate all such files. After changing config.tm.hcl, stack paths, or generator/mixin logic, run it again.
  • Do not edit generated files by hand β€” they are overwritten by terramate generate. Edit only stack.tm.hcl, config.tm.hcl, and the generator/mixin sources.

Example: adding a new app stack under stacks/terraform/envs/stg/eks/apps/app3/ with a config.tm.hcl that sets globals "app" { image = "..."; container_name = "..."; container_port = 8080; } will, after terramate generate, get main.tf from generate_app.tm.hcl and backend.tf / terraform.tf / kubernetes.tf from the mixins (because of the kubernetes tag).


Getting Started

1. Create a repository from this template

Click Use this template on GitHub to create your own repository, then clone it.

2. Prerequisites

  1. Install asdf.

  2. Install plugins and tools (versions are in .tool-versions):

    asdf plugin add terramate && \
    asdf plugin add terraform && \
    asdf plugin add opentofu && \
    asdf plugin add pre-commit && \
    asdf install
  3. AWS credentials (for bootstrapping and local runs): Use one of the AWS provider authentication methods. We recommend aws-vault.

  4. Pre-commit (optional but recommended):

    pre-commit install

    Hooks keep Terramate and Terraform formatting and generation in sync when you commit.

3. Configure Terraform State Bucket and OIDC (bootstrap)

  1. Root config β€” In the repo root, edit config.tm.hcl:

    • Set your S3 state bucket name and region under globals "terraform" "backend".
    • Set your GitHub repo(s) under globals "aws" "oidc" "github_repositories".
  2. Generate and deploy bootstrap stacks:

    terramate generate
    terramate run -C _bootstrap terraform init
    terramate run -C _bootstrap terraform apply
  3. Migrate bootstrap state to S3: Remove tags = ["no-backend"] from the stack.tm.hcl in _bootstrap/oidc-aws-github and _bootstrap/terraform-state-bucket, then:

    terramate generate
    terramate run -C _bootstrap terraform init

    Terraform will migrate existing state into the new backend.

4. Terramate Cloud (optional)

  1. Go to Terramate Cloud, sign up or log in, and create an organization.
  2. Set terramate.config.cloud in terramate.tm.hcl to your organization (and region if needed).
  3. Optionally configure Slack under Integrations for notifications.

Day-to-Day Developer Workflows

Cloning an environment with terramate clone

To create a new environment (e.g. dev) by copying an existing one (e.g. stg):

terramate clone stacks/terraform/envs/stg stacks/terraform/envs/dev

What this does: Duplicates the stack tree under stg into dev and assigns new stack IDs to the cloned stacks (so they are independent stacks with their own state).

Next steps:

  1. Adjust environment config β€” Edit stacks/terraform/envs/dev/config.tm.hcl:

    • Set globals "terraform" { env = "dev" }.
    • Change VPC name, CIDR, subnets, EKS cluster name, namespace, etc. so they don’t conflict with stg/prd.
  2. Regenerate code:

    terramate generate
  3. Commit and push (and add the new env to CI if you use a matrix over environments).

You can then run init/preview/deploy against stacks/terraform/envs/dev the same way as for stg or prd.

Creating a new application stack

To add a new app (e.g. app3) in staging:

  1. Create the stack (and give it a name, description, and the kubernetes tag so it gets the Kubernetes provider and app generator):

    terramate create stacks/terraform/envs/stg/eks/apps/app3 \
      --name "my-new-app-stg" \
      --description "My new application" \
      --tags kubernetes
  2. Add app configuration β€” Create stacks/terraform/envs/stg/eks/apps/app3/config.tm.hcl:

    globals "app" {
      image          = "your-registry/your-image:tag"
      container_name = "my-app"
      container_port = 8080
    }

    You can also set replicas, resources.requests, etc. if your generator supports them.

  3. Generate Terraform files:

    terramate generate

    The generators and mixins will produce main.tf (from generate_app.tm.hcl), backend.tf, terraform.tf, and kubernetes.tf in the new stack. No need to copy these by hand.

  4. Plan and apply (locally or via CI) for the changed stacks, e.g.:

    terramate script run -C stacks/terraform/envs/stg init
    terramate script run -C stacks/terraform/envs/stg preview
    terramate script run -C stacks/terraform/envs/stg deploy

Upgrading generator templates

When an underlying Terraform module changes (new version, new required attributes, restructured resources), you may need to update the code generation templates. Generators are versioned so you can roll out template changes to one environment at a time.

How it works: Each generator block has a condition = global.generators.version == "v1" guard. The active version is set via globals "generators" { version = "v1" } in the root config.tm.hcl and can be overridden per environment.

Step-by-step: rolling out a new generator version to staging first

  1. Create the new version -- copy the current version directory:

    cp -r imports/generators/v1 imports/generators/v2
  2. Update the templates in imports/generators/v2/ -- change module versions, add/remove attributes, restructure resources as needed. Update the condition in each block:

    generate_hcl "main.tf" {
      condition = global.generators.version == "v2"
      # ... updated template
    }
  3. Import the new version -- add it to imports.tm.hcl:

    import { source = "./imports/mixins/*.tm.hcl" }
    import { source = "./imports/generators/v1/*.tm.hcl" }
    import { source = "./imports/generators/v2/*.tm.hcl" }

    Both versions are imported, but only the one matching each environment's global.generators.version generates output.

  4. Switch staging to v2 -- override the global in stacks/terraform/envs/stg/config.tm.hcl:

    globals "generators" {
      version = "v2"
    }
  5. Regenerate:

    terramate generate

    Only staging stacks get the new templates. Production stays on v1. Commit, open a PR, and CI plans only the changed (staging) stacks.

  6. Roll out to production -- once staging is validated, either update the root default in config.tm.hcl to "v2" or override it in stacks/terraform/envs/prd/config.tm.hcl. Run terramate generate again.

  7. Clean up -- after all environments are on v2, remove the v1 directory, remove its import from imports.tm.hcl, and remove the per-env overrides.

Running Terraform locally

From the repo root:

# Initialize all stacks under an environment
terramate script run -C stacks/terraform/envs/stg init

# Preview (plan) changed stacks
terramate script run -C stacks/terraform/envs/stg preview

# Deploy (plan + apply) changed stacks
terramate script run -C stacks/terraform/envs/stg deploy

Use -C stacks/terraform/envs/prd (or dev) for another environment. Scripts are defined in stacks/terraform/workflows.tm.hcl.


Available Terramate Scripts

Defined in stacks/terraform/workflows.tm.hcl. See Terramate Scripts.

Script Description
init Terraform init (backend + providers)
preview Terraform validate + plan; syncs preview to Terramate Cloud
deploy Terraform validate + plan + apply; syncs deployment to Terramate Cloud
drift detect Plan for drift and sync status to Terramate Cloud
drift reconcile Apply drift.tfplan to reconcile drift
terraform render Show Terraform plan output (e.g. for PR comments)

Run with:

terramate script run -C <path> <script_name>
# e.g. terramate script run -C stacks/terraform/envs/stg deploy

GitHub Actions Workflows

Workflows live in .github/workflows/. They use a matrix over environments (e.g. stg, prd) and AWS OIDC for authentication.

Workflow Trigger Purpose
Preview (preview.yml) Pull requests to main Format checks, terramate generate, init + plan on changed stacks; syncs preview to Terramate Cloud
Deploy (deploy.yml) Push to main Init + deploy (plan + apply) for changed stacks; runs drift detect after apply
Drift Detection (drift-detection.yml) Manual / schedule Init all stacks, drift detect, optionally reconcile stacks with reconcile tag

Terramate Cloud Integration

  • Preview and Deploy scripts sync plan/deployment data to Terramate Cloud when terramate.config.cloud is set in terramate.tm.hcl.
  • Use Terramate Cloud for stack overview, drift status, and (optional) Slack notifications. Configure the organization and Slack under your Cloud account settings.

Key Files Reference

File Purpose
config.tm.hcl (root) Generator version, Terraform version, backend, providers, OIDC repos
imports.tm.hcl Import declarations for mixins and active generator version(s)
terramate.tm.hcl Project config: required_version, cloud, git, run env, experiments
workflows.tm.hcl (root) Root-level Terramate scripts
stacks/terraform/workflows.tm.hcl init, preview, deploy, drift, terraform render scripts
stacks/terraform/envs/stg/config.tm.hcl Staging env: terraform.env, vpc, eks, namespace globals
stacks/terraform/envs/prd/config.tm.hcl Production env config
stacks/opentofu/config.tm.hcl OpenTofu version and backend key overrides
imports/generators/v1/*.tm.hcl Versioned code generators (VPC, EKS, namespace, app)
imports/mixins/*.tm.hcl Backend, Terraform block, Kubernetes provider mixins

About

Reference architecture for managing AWS with Terramate and Terraform or OpenTofu

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages