Skip to content
Cloudly Engineer logo

The Cloudly Engineer

Enough talk, let's build!

  • Home
  • About Me
  • All Posts
  • Downloads
  • Register
  • Other great cloud related sites
  • Password Reset
  • Log Out
Search

Tag: terraform-setup

multiple overlay patterns of a colorful design

Terraform AWS Multi-Account Setup

Terraform is an elite open-source software that helps solve a lot of cloud automations. It’s very scalable and easy to use. But Terraform’s flexibility has caused debates on how to setup a perfect code directory. A code directory should be setup in a way where it’s easy to read, quickly deploy to various environments and accounts without having to repeat or copy and paste code. Terragrunt to the rescue! That helped tremendously but it can become challenging to manage multiple tools or the code directory structure becomes unmanageable in the long run. In this post I like to share another method of Terraform directory setup for AWS multiple accounts and environments. I’m not saying this is the best or better solution than other ones… it’s just another way of doing AWS Multi-Account setup with Terraform.

Old Way

This is the structure I used to use with Terragrunt. Terragrunt would dynamically change the S3 Backend Key parameter based on the directory structure.

├── dev
│   ├── inputs.yml
│   ├── terragrunt.hcl
│   └── vars.tf
├── staging
│   ├── inputs.yml
│   ├── terragrunt.hcl
│   └── vars.tf
├── terragrunt.hcl
├── main.tf

New Way

Now this is the structure I use, this setup doesn’t require Terragrunt. Notice it’s less files and it’s all Terraform files.Let’s review some of the files in detail.

├── accounts
│   ├── dev
│   │   ├── backend.conf
│   │   └── terraform.tfvars
│   └── staging
│       ├── backend.conf
│       └── terraform.tfvars
├── Makefile
├── main.tf

For this setup the main.tf still declares the backend but without any of its parameters! This is called partial configuration in Terraform. We’ll define the rest in the backend.conf file.

terraform {
  backend "s3" {
    key = "tf-partial-backend-example/terraform-main.tfstate"
  }
}

Backend.conf

We need to give all the remaining required and optional parameters of the “s3” backend defined in the main.tf in this backend.conf file. In this example I’m declaring the key path in the main.tf so it will be the same for every S3 bucket. If I wanted it to be unique then I would add it to the backend.conf. You can do that with any of the parameters!

#  Terraform state for S3 backend config variables
bucket   = "dev-example-s3-bucket-tf"
region   = "us-east-1"
role_arn = "arn:aws:iam::123456789:role/svc-terraform-role"

terraform.tfvars

This file defines all the values for your variables for that specific environment or account.

region      = "us-east-1"
environment = "dev"
account_id  = "123456789"

Terraform init, plan, and apply

In this design initiating terraform requires few options. We need to the define the rest of the backend configuration, we do that by passing the -backend-config option. Next option is the -reconfigure, is very important because we’re working with multiple accounts and environments we need to ensure the local cache .terraform* files are well, reconfigured each time. That’s the catch for this pattern!

terraform init -backend-config accounts/dev/backend.conf -reconfigure

# Expected output
Terraform has been successfully initialized!

# Plan
terraform plan -var-file accounts/dev/terraform.tfvars

# Apply
terraform apply -var-file accounts/dev/terraform.tfvars

Makefile

You probably noticed a file called “Makefile” in the directory. A makefile is just a way to shorthand or automate our long commands. Here’s a makefile for Terraform.

SHELL := /usr/bin/env bash

# HOW TO EXECUTE

# Executing Terraform PLAN
#	$ make tf-plan-example env=<env>
#    e.g.,
#       make tf-plan-example env=dev

# Executing Terraform APPLY
#   $ make tf-apply-example env=<env>

# Executing Terraform DESTROY
#	$ make tf-destroy-example env=<env>

all-test: clean tf-plan-example

.PHONY: clean
clean:
	rm -rf .terraform

.PHONY: tf-plan-example
tf-plan-example:
	terraform fmt && terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform validate && terraform plan -var-file accounts/${env}/terraform.tfvars

.PHONY: tf-apply-example
tf-apply-example:
	terraform fmt && terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform validate && terraform apply -var-file accounts/${env}/terraform.tfvars -auto-approve

.PHONY: tf-destroy-example
tf-destroy-example:
	terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform destroy -var-file accounts/${env}/terraform.tfvars

Now all we have to do for TF init is call like this:

make tf-plan-example env=dev

Or for the staging environment like this

make tf-plan-example env=staging

Apply time!

make tf-apply-example env=dev

# aws_ssm_parameter.this will be created
  + resource "aws_ssm_parameter" "this" {
      + arn       = (known after apply)
      + data_type = (known after apply)
      + id        = (known after apply)
      + key_id    = (known after apply)
      + name      = "example-param-dev"
      + tags      = {
          + "Environment" = "dev"
          + "Name"        = "example-param-dev"
          + "Owner"       = "Waleed"
          + "Region"      = "us-east-1"
        }
      + tags_all  = {
          + "Environment" = "dev"
          + "Name"        = "example-param-dev"
          + "Owner"       = "Waleed"
          + "Region"      = "us-east-1"
        }
      + tier      = "Standard"
      + type      = "String"
      + value     = (sensitive value)
      + version   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
aws_ssm_parameter.this: Creating...
aws_ssm_parameter.this: Creation complete after 0s [id=example-param-dev]
example param dev screenshot
Example param dev screenshot

Now let’s deploy the same code in a different region and environment with just a single change in the command line.

make tf-apply-example env=staging

# aws_ssm_parameter.this will be created
  + resource "aws_ssm_parameter" "this" {
      + arn       = (known after apply)
      + data_type = (known after apply)
      + id        = (known after apply)
      + key_id    = (known after apply)
      + name      = "example-param-staging"
      + tags      = {
          + "Environment" = "staging"
          + "Name"        = "example-param-staging"
          + "Owner"       = "Waleed"
          + "Region"      = "us-west-2"
        }
      + tags_all  = {
          + "Environment" = "staging"
          + "Name"        = "example-param-staging"
          + "Owner"       = "Waleed"
          + "Region"      = "us-west-2"
        }
      + tier      = "Standard"
      + type      = "String"
      + value     = (sensitive value)
      + version   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
aws_ssm_parameter.this: Creating...
aws_ssm_parameter.this: Creation complete after 2s [id=example-param-staging]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
example param staging screenshot
Example param staging screenshot

Now you have applied the same code/resources with different values in different S3 buckets in multiple AWS accounts. Again, this is just another option to do Terraform AWS Multi-Account Setup.

Donate a coffee! 🙂

One-Time
Monthly
Yearly

Make a monthly donation

Make a yearly donation

Choose an amount

$2.00
$5.00
$10.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly

Learn more about the backend configuration.

Here’s the complete code in Github.

November 19, 2021November 19, 2021

Waleed S.3 Comments

Recent Posts

  • Amazon EKS IAM roles and policies with Terraform August 2, 2022
  • Terraform AWS Multi-Account Setup November 19, 2021
  • Terraform AWS KMS Multi-Region Keys November 12, 2021
  • Export AWS Security Groups & rules to CSV October 27, 2021
  • Get started with EC2 Image Builder in Terraform August 23, 2021
  • AWS Three-Tier VPC with ALB in Terraform June 3, 2021
  • AWS Three-Tier VPC network with Terraform April 15, 2021
  • AWS Service Control Policies with Terraform January 16, 2021
  • AWS CloudShell December 15, 2020
  • Intro to Terragrunt and Terraform November 27, 2020
  • Create new AWS accounts with Organizations and Terraform November 16, 2020
  • Create an EC2 IAM role with Terraform October 26, 2020
  • AWS IAM groups and policies – Terraform September 1, 2020
  • AWS KMS Customer Managed CMK with Terraform August 26, 2020
  • AWS Key management service (KMS) – Part 1 May 21, 2020

Want to learn more and stay up to date?

John
Smith
johnsmith@example.com

Categories

  • Aws
  • Certifications
  • Cloud Engineer
  • Google Cloud
  • Google Firebase
  • Planning
  • Training

Tags

AMI announcement api keys app asymmetric automation aws aws-alb aws-vpc aws access keys aws accounts aws account settings aws billing aws email marketing aws kms aws mfa aws multi-account aws organizations aws sign up azure best practice certificates certification certifications certified cheat sheets cli cloud cloud accounts cloud computing cloud engineer interview questions cloud governance cloudshell cloud storage cmk cmks console-tips-tricks dns ebs ec2-iam-role ec2-security-groups eks encryption exam export-security-groups firebase firebase realtime database firebase storage free tier google google cloud GovCloud sign up guide IaC iam iam-groups iam-policies image-builder install ios JavaScript jobs keys kms mfa troubleshoot naming organize permissions realtime database root mfa route 53 routing policies s3 security security questions security rules service control policies setup swift symmetric terraform terraform-modules terraform-setup terragrunt tips tools training vpc web white-papers

Search

Give a free coffee! :)

Sign up for newsletters

John
Smith
johnsmith@example.com
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Follow me on Twitter

My Tweets
Cloudly Engineer logo
New PostsGuide to become a great cloud engineer
John
Smith
johnsmith@example.com
 

Loading Comments...