Skip to content
Cloudly Engineer logo

The Cloudly Engineer

Enough talk, let's build!

  • Home
  • About Me
  • All Posts
  • Downloads
  • Register
  • Other great cloud related sites
  • Password Reset
  • Log Out
Search

Tag: aws multi-account

multiple overlay patterns of a colorful design

Terraform AWS Multi-Account Setup

Terraform is an elite open-source software that helps solve a lot of cloud automations. It’s very scalable and easy to use. But Terraform’s flexibility has caused debates on how to setup a perfect code directory. A code directory should be setup in a way where it’s easy to read, quickly deploy to various environments and accounts without having to repeat or copy and paste code. Terragrunt to the rescue! That helped tremendously but it can become challenging to manage multiple tools or the code directory structure becomes unmanageable in the long run. In this post I like to share another method of Terraform directory setup for AWS multiple accounts and environments. I’m not saying this is the best or better solution than other ones… it’s just another way of doing AWS Multi-Account setup with Terraform.

Old Way

This is the structure I used to use with Terragrunt. Terragrunt would dynamically change the S3 Backend Key parameter based on the directory structure.

├── dev
│   ├── inputs.yml
│   ├── terragrunt.hcl
│   └── vars.tf
├── staging
│   ├── inputs.yml
│   ├── terragrunt.hcl
│   └── vars.tf
├── terragrunt.hcl
├── main.tf

New Way

Now this is the structure I use, this setup doesn’t require Terragrunt. Notice it’s less files and it’s all Terraform files.Let’s review some of the files in detail.

├── accounts
│   ├── dev
│   │   ├── backend.conf
│   │   └── terraform.tfvars
│   └── staging
│       ├── backend.conf
│       └── terraform.tfvars
├── Makefile
├── main.tf

For this setup the main.tf still declares the backend but without any of its parameters! This is called partial configuration in Terraform. We’ll define the rest in the backend.conf file.

terraform {
  backend "s3" {
    key = "tf-partial-backend-example/terraform-main.tfstate"
  }
}

Backend.conf

We need to give all the remaining required and optional parameters of the “s3” backend defined in the main.tf in this backend.conf file. In this example I’m declaring the key path in the main.tf so it will be the same for every S3 bucket. If I wanted it to be unique then I would add it to the backend.conf. You can do that with any of the parameters!

#  Terraform state for S3 backend config variables
bucket   = "dev-example-s3-bucket-tf"
region   = "us-east-1"
role_arn = "arn:aws:iam::123456789:role/svc-terraform-role"

terraform.tfvars

This file defines all the values for your variables for that specific environment or account.

region      = "us-east-1"
environment = "dev"
account_id  = "123456789"

Terraform init, plan, and apply

In this design initiating terraform requires few options. We need to the define the rest of the backend configuration, we do that by passing the -backend-config option. Next option is the -reconfigure, is very important because we’re working with multiple accounts and environments we need to ensure the local cache .terraform* files are well, reconfigured each time. That’s the catch for this pattern!

terraform init -backend-config accounts/dev/backend.conf -reconfigure

# Expected output
Terraform has been successfully initialized!

# Plan
terraform plan -var-file accounts/dev/terraform.tfvars

# Apply
terraform apply -var-file accounts/dev/terraform.tfvars

Makefile

You probably noticed a file called “Makefile” in the directory. A makefile is just a way to shorthand or automate our long commands. Here’s a makefile for Terraform.

SHELL := /usr/bin/env bash

# HOW TO EXECUTE

# Executing Terraform PLAN
#	$ make tf-plan-example env=<env>
#    e.g.,
#       make tf-plan-example env=dev

# Executing Terraform APPLY
#   $ make tf-apply-example env=<env>

# Executing Terraform DESTROY
#	$ make tf-destroy-example env=<env>

all-test: clean tf-plan-example

.PHONY: clean
clean:
	rm -rf .terraform

.PHONY: tf-plan-example
tf-plan-example:
	terraform fmt && terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform validate && terraform plan -var-file accounts/${env}/terraform.tfvars

.PHONY: tf-apply-example
tf-apply-example:
	terraform fmt && terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform validate && terraform apply -var-file accounts/${env}/terraform.tfvars -auto-approve

.PHONY: tf-destroy-example
tf-destroy-example:
	terraform init -backend-config accounts/${env}/backend.conf -reconfigure && terraform destroy -var-file accounts/${env}/terraform.tfvars

Now all we have to do for TF init is call like this:

make tf-plan-example env=dev

Or for the staging environment like this

make tf-plan-example env=staging

Apply time!

make tf-apply-example env=dev

# aws_ssm_parameter.this will be created
  + resource "aws_ssm_parameter" "this" {
      + arn       = (known after apply)
      + data_type = (known after apply)
      + id        = (known after apply)
      + key_id    = (known after apply)
      + name      = "example-param-dev"
      + tags      = {
          + "Environment" = "dev"
          + "Name"        = "example-param-dev"
          + "Owner"       = "Waleed"
          + "Region"      = "us-east-1"
        }
      + tags_all  = {
          + "Environment" = "dev"
          + "Name"        = "example-param-dev"
          + "Owner"       = "Waleed"
          + "Region"      = "us-east-1"
        }
      + tier      = "Standard"
      + type      = "String"
      + value     = (sensitive value)
      + version   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
aws_ssm_parameter.this: Creating...
aws_ssm_parameter.this: Creation complete after 0s [id=example-param-dev]
example param dev screenshot
Example param dev screenshot

Now let’s deploy the same code in a different region and environment with just a single change in the command line.

make tf-apply-example env=staging

# aws_ssm_parameter.this will be created
  + resource "aws_ssm_parameter" "this" {
      + arn       = (known after apply)
      + data_type = (known after apply)
      + id        = (known after apply)
      + key_id    = (known after apply)
      + name      = "example-param-staging"
      + tags      = {
          + "Environment" = "staging"
          + "Name"        = "example-param-staging"
          + "Owner"       = "Waleed"
          + "Region"      = "us-west-2"
        }
      + tags_all  = {
          + "Environment" = "staging"
          + "Name"        = "example-param-staging"
          + "Owner"       = "Waleed"
          + "Region"      = "us-west-2"
        }
      + tier      = "Standard"
      + type      = "String"
      + value     = (sensitive value)
      + version   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
aws_ssm_parameter.this: Creating...
aws_ssm_parameter.this: Creation complete after 2s [id=example-param-staging]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
example param staging screenshot
Example param staging screenshot

Now you have applied the same code/resources with different values in different S3 buckets in multiple AWS accounts. Again, this is just another option to do Terraform AWS Multi-Account Setup.

Donate a coffee! 🙂

One-Time
Monthly
Yearly

Make a monthly donation

Make a yearly donation

Choose an amount

$2.00
$5.00
$10.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly

Learn more about the backend configuration.

Here’s the complete code in Github.

November 19, 2021November 19, 2021

Waleed S.3 Comments

Create new AWS accounts with Organizations and Terraform

AWS Organizations

It’s important to understand what AWS Organizations service is in order to create new AWS accounts with Terraform. Even if you don’t want to create new AWS accounts with Terraform and instead create them manually, knowing AWS organizations features will help you to be a top notch cloud engineer! The introduction video of what AWS Organizations is the best place to start.

What is AWS Organizations introduction by AWS.

Besides the rich functionality it provides, the next best thing about is that it’s 100% free! Here’s an official link to the AWS documentation. Be sure to read the AWS Organizations best practices too.

AWS Organizations is not enabled by default. Once we create the resource via Terraform it will be enabled.

AWS Organizations default state
AWS Organizations default state

AWS Organizations Permissions

At minimum and possibly more you will need the following IAM permissions. Remember this needs to be in your management/root account.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ManageOrganizations",
      "Effect": "Allow",
      "Action": [
        "organizations:CreateOrganization",
        "organizations:EnableAWSServiceAccess",
        "organizations:DisableAWSServiceAccess",
        "organizations:DescribeOrganization",
        "organizations:ListRoots",
        "organizations:ListAccounts",
        "organizations:ListAWSServiceAccessForOrganization"
      ],
      "Resource": "*"
    }
  ]
}

AWS Organizations Terraform Code

If you have not used Terraform before I would encourage you to start at Terraform and Terragrunt introduction. Otherwise continue. Remember this needs to be applied on your management/root account.

# Provides a resource to create an AWS organization.
resource "aws_organizations_organization" "this" {

  # List of AWS service principal names for which 
  # you want to enable integration with your organization

  aws_service_access_principals = [
    "cloudtrail.amazonaws.com",
    "config.amazonaws.com",
    "sso.amazonaws.com"
  ]

  feature_set = "ALL"
}

After applying just the code above we have enabled AWS Organizations… and it’s requesting an email confirmation. Be sure to check your inbox and confirm within 24 hours.

AWS Organizations enabled
AWS Organizations enabled

Only after we confirmed the email address we can now invite existing AWS accounts to join our organization or create new ones!

Import an existing AWS Organizations

If you have already created/enabled AWS Organizations with the AWS CLI or through the console you do have the option to import and manage with Terraform going forward.

# If you using Terraform only
terraform import aws_organizations_organization.my_org o-1234567

# If you use Terragrunt
terragrunt import aws_organizations_organization.my_org o-1234567

Create new AWS account

The Terraform code below will create a new member aka AWS account in the organization. But in order to achieve well-architected structure you must place your accounts in OUs. This way the Service Control Policies (SCPs) will be applied to the account immediately. Remember to follow a naming convention for your account and OU names.

resource "aws_organizations_organizational_unit" "workload" {
  name      = "workload"
  parent_id = aws_organizations_organization.this.roots[0].id
}


resource "aws_organizations_organizational_unit" "dev" {
  name      = "dev"
  parent_id = aws_organizations_organizational_unit.workload.id

  depends_on = [
    aws_organizations_organizational_unit.workload
  ]
}

resource "aws_organizations_account" "dev" {
  # A friendly name for the member account
  name  = "example-dev"
  email = "validemail@domain.com"

  # Enables IAM users to access account billing information 
  # if they have the required permissions
  iam_user_access_to_billing = "ALLOW"

  tags = {
    Name  = "engineer-dev"
    Owner = "Waleed"
    Role  = "development"
  }

  parent_id = aws_organizations_organizational_unit.dev.id
}

After applying this Terraform code you will see a new account member in the AWS Organizations. Sadly, you will not get a set of credentials automatically. If your organization has enabled AWS SSO then you can use those credentials to switch roles to the new account after you add the account for your users/groups.

Getting the new AWS Accounts password

Since this account was created from AWS Organizations you have to navigate to the AWS console to sign in as the root account. Then enter your new accounts email address and then select Forgot Password. Then enter the captcha code and you will receive an email on setting a new password for your new AWS account. Ensure to setup MFA after logging in to secure your account login.

Code structure

Here’s the way I structured my code for this project. I use Terragrunt on top of Terraform. (You don’t have to use Terragrunt at all for this to work, it’s just one way to manage Terraform code at scale. If you want to learn how to setup your Terraform structure without Terragrunt, take a look at this post.) Management/root is my only account/environment in this project.

aws/
   org
    ├── README.md
    ├── main.tf
    ├── management
    │   ├── inputs.yml
    │   ├── terragrunt.hcl
    │   └── vars.tf
    └── terragrunt.hcl

That’s it for now. In future posts I’ll continue to cover more of AWS Organizations by creating OU’s, Service policies and more so be sure to subscribe!

TOP 13 CLOUD ENGINEER POSITION INTERVIEW QUESTIONS AND ANSWERSBe prepared for you interview!
John
Smith
johnsmith@example.com

November 16, 2020December 26, 2021

Waleed S.2 Comments

AWS Multi-Account Architecture

For small organizations or small dev/test having a single AWS account will most likely do just fine. But for production workloads you’ll need to consider AWS multi-account architecture. There are many reasons you’ll need to use AWS multi-account architecture (listed below), so it’s best to start right. Before you get started take a minute to read this https://cloudly.engineer/2019/wait-dont-create-your-aws-account-yet/aws/. You’ll have to think about the size of your team (people), timeline, budgets and team experience. Each account will be purposely created for specific role. One account would just be the payer account, which is your management account. This AWS account will host your AWS organizations services, too. Do enable the standard IAM account settings, enable CloudTrail logs and other basic security services. Do not run any other services on this account!

Why Multiple AWS accounts?

  • Cost separation for budgets and reporting purpose
    • Can easily consolidate billing!
  • Bypass AWS services hard quota limitations
  • Security separation
  • Error tolerance or blast radius problems
  • Governance and compliance purposes
  • Different teams and contracts
  • Code pipeline and much more.

Account Permissions

AWS’s organizations come with another key component besides the consolidated payment. The Service Control Policies (SCPs). SCP’s are just service only permissions for accounts. It’s not user permissions. You can block or explicitly enable services from the management account to its child or to any child account directly. Or even block certain regions or certain EC2 (Elastic Cloud Compute) instances types. Everyone should block those high performance compute (HPC) instances if you don’t plan on using them ever! Otherwise an accidental use of HPC instances can cost you thousands in just few hours.

AWS Multi-Account architectures

Starter Pack

This is a fast and scalable option. Create your management account then utilize AWS Control Tower to create your other accounts.

starter pack architecture
Production Starter OU layout

Well-Architected Structure

Well-Architected structure
Well-Architected structure

The Workspace OU is for accounts that will provision Amazon Workspaces, virtual desktops. The infrastructure OU is for the core or shared resources, such as network, MS active directory account, image builder, pipelines, etc. The sandbox OU is for AWS accounts to do testing and learning of various AWS services and features.

Organizational Units

AWS Organization Units or OU’s are not the same as Microsoft Active Directory OU’s. OU’s here are a way of grouping AWS accounts to manage account permissions (SCPs) in a manageable and scalable manner.

Benefits of OU’s:

  • Share SCPs
  • Common functionality
  • Share resources across multiple accounts within an OU

Management Account

AWS Control Tower is the easiest way to manage your OU’s, accounts, AWS Single Sign On, apply guardrails (SCPs and AWS Config Rules), and manage your landing zone.

The management account will host all your billing information. The breakdown of costs per account and much more. Do set your billing alerts here! As of the time of this writing I wouldn’t recommend using the Amazon CloudWatch billing alert. I recommend using AWS budgets feature only! Why? Because the billing metrics value is different than the AWS budgets. The AWS budgets is the total cost while the CloudWatch metric is only the AWS services; it’s missing any AWS Marketplace charges.

Subscribe! I’ll be posting how to create accounts with the AWS CLI and managing all accounts with Terraform! Here’s how to provision an AWS account with Terraform.

John
Smith
johnsmith@example.com

As always if you see any errors, mistakes, have suggestions or questions please comment below. Don’t forget to like, share, and subscribe for more! 🙂

August 5, 2019December 23, 2021

Waleed S.2 Comments

Recent Posts

  • Amazon EKS IAM roles and policies with Terraform August 2, 2022
  • Terraform AWS Multi-Account Setup November 19, 2021
  • Terraform AWS KMS Multi-Region Keys November 12, 2021
  • Export AWS Security Groups & rules to CSV October 27, 2021
  • Get started with EC2 Image Builder in Terraform August 23, 2021
  • AWS Three-Tier VPC with ALB in Terraform June 3, 2021
  • AWS Three-Tier VPC network with Terraform April 15, 2021
  • AWS Service Control Policies with Terraform January 16, 2021
  • AWS CloudShell December 15, 2020
  • Intro to Terragrunt and Terraform November 27, 2020
  • Create new AWS accounts with Organizations and Terraform November 16, 2020
  • Create an EC2 IAM role with Terraform October 26, 2020
  • AWS IAM groups and policies – Terraform September 1, 2020
  • AWS KMS Customer Managed CMK with Terraform August 26, 2020
  • AWS Key management service (KMS) – Part 1 May 21, 2020

Want to learn more and stay up to date?

John
Smith
johnsmith@example.com

Categories

  • Aws
  • Certifications
  • Cloud Engineer
  • Google Cloud
  • Google Firebase
  • Planning
  • Training

Tags

AMI announcement api keys app asymmetric automation aws aws-alb aws-vpc aws access keys aws accounts aws account settings aws billing aws email marketing aws kms aws mfa aws multi-account aws organizations aws sign up azure best practice certificates certification certifications certified cheat sheets cli cloud cloud accounts cloud computing cloud engineer interview questions cloud governance cloudshell cloud storage cmk cmks console-tips-tricks dns ebs ec2-iam-role ec2-security-groups eks encryption exam export-security-groups firebase firebase realtime database firebase storage free tier google google cloud GovCloud sign up guide IaC iam iam-groups iam-policies image-builder install ios JavaScript jobs keys kms mfa troubleshoot naming organize permissions realtime database root mfa route 53 routing policies s3 security security questions security rules service control policies setup swift symmetric terraform terraform-modules terraform-setup terragrunt tips tools training vpc web white-papers

Search

Give a free coffee! :)

Sign up for newsletters

John
Smith
johnsmith@example.com
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Follow me on Twitter

My Tweets
Cloudly Engineer logo
New PostsGuide to become a great cloud engineer
John
Smith
johnsmith@example.com
 

Loading Comments...