Use S3 To Store Terraform State

It's probably a good idea to use a backend to store your Terraform state. This way you can work on your infrastructure as part of a team without others needing access to your local files. It also means that you don't have to worry about losing access to your computer in one way or another.

In this tutorial, we will configure Terraform to store the state in a private S3 bucket that you cant grant others access to through IAM permissions.

Related Posts

• Use PostgreSQL To Store Terraform State
• Terraform Cheatsheet

Steps

Create Bucket To Hold State

Firstly, create a private S3 bucket. You could do that through Terraform, but for simplicity we are going to assume that bucket was manually created through the AWS console.

Bucket Versioning

It is recommended that one enables the bucket versioning feature when creating the S3 bucket so that one could recover if someone were to accidentally delete the files.

If you are concerned about someone deleting the bucket itself, one can setup iterative S3 backups to another bucket through Duplicity.

Terraform Backend Configuration

To configure an S3 bucket as your backend, one needs to specify the backend in a top level terraform block like so:

terraform {
  backend "s3" {
    bucket = "my-state-bucket-name"
    key    = "my-terraform-project"
    region = "eu-west-2"
    shared_credentials_file = "~/.aws/credentials"
  }
}

I had issues with authentication, and found the easiest way was to specify the shared_credentials_file as shown in the snippet. You will need to change my-state-bucket-name to whatever the name is of the bucket you manually deployed earlier.

Full Example Code

A fully working example that asks you for a name to give a bucket before then creating it, would be like so:

terraform {
  backend "s3" {
    bucket = "tutorial-bucket-state"
    key    = "my-terraform-project"
    region = "eu-west-2"
    shared_credentials_file = "~/.aws/credentials"
  }
}

provider "aws" {
    region = "eu-west-2"
    shared_credentials_file = "~/.aws/credentials"
}

variable "bucket_name" {
    type = string
    description = "Provide the name to give an S3 bucket for Terraform to deploy."
}

resource "aws_s3_bucket" "simple_bucket" {
  bucket = var.bucket_name
  acl    = "private"

  tags = {
    Name = "myBucketTagName"
  }
}

For simplicity, this configuration uses the same AWS credentials file for both the S3 bucket and the deployment itself. However, you could use two different IAM users/files/credentials, and the minimum IAM policy that the state bucket user would require can be found here. This is particularly useful if one wishes to store the state in a separate AWS account.

Initialize

Once that's all done, one needs to initialize the project with:

terraform init

If you are trying to retrospectively do this to an existing codebase that has existing state, then one is going to have to run:

terraform init -migrate-state

However, doing this will require Terraform to have access to both the existing state and the new location. This is not a big deal if your existing state is in local files, but can be a bit of a configuration nightmare if not and I would refer you to the official docs for all of the configuration options.

Apply

Once you have finished initialization, you can then tell Terraform to apply your configuration to deploy the infrastructure.

terraform apply

You will now see that the code deployed an S3 bucket for you, and there is a state file in your state bucket(1), with a file name equal to the key value in your terraform block (2).

References

• Terraform Docs - S3
• Github - Terraform - Error loading state: AccessDenied: Access Denied (AWS S3 backend)