Terraform, an open-source infrastructure as code (IAC) tool developed by HashiCorp, provides an effective means to define and provision infrastructure resources. You can automate the process of creating, editing, and deleting resources across cloud-based and on-premise environments using Terraform's powerful features. One of the critical implementations that makes it so effective is state management. 

In this blog, we will delve into the significance of Terraform state, its management, and practical examples to illustrate its importance.

Learn more about IaC here: What is Infrastructure-as-Code (IaC)?

What is Terraform State?

At its core, Terraform state provides an overview of your infrastructure's resources and properties; this represents what Terraform estimates will exist based on your configuration. Terraform uses state information to understand which resources have been created, updated, or destroyed during each run. This information allows Terraform to make intelligent decisions regarding which resource should be created, updated, or destroyed during future runs.

Terraform state is stored in a file named terraform.tfstate in the root directory of your Terraform project. This JSON file stores information about all your resources, such as IDs, attributes, and dependencies.

Try the Terraform State Lab for free

Why is Terraform State Important?

The importance of the Terraform state cannot be overstated. Here are some key reasons why it's crucial:

Tracking Resource State: Terraform uses its state file to accurately account for your infrastructure's current state. Terraform automatically tracks the state of your resources to understand their current configuration and to make informed decisions when applying changes. 

Resource Dependency Management: The state also helps Terraform understand the dependencies between resources. If Resource A depends on Resource B, Terraform needs to know this relationship to provision them in the correct order. State information guides Terraform in maintaining this order during operations. Learn more about Terraform dependencies from this blog: How To Use Terraform depends on Meta-Argument.

Plan and Apply Operations: Before making changes to your infrastructure, Terraform generates an execution plan that outlines what it intends to do. This plan compares the desired state (your configuration) and the current state (the state file). Terraform wouldn't know what changes to propose or apply without the state.

Resource Deletion and Cleanup: Terraform employs the state to identify resources you no longer require for deletion. Based on this state information, you only need to pay for what's being actively utilized, helping manage costs more effectively.

How to Manage Terraform State? 

Terraform keeps an eye on all resources under its management and their current states, ensuring accurate infrastructure updates. There are two primary methods for doing so - local and remote.

Local State Management

Local state management is the default approach employed by Terraform. Under this method, Terraform creates a local state file in the same directory as your Terraform configuration files to facilitate state administration locally. Here's how you can accomplish it:

Initializing a Terraform Project

The first step is to initialize your Terraform project. This command sets up the necessary plugins and backend configurations.

terraform init

Creating Resources

Define infrastructure resources in your Terraform configuration file (e.g., main.tf). For example, to launch an AWS EC2 instance:

resource "aws_instance" "demo" {
  ami       	= "ami-0c88g674cbfafe1f0"
  instance_type = "t2.micro"
}

Apply the configuration to create the resources:

terraform apply 

Terraform will add the details of the provisioned resource in the terraform.tfstate file. 

Modifying Resources

To modify existing resources, make changes to your Terraform configuration and apply the changes again. For instance, we can add a tag to the resource created in the previous step.

resource "aws_instance" "demo" {
  ami       	= "ami-0c88g674cbfafe1f0"
  instance_type = "t2.micro"
  tags = {
    Name = "MyDemoInstance"
  }
}

Apply the configuration to modify the resources:

terraform apply

Terraform will identify any discrepancies between your desired state in your configuration and what is currently stored in the local state file and make any necessary updates.

Deleting Resources and Cleanup

To delete resources, remove them from your configuration file and apply the changes.

The resources will be destroyed, and the state file will be updated to reflect the changes.

Challenges with Local State Management

Below are the limitations of local state management:

• Limited Collaboration: Local state files are tied to individual machines, making it more difficult for teams to collaborate effectively.
• Risk of Data Loss: Local state files are vulnerable to data loss if a machine crashes or the state file is accidentally deleted.
• Concurrency Issues: Within teams, multiple members may attempt to implement changes simultaneously, leading to conflicts in configuration.

Remote State Management

Remote state management is a more robust approach suitable for team environments and production use cases. In this method, the Terraform state file is stored remotely in a shared location that all team members can access. Common choices for remote state storage include Amazon S3, Azure Blob Storage, and Terraform Cloud. Here's how to set up remote state management:

Initialize Remote State

Initialize your Terraform project:

terraform init

However, instead of using the default local state backend, specify a remote state backend in your configuration.

Configure Remote State Backend

In your Terraform configuration (e.g., main.tf), specify the backend configuration to use the remote state. For example, using Amazon S3 as the remote state backend: 

terraform {
  backend "s3" {
	bucket     	= "aws-s3-bucket-demo"
	key        	= "s3://aws-s3-bucket-demo/statefile.tfstate"
	region     	= "us-east-1"
	encrypt    	= true
	dynamodb_table = "demo-dynamodb-table"
  }
}

In this example, we use an S3 bucket to store the state file and DynamoDB for locking. Below are the details of the above configurations:

• bucket: This is the name of the S3 bucket where Terraform will store its state file and related data. 
• key: This is the path within the S3 bucket where Terraform will store its state file. 
• region: This specifies the AWS region in which the S3 bucket is located.
• encrypt: When set to true, it means that Terraform will encrypt the state file when storing it in S3, providing additional security.
• dynamodb_table: This setting is used for state locking, which prevents concurrent state modifications.

Want to learn about Terraform variables? Check out this blog: Terraform Variables: Types & Use Cases for Beginners.

Apply Changes with Remote State

Apply your Terraform configuration using the command below:

terraform apply

Terraform will store the state remotely in the specified backend, making it accessible to all team members.

Benefits of Remote State Management

Remote state management offers several advantages:

• Improved collaboration: All team members can now access and update the same state file without manually sharing and synchronizing state files.
• Increased security: Remote state storage solutions often have built-in security features, including access control mechanisms.
• Better data protection: Cloud-based remote state solutions typically offer data redundancy, backups, and versioning to protect state data from accidental deletion or corruption.

Choose an appropriate remote state management solution to ensure better collaboration, data integrity, and concurrency control for your Terraform projects.

Let's walk through a practical example of how to manage Terraform state. In this example, we'll demonstrate both local and remote state management.

Terraform State Management Example

In this scenario, we want to use Terraform to provision and manage AWS EC2 instances.

1. Local State Management

Step 1: Set Up Your Project

Create a new directory for your Terraform project and navigate into it: 

mkdir terraform-ec2-demo
cd terraform-ec2-demo

Inside this directory, create a Terraform configuration file named main.tf with the following content:

# main.tf
provider "aws" {
  region = "us-east-1"
}
resource "aws_instance" "demo" {
  ami         	 = "ami-0c88g674cbfafe1f0"
  instance_type = "t2.micro"
}

This configuration defines an AWS EC2 instance resource.

Step 2: Initialize and Apply

Initialize your Terraform project:

terraform init

Now, apply the configuration to create the EC2 instance:

terraform apply

Terraform will generate a state file named terraform.tfstate in the same directory. This file stores information about the provisioned EC2 instance.

Step 3: Modify and Update.

Let's make a change to the configuration. Update main.tf by adding a "Name" tag to the EC2 instance:

# main.tf 
provider "aws" {
  region = "us-east-1"
}
resource "aws_instance" "demo" {
  ami       	 = "ami-0c88g674cbfafe1f0"
  instance_type = "t2.micro"
  tags = {
	Name = "DemoEC2Instance"
  }
}

Apply the changes:

terraform apply

Terraform will detect the modification and update the EC2 instance accordingly. The state file is also updated to reflect the new configuration.

Step 4: Delete Resources and Cleanup

To delete the EC2 instance, remove below configuration in the main.tf:

# main.tf
provider "aws" {
  region = "us-east-1"
}

Apply the changes to delete the resource:

terraform apply

The EC2 instance will be deleted, and the state file will be updated accordingly.

2. Remote State Management with Amazon S3 and DynamoDB

Step 1: Create an S3 Bucket and DynamoDB Table

• Log in to the AWS Management Console.
• Create an S3 bucket to store your Terraform state files.
• Enable versioning for the S3 bucket for data protection.
• Create a DynamoDB table to manage state locking. You can follow the steps mentioned in the previous section.

Step 2: Configure Remote State in Terraform.

Update your main.tf configuration to use remote state management with S3 and DynamoDB:

# main.tf
provider "aws" {
  region = "us-east-1"
}
terraform {
 backend "s3" {
	bucket     	= "aws-s3-bucket-demo"
	key        	= "/home/ubuntu/demo/statefile.tfstate"
	region     	= "us-east-1"
	encrypt    	= true
	dynamodb_table = "demo-dynamodb-table"
  }
}
resource "aws_instance" "demo" {
  ami       	 = "ami-0c88g674cbfafe1f0"
  instance_type = "t2.micro"
  tags = {
	Name = "DemoEC2Instance"
  }
}

Step 3: Initialize and Apply with Remote State

Initialize your Terraform project to configure the remote state:

terraform init

Now, apply the configuration to create the EC2 instance:

terraform apply

Terraform will provision an EC2 instance and store its state in an S3 bucket using DynamoDB for locking purposes.

Remote state management enables team members to work together and ensure the security and integrity of your Terraform state.

The above examples illustrate how Terraform state can be managed locally and remotely using AWS EC2 instances as infrastructure resources. Select an approach suitable for managing the state of your projects based on your needs.

Best Practices for Managing Terraform State

Here are some best practices for managing Terraform state:

• Keep state in a remote backend. By default, Terraform stores state locally on disk; however, for optimal sharing and recovery, it would be much more helpful to store state remotely in the Cloud. Doing this allows teammates to share it more efficiently and allows for easy recovery in case of loss of state.
• Create separate state files for every environment. It's wise to keep individual state files for different environments, for example, production, staging, and development environments. It prevents a situation where a change in one environment affects the other environments by mistake. 
• Utilize version control with your Terraform configuration. This allows you to keep track of changes made and roll back if necessary. It also enables you to keep an audit trail of each modification that’s made.
• Regularly back up your state file. Regular backups of your state files can help ensure recovery should they become lost or deleted unexpectedly.

Conclusion

Management of Terraform state is an integral component of infrastructure as code. From local to remote state management solutions, understanding how Terraform keeps track of resources and handles concurrency is vital for error-free infrastructure deployments. Following best practices can ensure collaboration, data integrity, and efficient management with Terraform.

Are you looking to polish your Terraform skills in a real-world environment? Enroll in our Terraform for Beginners Course, which covers all of Terraform fundamentals. It includes video lectures, interactive exercises, and hands-on labs to help you internalize concepts and commands.