AWS S3

How to create an AWS S3 bucket using Terraform

The steps outlined in this blog post provide a guide through this basic process, while advanced configurations demonstrate its power for managing AWS S3 buckets.

When working with applications hosted on the cloud, you’ll most likely need a data storage solution. Some of the storage options that can be set up in the cloud include object storage, block storage, and file storage. While they vary depending on the provider, they typically offer scalability, flexibility, and cost-effectiveness and can be easily set up with an IaC tool such as Terraform. 

In this blog post, we look at how to create an AWS S3 bucket, one of AWS’ popular storage options, using Terraform. 

Why Terraform?

Before diving in further, let's understand why Terraform is such an attractive solution for infrastructure provisioning in AWS. Terraform allows you to define your infrastructure using code, providing an easily manageable way of expressing desired states with versioning, collaboration, and automation. Additionally, Terraform has excellent support for AWS. In fact, it's one of the most popular cloud providers used with Terraform.

Benefits of Using Terraform with AWS (S3 Buckets)

Here are some of the benefits of using Terraform to provision AWS resources:

  • Declarative Configuration: Terraform uses a declarative configuration language. This means that you define the desired end state of your infrastructure and let Terraform figure out how to get there. Instead of writing complex scripts that describe how to create, modify, and delete resources, you write simple configuration files that define the desired state of your infrastructure. Terraform then uses this configuration to create, modify, or delete resources as necessary to achieve the desired state.
  • Scalability: Terraform is well-suited for managing large-scale infrastructures. Its parallel execution capability enables the provisioning of multiple resources concurrently, speeding up the deployment process for extensive environments.
  • Reduced human error: Terraform automates the process of provisioning and managing your AWS resources, which can help reduce the risk of human error. This is because Terraform is able to follow your configuration file to the letter, and it will not make any changes to your infrastructure unless you explicitly tell it to do so.
  • Reproducibility: Terraform configurations are version-controlled and shared, making it simple to reproduce your infrastructure across various environments. This is ideal for testing new deployments or rolling out changes to production.
  • Reusability: Terraform modules offer a convenient way of encapsulating common infrastructure patterns for reuse across projects, saving both time and effort by not having to recreate configuration repeatedly.
  • Consistency: Terraform can help ensure that your infrastructure remains consistent across environments, helping reduce errors and simplify managing it. This can make managing infrastructure simpler.

Here are some specific benefits of using Terraform to create AWS S3 buckets:

  • Easy to Use: Terraform makes creating and managing S3 buckets simple. To get started, specify their desired properties, such as name, ACLs, and storage class, within a Terraform configuration file. Terraform will take care of creating the buckets in your AWS account.
  • Repeatable: Terraform configurations ensure your S3 buckets will be created consistently every time the Terraform apply command is run.
  • Version-controlled: Terraform configurations allow you to track changes made to S3 buckets over time, providing an effective means for rolling back to previous versions of your configuration if necessary.
  • Reusable: Terraform modules can be created to store common S3 bucket configurations. This saves both time and effort, as you won't need to write the same configuration repeatedly.
  • State Management: Terraform maintains a state file that keeps track of the current state of the infrastructure, including AWS S3 buckets. This state file enables Terraform to understand the differences between the declared configuration and the actual infrastructure, facilitating updates and modifications.

In the next section, we’ll see how to create an AWS S3 bucket using step. 

Prerequisites for Creating an AWS S3 Bucket with Terraform

Before diving into the hands-on process, ensure you have these necessary prerequisites in place:

AWS Account: An active AWS account to access and utilize AWS services, including S3. If you don’t have one, you can sign up here. 

IAM User with S3 Bucket Creation Permissions: To create S3 buckets using Terraform, you'll need an IAM (Identity and Access Management) user with the appropriate permissions.  

Terraform Installed: Have Terraform installed on your local machine. If you don’t have it, you can follow this link to install it.

Creating AWS S3 bucket using Terraform 

Now, let's go through the step-by-step process of creating an AWS S3 bucket using Terraform.

Step 1: Set Up AWS Credentials

You need to configure your AWS credentials to interact with AWS services using Terraform. You can either set up your credentials using the AWS CLI or provide them directly in your Terraform configuration. For simplicity, let's use the AWS CLI (learn more on how to configure credentials here).

$ aws configure

AWS Access Key ID [None]: YOUR_ACCESS_KEY_ID
AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY
Default region name [None]: YOUR_REGION_NAME
Default output format [None]: json

Replace YOUR_ACCESS_KEY, YOUR_SECRET_KEY, and YOUR_REGION with your AWS access key, secret key, and preferred AWS region, respectively.

Step 2: Create a Terraform Configuration File

Create a new directory for your Terraform configuration, and inside that directory, create a file named main.tf. This file will contain your Terraform configuration for creating an S3 bucket.

# main.tf
provider "aws" {
  region = "us-east-1"
}
resource "aws_s3_bucket" "my_bucket" {
  bucket  = "my-unique-bucket-name"
  tags    = {
	Name          = "MyS3Bucket"
	Environment    = "Production"
  }
}

In this example, we have defined an AWS provider specifying the region and a basic S3 bucket resource. The configuration uses "us-east-1" as the preferred AWS region and "my-unique-bucket-name" as the name for the S3 bucket.

Step 3: Initialize Terraform

Navigate to your Terraform configuration directory in the terminal and run the following command to initialize your working directory:

terraform init

This command downloads the necessary provider plugins and sets up your Terraform environment.

Step 4: Preview Changes

After initialization, run the following command to preview the changes Terraform will make to your infrastructure:

terraform plan

Review the output to ensure that Terraform will create the resources as expected. If everything looks good, you can proceed to the next step.

Step 5: Apply Changes

To create the S3 bucket, execute the following command:

terraform apply

Terraform will prompt you to confirm the execution of the plan. Type yes and press Enter. Terraform will then create the S3 bucket and output information about the resources created.

Step 6: Verify the S3 Bucket

Visit the AWS Management Console and navigate to the S3 service. You should see the newly created bucket with the name you specified in your Terraform configuration.

Congratulations! You've successfully created an AWS S3 bucket using Terraform. But this is just the beginning. Let's explore some additional configurations and best practices.

Advanced Configurations - Using Terraform with AWS S3

Versioning

Enabling versioning for your S3 bucket helps you manage and retain multiple versions of an object. To enable versioning, modify your main.tf file:

# main.tf
resource "aws_s3_bucket" "my_bucket" {
  bucket  = "my-unique-bucket-name"
  tags    = {
  Name           = "MyS3Bucket"
  Environment    = "Production"
  }
}
resource "aws_s3_bucket_versioning" "versioning_example" {
  bucket = aws_s3_bucket.my_bucket.id
  versioning_configuration {
    status = "Enabled"
  }
}

Logging

You can configure S3 bucket logging to record all requests made to your bucket. Add the following configuration to enable logging:

# main.tf
resource "aws_s3_bucket" "example" {
  bucket = "my-tf-example-bucket"
    tags = {
      Name        = "MyS3Bucket"
      Environment = "Production"
    }
}
resource "aws_s3_bucket_acl" "example" {
  bucket = aws_s3_bucket.example.id
  acl    = "private"
}
resource "aws_s3_bucket" "log_bucket" {
  bucket = "my-tf-log-bucket"
    tags = {
      Name        = "MyLogBucket"
      Environment = "Production"
    }
}
resource "aws_s3_bucket_acl" "log_bucket_acl" {
  bucket = aws_s3_bucket.log_bucket.id
  acl    = "log-delivery-write"
}
resource "aws_s3_bucket_logging" "example" {
  bucket        = aws_s3_bucket.example.id
  target_bucket = aws_s3_bucket.log_bucket.id
  target_prefix = "log/"
}

This example creates a separate S3 bucket for logs and configures the main bucket to log to the log bucket.

Access Control

AWS Identity and Access Management (IAM) allows you to control access to your S3 bucket. You can define IAM policies and attach them to your bucket. Here's a simple example:

# main.tf
resource "aws_s3_bucket" "example" {
  bucket = "my-tf-test-bucket"
}
resource "aws_s3_bucket_policy" "allow_access_from_another_account" {
  bucket = aws_s3_bucket.example.id
  policy = data.aws_iam_policy_document.allow_access_from_another_account.json
}
data "aws_iam_policy_document" "allow_access_from_another_account" {
  statement {
    principals {
      type        = "AWS"
      identifiers = ["123456789012"]
    }
    actions = [
      "s3:GetObject",
      "s3:ListBucket",
    ]
    resources = [
      aws_s3_bucket.example.arn,
      "${aws_s3_bucket.example.arn}/*",
    ]
  }
}

This example creates an IAM user and grants full access to the S3 bucket.

Conclusion

Terraform's easy, reproducible method for creating AWS S3 buckets makes the task more manageable than ever. The steps outlined in this blog post provide a guide through this basic process, while advanced configurations demonstrate its power for managing AWS S3 buckets.

Terraform's efficiency, consistency, flexibility, and comprehensive feature set make it a powerful tool for automating infrastructure provisioning and management. Its ability to simplify infrastructure deployments, reduce errors, and ensure consistent configurations across environments has made it a valuable tool for DevOps teams and infrastructure professionals.

Are you looking to polish your Terraform skills in a real-world environment? Explore our comprehensive Terraform for Beginners Course for hands-on learning. The course covers all of Terraform fundamentals using video lectures, interactive exercises, and hands-on labs to help you internalize concepts and commands.

Terraform Basics Training Course | KodeKloud
Learn Terraform with hands-on labs
KodeKloud logo

Avi

• 7 min read