Copying S3 Bucket Lifecycle Policies in Seconds

USE CASE

Going with the flow of the DevOps lifecycle, Engineers usually do testing on UAT and then release the same on production systems. So here you have got this UAT/testing bucket that you've been working on, and you've finally tested the perfect lifecycle policy. So for the next step, you want to apply the same policy to multiple production buckets. Now doing this manually can be a tedious task, but that's why I am writing this blog to make this a lot easier through the use of some AWS CLI and scripting commands. In this blog post, we're going to show you how to transfer an S3 bucket lifecycle policy from your UAT/testing bucket to your production bucket using the AWS CLI.

STEPS

Create one s3 bucket with the name: divyamnewprojectuat , for this bucket i will be adding the lifecycle policy manually.
Create two more s3 buckets with name : divyamnewprojectprod1, divyamnewprojectprod2.

[ec2-user@ip-172-31-5-124 ~]$ aws s3 mb s3://divyamnewprojectuat/
make_bucket: divyamnewprojectuat
[ec2-user@ip-172-31-5-124 ~]$ aws s3 mb s3://divyamnewprojectprod1/
make_bucket: divyamnewprojectprod1
[ec2-user@ip-172-31-5-124 ~]$ aws s3 mb s3://divyamnewprojectprod2/
make_bucket: divyamnewprojectprod2
[ec2-user@ip-172-31-5-124 ~]$

Go to Amazon S3\>Buckets\>divyamnewprojectuat\>Lifecycle configuration and setup your bucket policy according to your requirement
Here I wanted to create a policy for my bucket to move current versions of objects between storage classes (Objects move to Standard-IA)
- This use case is mostly applicable for storing logs.

Lifecycle policy with name "moveto" created!

Now, we need to get the lifecycle configurations of this bucket and redirect them into a JSON file.

//COMMAND
aws s3api get-bucket-lifecycle-configuration --bucket {bucketname} >> {filename}.json


[ec2-user@ip-172-31-5-124 ~]$ aws s3api get-bucket-lifecycle-configuration --bucket divyamnewprojectuat >> outputuatnew.json
[ec2-user@ip-172-31-5-124 ~]$ cat outputuatnew.json
{
    "Rules": [
        {
            "Filter": {}, 
            "Status": "Enabled", 
            "Transitions": [
                {
                    "Days": 30, 
                    "StorageClass": "STANDARD_IA"
                }
            ], 
            "ID": "moveto"
        }
    ]
}
[ec2-user@ip-172-31-5-124 ~]$

Now we will attach this policy to the production buckets through another command and using a for loop.

[ec2-user@ip-172-31-5-124 ~]$ for a in divyamnewprojectprod1 divyamnewprojectprod2; do aws s3api put-bucket-lifecycle-configuration --bucket $a --lifecycle-configuration file://outputuatnew.json; done
[ec2-user@ip-172-31-5-124 ~]$

BOOM! LIFECYCLE POLICY APPLIED TO BOTH BUCKETS!

So That's it for this blog, I Hope this will give you some insights about how AWS CLI commands can help you automate your work and redundant tasks. Do share your views and this blog if you like the content. Also, read another blog(5 Essential AWS S3 Commands for SREs with Practical Examples) on s3 Here .

AWS CLI Tricks: Copying S3 Bucket Lifecycle Policies in Seconds

Table of contents

USE CASE

STEPS