DVwrites
Optimizing Cloud Costs using Automation (Python): Deleting Unused EBS Snapshots with Python and AWS Lambda

Optimizing Cloud Costs using Automation (Python): Deleting Unused EBS Snapshots with Python and AWS Lambda

Divyam Sharma's photo
Divyam Sharma
·

2 min read

Table of contents

The Challenge: Cloud spend can spiral out of control if unutilized resources accumulate. EBS snapshots, S3 buckets, and clusters left behind by development teams can silently eat away at your budget.

The Solution: Leverage DevOps/SRE expertise to automate resource cleanup. In this blog, we'll focus on deleting stale EBS snapshots with a Python Lambda function powered by boto3.

How it Works:

  1. Discovery: The Lambda function scans all your EBS snapshots using the AWS API.

  2. Attachment Check: It identifies volumes associated with each snapshot.

  3. Instance Verification: For attached volumes, it confirms their connection to running instances.

  4. Cleanup: Unattached or orphaned snapshots (not connected to running instances) are automatically deleted, reclaiming valuable storage space.

Benefits:

  • Reduced Costs: Eliminates unnecessary snapshot charges, leading to direct cost savings.

  • Improved Resource Management: Ensures efficient use of cloud resources, promoting better budgeting and organization.

  • Automated Efficiency: Leverages automation to handle repetitive tasks, freeing up DevOps/SRE teams for more strategic work.

Lets convert these steps to code:

import boto3

def lambda_handler (event,context):
    ec2 = boto3.client('ec2')

    #Get EBS Snapshots
    instance_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name','Values': ['running']}])
    active_instance_ids= set()

    response = ec2.describe_snapshots(OwnerIds=['self'])

    for reservation in instance_response['Reservations']:
        for instance in reservation['Instances']:
            active_instance_ids.add(instance['InstanceId'])
# Iterate through the snapshots and delete snapshots with no volume id,also delete volumes not attached to running instance
    for snapshot in response['Snapshots']:
        snapshot_id = snapshot['SnapshotId']
        volume_id = snapshot.get('VolumeId')

        if not volume_id:
            ec2.delete_snapshot(SnapshotId=snapshot_id)
            print(f"Deleted EBS snapshot {snapshot_id} successfully as it was not attached to any volume.")

        else:
            try:
                volume_response = ec2.describe_volumes(VolumeIds=[volume_id])
                if not volume_response['Volumes'][0]['Attachments']:
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted Snapshot {snapshot_id} as it was taken from a volume not attached to any running instance")
            except  ec2.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Snapshot deleted as associated Volume not found ")

  • Run this code as lambda function

  • Make sure to add the necessary permissions to the IAM role being used by the Lambda function (CONFIGURATION -> PERMISSION)

  • Increase the timeout if needed. (CONFIGURATION)

SNAPSHOT DELETED!

SUMMARY

By automating unused EBS snapshot deletion, you can significantly reduce cloud costs and optimize resource utilization. This is just one example of how DevOps/SRE teams can leverage automation to drive cloud efficiency. With proactive steps like these, you can ensure your cloud environment is cost-effective and well-managed.

I hope this blog post provides a more engaging and informative explanation of the provided code and its significance in managing cloud costs!

NOTE: This is a demonstration of how python automation can be done on cloud infrastructure, always be careful of deleting any snapshots which are used in production

PROJECT IDEA CREDITS:

Abhishek Veeramala

YOUTUBE

AWSlambdacost-optimisationPython