Mitigating AWS Volume Accumulation: Understanding and Addressing the Issue of Excessive Volume Growth

Last updated on 

Problem

Excessive accumulation of volumes on AWS. You can check the quantity using the AWS Command Line Interface with some tricks: 

aws --profile your_profile ec2 describe-volumes --region us-west-2 --filters Name=status,Values=available | jq '.[] | .[].VolumeId' | wc -l

719

This command is using the AWS Command Line Interface (CLI) along with the jq tool to perform several operations:

  1. aws --profile your_profile ec2 describe-volumes --region us-west-2 --filters Name=status,Values=available: This part of the command uses the AWS CLI to describe available EBS volumes in the specified AWS region (us-west-2) under the AWS profile named your_profile. It uses a filter to only show volumes with the status "available".
  2. | jq '.[] | .[].VolumeId': This part of the command uses jq, a lightweight and flexible command-line JSON processor, to process the JSON output from the previousAWS CLI command. It extracts the VolumeId field from each item in the JSON array.
  3. | wc -l: Finally, this part of the command uses the wc command with the -l option to count the number of lines in the output. Since each line corresponds to a VolumeId, this effectively counts the number of available EBS volumes in the specified region and profile.

In summary, this command fetches information about available EBS volumes in a specific AWS region and profile, extracts the VolumeId from the JSON output, and then counts the number of available volumes.

Cause

The delete on termination option was not set to yes in my ASG Launch template, leading to the accumulation of volumes when instances inside the auto-scaling group were deleted.

Solution

You could go directly to the AWS panel and delete them one by one, but when the quantity is significant, this becomes unfeasible.

A more efficient method:

Iterates through each available EBS volume in the AWS region us-east-1 under the AWS profile your_profile and deletes them one by one.

for VOLUME_ID in `aws --profile your_profile ec2 describe-volumes --region us-east-1 --filters Name=status,Values=available | jq '.[] | .[].VolumeId' | sed 's/"//g'`; do aws --profile your_profile ec2 delete-volume --region us-east-1 --volume-id ${VOLUME_ID}; done

This command performs a series of operations using the AWS CLI and other Unix command-line tools:

  1. for VOLUME_ID in `aws --profile your_profile ec2 describe-volumes --region us-east-1 --filters Name=status,Values=available | jq '.[] | .[].VolumeId' | sed 's/"//g'`; do: This part initiates a loop where each iteration will process one available EBS volume. It uses the AWS CLI to describe available EBS volumes in the specified AWS region (us-east-1) under the AWS profile named your_profile. The jq command extracts the VolumeIdfield from the JSON output, and sed removes any double quotes (") from the VolumeId.
  2. aws --profile your_profile ec2 delete-volume --region us-east-1 --volume-id ${VOLUME_ID};: Within the loop, this command uses the AWS CLI to delete the EBS volume identified by ${VOLUME_ID} in the specified region (us-east-1) under the AWS profile your_profile.
  3. done: This concludes the loop, meaning the command will repeat the delete operation for each available EBS volume found in the initial AWS CLI command.

In summary, this command finds all available EBS volumes in a specific AWS region and profile, then deletes each one in sequence.