Archived

This repo has been archived. Some of the useful functions have move to: USDOT-SDC/dev-utils/resource-cleanup

cleanup-worker

Utilities to identify and delete unused AWS resources within the US DOT Secure Data Commons (SDC) ecosystem.

Table of Contents

I. Release Notes

II. Usage Example

III. Configuration

IV. Installation

V. Design and Architecture

VI. Unit Tests

VII. File Manifest

VIII. Development Setup

IX. Release History

X. Contact Information

XI. Contributing

XII. Known Bugs

XIII. Credits and Acknowledgment

XIV. CODE.GOV Registration Info

XV. Code Quality Monitor

---

I. Release Notes

January 7, 2022. SDC cleanup-worker Release 1.0

What's New in Release 1.0

• AWS infrastructure to periodically detect and delete unused EBS volumes:
  • Lambda function implementation to support removal of unused volumes
  • Terraform scripts to provision EventBridge Rule, lambda function, and IAM roles and policies to support the functionality

II. Usage Example

III. Configuration

IV. Installation

Unused EBS volume cleanup installation

Deployment Plan

• Clone the cleanup-worker repository into a Linux environment (e.g., SDC build machine)
• Change to the cleanup-worker folder
• Change to the unused_volumes/deploy/terraform/ folder
  • cd unused_volumes/deploy/terraform/
• Execute the following commands to deploy the infrastructure:
  • terraform init
  • terraform apply -var-file=config/dev.tfvars

Test Plan

• Log on into the AWS Console for the SDC system account, navigate to the EC2 management area
• Navigate to volumes, create a stand-alone volume of 1 Gb and leave it unattached to any instance
• Wait for the EventBridge rulle to run as scheduled and verify that the volume is deleted.
  • Also, examine CloudWatch logs for the dev-cleanup-worker lambda function and verify that there are entries for deleted volume

START RequestId: bb4f6eb0-abc9-4eb9-aa6c-756a1ba19f4f Version: $LATEST
[INFO]	2022-01-06T14:24:42.544Z	bb4f6eb0-abc9-4eb9-aa6c-756a1ba19f4f	cleanup-worker lambda started.
[INFO]	2022-01-06T14:24:45.345Z	bb4f6eb0-abc9-4eb9-aa6c-756a1ba19f4f	Deleting volume vol-0814f00b7c83eff90
END RequestId: bb4f6eb0-abc9-4eb9-aa6c-756a1ba19f4f

Rollback Plan

• From the unused_volumes/deploy/terraform/ folder, execute the following command:
  • terraform destroy -var-file=config/dev.tfvars

V. Design and Architecture

VI. Unit Tests

VII. File Manifest

VIII. Development Setup

Prerequisites

IX. Release History

January 7, 2022. SDC cleanup-worker Release 1.0

X. Contact Information

For any queries you can reach to [email protected]

XI. Contributing

XII. Known Bugs

XIII. Credits and Acknowledgment

Thank you to the Department of Transportation for funding to develop this project.

XIV. CODE.GOV Registration Info

Agency: DOT

Short Description: The Secure Data Commons is an online data warehousing and analysis platform for transportation researchers.

Status: Production

Tags: transportation, connected vehicles, intelligent transportation systems

Labor Hours:

Contact Name: [email protected]

XV. Code Quality Monitor

---

Back to top