Abstract

With the explosion of both demand and cost of GPU resources over the past few years, it's essential to exploit any direct availability of GPU resources. Here, a simple Infrastructure as Code platform is created to facilitate this via cost-effective AWS EC2 instances.

Methodology

The tech stack for this project is as follows: Terraform is used to create AWS G4dn spot instances which use the NVidia T4 GPU. The build is structured in order to minimize required resources by using a GitOps approach with Github Actions. The project creates and manages its own Terraform backend so it won't need to be separately maintained.

At time of writing, a new Nvidia T4 GPU runs about $700, plus the cost of the computing and networking hardware. The g4dn.xlarge spot price is in the neighborhood of $0.25 per hour, depending on AWS region and other factors. Other resources such as a VPC and Security Group are also created, but their cost is minimal compared to the instance itself.

Results

A means of access to GPU resources has been created. Quantities minimized here include infrastructure complexity, human effort, and most importantly, monetary cost.