Terraform and IaC - creating EC2 Instance

I'm revisiting the post I did and creating the same EC2 instance using Terraform instead of going through the AWS Console.

I used the reference from Terraform's Registry to create my HCL script: aws-instance documentation

Here is my main.tf file that I created.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.49.0"
    }
  }

  required_version = ">= 1.2.0"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "spring_boot_webserver" {
  ami           = "ami-0b5eea76982371e91"
  instance_type = "t2.micro"

  tags = {
    Name = "iac-java-webapp"
  }

  key_name = "java-demo-keypair"

  associate_public_ip_address = true
  subnet_id                   = "subnet-01b088baff86159af"
  vpc_security_group_ids = [
    "sg-0a5cddc1e38f7b9b5"
  ]
}

It was pretty cool to go into my AWS Console (after waiting about 30 seconds) and seeing the EC2 Instance that was created by running my Terraform script:

Terraform and IaC

This weekend I started looking into Terraform (specific product) and the general concept of Infrastructure as Code, or IaC.

First step was reading articles and watching tutorials about IaC.  

I've always been on the software side of the IT world.  

Throughtout my career, I've trusted colleagues on other teams to stand up the firewalls, network, co-located physical servers (later VMs) that ran my Java / Angular / .NET applications.  

I've always maintained a great rapport with friends on the ITOps/Infrastructure side of the house, and occasionally participated in discussions about servers, racks, blades, etc.

However, for the most part this was never anything I really dug into that deeply, and as a hands-on Java developer, I never had an opportunity to go to a co-located facility to run cable.

With the trend toward organizations leveraging Cloud solutions like AWS, I'm seeing both a need and a great opportunity as a Software Developer to learn about writing code that sets up infrastructure.

I downloaded Terraform on my Windows home development machine.

Initial thoughts:

• Absolute amazement at how simple the download and setup was.  
• It consisted of an 18.7 MB ZIP file that has a single EXE file
• Created a directory C:\terraform\1.3.7, unzipped to here, and added that to my PATH

AWS Storage: EBS, EFS, S3

Here are some of the main storage options in AWS: 

• EBS (Elastic Block Store)
• EFS (Elastic File System)
• S3 (Simple Storage Service)

I created the following diagram to help visualize how these work:

Instance Store: An EC2 Instance has local store, similar to a physical or virtual machine. 

For a Linux-based EC2 instance (such as Amazon Linux 2), there will be the typical / (root) directory, and standard Linux directories such as /bin, /etc, /opt, /tmp

For a Windows-based EC2 Instance, there is the typical C:\ directory. 

The problem with Instance Store is that it is not persistent or durable.  If the EC2 Instance is terminated, the data that was stored in /opt is gone. 

I/O is fast because there is no network latency between the EC2 Instance and its Instance Store.

EBS: Persistent, but is attached to a single EC2 Instance.  I think of it like plugging in an additional hard drive.  You can't connect it to multiple EC2 Instances because File Systems such as EXT4, NTFS don't handle that.  I/O is fast because there is no network latency between the EC2 Instance and EBS. 

EFS: Persistent, can be shared by multiple EC2 Instances and On-Prem Servers.  I think of this like a mounted drive on a NAS.

S3: Object-based.  Objects are placed into/retrieved from buckets using HTTPS.  This is going to be much less performant than other storage mechanisms, but would be the most globally-accessible. 

ECS: Running Tasks and Task Definitions

Yay, Friday night, personal learning time!  

I'm starting to look into next steps of "how would I deploy new versions of my Spring Boot Application to ECS"?

Initial thoughts would be to do the following:

1. Update Spring Boot App
2. Test locally
3. Commit/push changes to GitHub
4. Create new local Docker Image
5. Push new Docker Image to ECR as a new version in the existing Repo
6. Create a new Revision of Task Definition (currently at Revision 1)
7. Stop current Running Task
8. Run Task using new version of Task Definition

First, taking a look at what I have running now in my ECS Cluster:

Here is the details of that cluster (when I click on the cluster-name link):

Finally, putting it all together from my previous posts, here are the details of the Task when I click on the ARN of the Task.  

And note: public IP Address of the Task is 54.237.231.47

Here is me testing that I can hit my Spring Boot App on this public IP Address:

http://54.237.231.47:8080/hello-world?name=philip

AWS VPC, Subnets, Route Tables, Internet Gateway, and CIDR Blocks

I'm a firm believer that every technology professional has stumbling blocks, something that no matter how hard they try, it's just difficult to "make that knowledge stick".

For some, it's RegEx, for others it could be DNS configuration.  

One of mine is around networking, subnets, and netmasks.  

Unfortunately, understanding these concepts is essential to AWS Virtual Private Cloud (VPC) configuration.  

Here's how I set up my VPC, Subnets, and, Route Tables, and Internet Gateways for my EC2 / ECS studies.

VPC: This is the starting point.  

I created a new VPC with the name web-apps and an IPv4 CIDR block of: 10.0.0.0/24

/24 is the equivalent of a Netmask of 255.255.255.0 

The way I've learned to think of CIDR blocks and Netmasks for IPv4 ... there are 4 Octets, and each Octet is 8 bits.  

/24 means that there are 24 1-bits in the Netmask, taking up 3 of the Octets, hence 255.255.255.0.

The higher the number (up to 32, or all 4 Octets, the more restrictive).

The lower the number (down to 0, or none of the Octets, the less restrictive).

So 0.0.0.0/0 on something like an Ingress Rule means "let the world in".  

Going back to my VPC, CIDR block of: 10.0.0.0/24

32 (all bits in an IPv4 Address) - 24 (from CIDR Block) = 8 (number of bits available for addresses in my VPC).

This means that I will have 2⁸ = 256 IP Addresses availabile to me, with a range of 10.0.0.0 -> 10.0.0.255

Note: have to subtract off 5 IP Addresses, based on Amazon's documentation: "The first four IP addresses and the last IP address in each subnet CIDR block are not available for your use, and they cannot be assigned to a resource, such as an EC2 instance.

Source: https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html

Subnets:

In a Production App, I would create different Subnets for different Availability Zones. Each Subnet must reside in a single Availability Zone.  

Since this is a learning project, I just used a single Subnet with a CIDR block that spanned the entire VPC.  

Route Tables:

I created a new Route Table (leaving the default one alone) and explicitly assoicated it with my web-apps-subnet.  This specifies how traffic should be routed, as the name would imply.

There are 2 routes.  It is saying "any traffic within the range of 10.0.0.0/24 is considered local traffic within the VPC.  All other traffic of 0.0.0.0/0 goes to the Internet Gateway igw-064640bbdb2f376ac

Security Group:

I have 2 inbound rules that allow for traffic on port 22 (so I can SSH into EC2 servers) and on port 8080 (port used by my Spring Boot App).

Network Interface (EC2):

My EC2 instance's Network Interface specifies the VPC, Subnet, and Security Group created in earlier steps.

EC2 and Java Spring Boot

These days, containers and ECS/EKS seem to be all the rage.  However, I wanted to make sure I could still install and run my application on an EC2 Instance.

I decided to create an EC2 Instance t2.micro) using the AMI for Amazon Linux 2.

Step 1: Installing Java 17 (Amazon Corretto)

I logged in using the private key from the Keypair I generated and installed Java 17:

login as: ec2-user

Authenticating with public key "java-demo-keypair"

Last login: Mon Dec 19 02:54:32 2022 from **********

       __|  __|_  )

       _|  (     /   Amazon Linux 2 AMI

      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/

[ec2-user@ip-10-0-0-72 ~]$ sudo yum install java-17-amazon-corretto

after all the dependencies were downloaded, installed, and this completed, I verified:

[ec2-user@ip-10-0-0-72 ~]$ java -version

openjdk version "17.0.5" 2022-10-18 LTS

OpenJDK Runtime Environment Corretto-17.0.5.8.1 (build 17.0.5+8-LTS)

OpenJDK 64-Bit Server VM Corretto-17.0.5.8.1 (build 17.0.5+8-LTS, mixed mode, sharing)

Finally, I added a Bash script to /etc/profile.d to set JAVA_HOME and PATH to include JAVA_HOME. 

[ec2-user@ip-10-0-0-72 ~]$ sudo vi /etc/profile.d/java.sh

export JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64

and sourced this Bash script so it would take effect immediately:

[ec2-user@ip-10-0-0-72 ~]$ source /etc/profile.d/java.sh

Step 2: Create Service to run Spring Boot App

Create java-user (in java-group) with a home directory of /opt/java-apps 

[ec2-user@ip-10-0-0-72 ~]$ sudo groupadd -r java-group

[ec2-user@ip-10-0-0-72 ~]$ sudo useradd -r -m -g java-group -d /opt/java-apps -s /bin/false java-user

                             

I copied the Spring Boot JAR into /opt/java-apps/demo.jar and created the following Systemctl Service file:

[root@ip-10-0-0-72 system]# sudo vi /etc/systemd/system/springboot.service

[Unit]
Description=Java Spring Boot Demo App
After=syslog.target

[Service]
WorkingDirectory=/opt/java-apps
ExecStart=/bin/java -jar demo.jar
SuccessExitStatus=143

User=java-user
Group=java-group

[Install]
WantedBy=multi-user.target

AWS Run Task Definition in Cluster

Finally, putting everything together, we now have an Image in ECR, a Cluster in ECS, and a Task Definition.

From the Task Definition, choose Deploy -> Run task

This brought me to a page with a number of sections about the Task to run. The first 2 were not collapsable, the other 4 were. 

• Environment 
• Deployment Configuration
• Networking (collapsed)
• Task Overrides (collapsed)
• Container Overrides (collapsed)
• Tags - optional (collapsed)

Environment

I changed from FARGATE to FARGATE_SPOT. If I were running an actual Production Application, I would have used FARGATE. However, my understanding is that this is going to run on spare/deprioritized AWS Capacity and cost about 70% less.

Deployment Configuration

Networking