In this part of the series, we continue to explain how to build a fully operational DevOps platform on AWS using Terraform.
In Part 1 and Part 2 of the series we focused on building out the AWS infrastructure (networking, routing, etc.) and DevOps Application servers (EC2 instances and configuration, database, Redis, etc.).
In Part 3 I will again build additional Terraform scripts which build out the Fargate cluster to run our application and an Elastic Container Registry (ECR) to store our application images. This will set us up for the final Part 4, where we will deploy our containerized application to AWS Fargate using the Jenkins scrips and our sample application code.
Let’s get started…
This next section of script will address building out an Elastic Container Environment utilizing the AWS Fargate service.
fg_cloudwatch.tf
This script will create a CloudWatch log group where our “Hello World” Fargate tasks will log all activities related to starting, stopping and configuring the containers.
fg_ecr.tf
This script is responsible for creating our Elastic Container Repository (ECR) “myapp-repo”where Jenkins will push our application images and Fargate will pull the image.
fg_elb.tf
When we later deploy copies of our application into containers, we will need a load balancer to manage load balancing to each container. Here we will create an application load balancer and associated listener and target group to place our container apps.
Our load balancer will be listening on port 80 and forwarding traffic to the containers on port 80 (HTTP). We will later reference the target group when building out our Fargate container service to ensure our apps are deployed behind this load balancer.
fg_iam.tf
Later when we build out a Fargate task, we will need to define two (2) roles for the task:
- A task IAM role which is an IAM role that can be used by the containers in a task. Our role needs permissions to ECR to fetch our image, and to CloudWatch to write task logs.
- A task execution role which is an IAM role that can be used by the containers in a task. It requires permissions to pull container images from Amazon ECR and to use the awslogs log driver
Here we define those 2 roles with the required policies and permissions.
fg_sg.tf
Here we define 2 security groups. The first allows TCP traffic on any port from the public subnet CIDR ranges. This will allow our load balancer, which resides in the public subnet, to forward traffic to our target containers.
In a production environment, you would want to tighten this up to specific ports. The second SG will be attached to our load balancer to only allow TCP/80 traffic to ingress into the load balancer.
fg_ecs.tf
In order to run our application in an AWS Container, we will utilize the Fargate Elastic Container Service, which allows us to avoid managing the infrastructure by simply deploying our application containers as Fargate tasks. Here we will create the Fargate cluster which will manage the infrastructure where our application will run.
fg_ecs_tasks.tf
Now that we have a Fargate cluster defined (above), we can define how our application will be deployed within the cluster. So, we create an ECS task definition and ECS service then point the task at our Elastic Container Repository URL where our Docker image resides.
The task definition defines what image to deploy, the resource constraints per task and the role under which to execute the task. The service defines how many copies of the task to run, the load balancer managing traffic to the tasks, which subnets to deploy to and which Fargate cluster to deploy our tasks.
The service continuously monitors the tasks to ensure there are a proper number of tasks running and will spawn additional tasks as needed to maintain the desired count.
templates/hello-world.json.tpl
In order to build out the ECS task templates, a template file needs to be created for each Fargate (ECS) task. The following template needs to be created in the templates sub-folder of your current working directory. Create this file in the “templates” sub-folder.
This template file defines the image to use for our container service, which is located in AWS Elastic Container Repository (ECR). It defines the port mappings on the ECS host and how that maps to the Docker container. We are mapping 2 ports (22) and (80).
The other important configuration is the logging configuration, which directs logs to a specific log group in CloudWatch Logs in our region.
Once all the scripts and templates are created, we can once again run terraform plan and terraform apply to build the remaining platform components and configure the EC2 instances.
Terraform plan
As you can see, we are now using a terraform template in one of the new scripts. Terraform reports an error satisfying plugin requirements. To fix this, you will need to once again run terraform init to install the terraform provider for processing templates and run another plan.
Terraform init
Let’s try terraform plan again with the provider installed. Success…The output is very long. I truncated the output to show only the tail end.
Terraform plan (take 2)
Terraform apply
Run another ‘plan’. You will see all of the new resources that we just built script for. Again, truncated here for brevity.
After several minutes, you will see the output shown above, including the Bastion’s Public IP address and the Load Balancer’s Public IP address. Make note of the DNS Name: hello-world_alb_dns_name. After we deploy our Jenkins job (in the next blog series), we will point our browser to this DNS Name to test the application deployment.
In this part of the series, we focused on building out ECS, ECR, CloudWatch, Load Balancers, Security Groups, and IAM resources required specifically for ECS/Fargate and ECR.
Below is a look at the “masterpiece” we have built to-date. All of the services we will be using as part of this platform has been constructed entirely using Terraform scripts. It is a secure, highly available platform with open source DevOps tools (Jenkins and GitLab).
Come back for the final part…
Come back for Part 4 in the series to learn how to use the DevOps platform to deploy containerized docker applications to Fargate.
In the next blog, I’ll show you how to configure GitLab to manage our code and how to build a Jenkins pipeline to built and deploy the code into ECR and Fargate, where we will have a fully functional Hello World application running behind a publicly accessible load balancer.