In this final part of a series on how to build a fully operational DevOps platform on AWS using Terraform, we show you how to build and deploy a demo application into the AWS Fargate Container Service.
In the first three parts of this series, I demonstrated building out a Virtual Private Cloud (VPC) with the networking, servers, databases, filesystems, load balancers, container registry, container service, security controls and a host of other services required for an application to securely operate in the AWS Cloud.
With Jenkins and GitLab servers in place on the AWS infrastructure, this final part will demonstrate:
- How to configure the 2 application servers
- How to upload application code to our GitLab instance
- How to write a Jenkins pipeline that integrates with GitLab AWS Fargate to download code
- How to build a Docker Image
- How to push the image to Elastic Container Registry (ECR); and
- How to deploy the demo application as a container into the Fargate service behind a publicly accessible application load balancer
Let’s get started…
At the end of part three, we ran a ‘terraform apply’ command to build out the container services in a VPC. As a result, we produced the following terraform output:
Several important variables are printed out at the end, including the Bastion’s Public IP address and the Load Balancer’s Public IP address. We will reference these variables to configure our applications. After we deploy our Jenkins job, we will point our browser to this DNS Name to test the application deployment, so we will need to take note of the DNS Name.
A couple of one-time tasks are required to setup the GitLab and Jenkins applications. The Jenkins root password must be entered into the Jenkins console and the GitLab database must be initialized and populated using the gitlab-rake command. I will begin by walking you through the initial configuration of both services, beginning with Jenkins.
Configure Jenkins
Accessing the Jenkins Web Console
To access the Jenkins web console, enter this DNS Name (printed above) into your favorite browser and append :8080 to the end of the URL. From there, you set the root password, install desired plugins, customize the configuration and begin to build your pipelines.
Example: http://private-apps-alb-1114129372.us-east-2.elb.amazonaws.com:8080/
Jenkins one-time setup tasks
When you first install the Jenkins Master server, you will be required to locate the auto-generated root password which is stored on the server in /var/lib/jenkins/secrets/initialAdminPassword.
Log into the Bastion host using SSH. From there SSH into the private IP address (obtained from the EC2 console) of the Jenkins Master server and display the contents of the password file. Use this password to log into the Jenkins Master web console by pasting the password into the Jenkins console at the ‘Administrator password’ prompt shown above.
Example output
Once the password is entered, you will be led through a few initial configuration screens (shown below). When you see the main user interface, the HA configuration will be complete. By HA, we mean that all data is stored externally on the EFS filesystems, allowing you to complete destroy the EC2 instance without losing data.
The autoscaling group will automatically deploy a new replacement EC2 instance and attach it to the shared data store allowing you to resume where you left off.
Jenkins Initial Configuration Screens – One-Time Task
Jenkins is now ready!
Configure GitLab
Accessing the GitLab Web Console
When the terraform script completed successfully, it printed out the Public DNS Name of the Application Load Balancer. Enter this DNS Name into your favorite browser to access the GitLab web console. Once in the console, you can create users, groups and projects and test remote access from your workstation.
Example: http://private-apps-alb-1114129372.us-east-2.elb.amazonaws.com/
Until you complete the initial configuration tasks below, you will see this screen.
Gitlab one-time setup tasks
Log into the Bastion host from your workstation using the previously created SSH key. From the Bastion host, you will SSH into the first GitLab servers to initialize the database with the gitlab-rake command.
You will see a long list of commands running, which is populating the external PostgreSQL database. A snip of the output is below.
After the gitlab-rake is complete, you can enter the public DNS name again into your browser and GitLab will be ready for you to enter your email address, change the root password and login with the root account. Sample screen flows follow.
Next, sign in using Username “root” and the password you entered above.
The Home screen will be presented, and you can create a new user, log in as the new user and create a new project to house your application code.
The application code you upload will be used to later build a docker image which will be run in as a container in the AWS serverless container service known as Fargate.
At this point, you have a working GitLab instance with all configuration data stored on shared EFS Storage. It is time to scale up the autoscaling group for GitLab from 1 to 2. This will spin up another GitLab instance behind the load balancer and provide for a highly available architecture.
To do this, simply update the variables.tf file as indicated below and run terraform apply. Shortly after, a 2nd EC2 instance will be deployed for the second GitLab instance. Verify the new instance in the AWS console.
Congratulations! The DevOps platform is setup and ready. Because all of the servers, databases and cache servers are highly available, you can safely terminate any component and within a few minutes, a replacement will be spun up and back to normal operation with no user interaction.
GitLab Define a Project
Create a new user and project to upload your PHP application code.
Define a new project named “hello-world” in GitLab to house your application code. Click the “+” sign, then “New project.”
Enter project name “hello-world.” Set Visibility to “Internal” and select “Create project.”
The new project will be created and a link to the project will be presented for SSH and HTTP access. The HTTP link will be utilized for the demo purposes to upload code and integrate with Jenkins.
Clone Hello-World PHP application from the provided demo GitHub repo. This code will then be pushed up to your new GitLab repository created above.
Create a new working directory for the local repository and run the following ‘git clone’ command. Verify the code is downloaded.
The sample application code is now cloned from the public GitHub repository to your local repository and can be uploaded to your GitLab repository.
Change your remote repository to the GitLab repository, changing the repo URL to match the repo name you created in GitLab.
Note: Change the highlighted repository name to the name of the GitLab repository you created above and ensure you are using HTTP protocol. Next, verify the remote repository and finally push the code to GitHub.
Verify the GitLab project was updated to include the sample PHP application files cloned from GitLab by accessing the URL of your GitLab instance and refreshing the page. (e.g. http://private-apps-alb-869593036.us-east-2.elb.amazonaws.com/rich/hello-temp)
Jenkins Plugins
A few additional Jenkins plugins need to be installed in order to build our pipeline. Add the plugins listed below by selecting “Manage Jenkins” -> “Manage Plugins” -> “Available” tab.
From here you can search for each plugin, select the checkbox next to each plugin and click “Install without restart”.
| CloudBees Credentials Plugin | GitLab Plugin | Amazon ECR plugin |
| CloudBees Docker Build and Publish plugin | Gitlab Hook Plugin | Docker plugin |
| Email Extension Plugin | docker-build-step |
The following extensions should be installed now:
Select the checkbox “Restart Jenkins when installation is complete and no jobs are running”, to allow Jenkins to reboot and the extensions to be activated.
Jenkins Credentials
Add credentials which will be used to access external resources, such as AWS cli and AWS Elastic Container Registry.
From the Jenkins Home screen select Credentials -> System -> Global Credentials pull down -> Add credentials. Add the listed credentials below.
Below is a list of credentials you will need to create.
Gitlab User
GitLab API Token
Used for GitLab over HTTP. To first create the token in GitLab, select the pull-down on the far-right, top and select Settings. From the Settings screen, choose “Access Tokens”. Git the token a name, expiration date and select api, then click “Create personal access token”.
A token will be created, which you can copy into your Jenkins credentials.
Once the token is created, go back to Jenkins and enter the API token and give it an identifiable name.
AWS Elastic Container Registry Access Key:
Used to push artifacts to ECR.
Jenkins Global Configuration
Prior to creating your pipeline, you must configure Jenkins global settings as shown below.
Manage Jenkins -> Configure System
Environment Variables
Find “Environment variables, select the checkbox and then click the “Add” button.
Create these 2 environment variables, substituting in the AWS Access Key ID and Secret Access Key provided for the ecr-remote-pusher user account created earlier.
If the keys were not saved, you can generate a new access key for that user and enter the keys here. These keys will be used when running the Jenkins task to authenticate to AWS, allowing the task to push artifacts to the ECR repository. Personal keys are redacted.
GitLab Connection
This is the URL and credentials used to access the GitLab instance. Enter your connection name, URL and the API token created earlier, in the credential store, to access GitLab via an API token.
Jenkins URL
This is the Jenkins URL used to send out notification from Jenkins, such as when creating a new user or resetting a password. This URL should already be setup for you.
Build a Jenkins Pipeline
Use the information below to create a pipeline in Jenkins, which will:
- Download your PHP code from your GitLab repository
- Build a docker image from the code
- Upload the docker image to your AWS ECR Repository
- Create or update your Elastic Container Service (ECS) with a load balanced set of containers/tasks
Jenkins – New Item – Freestyle Project
Enter a name for your project: “Hello-World” and select OK. A new screen will be presented. Look for the settings listed below and fill out your project details as shown below.
General -> GitLab Connection
<Your GitLab Repository URL>
This should already be pre-populated.
Source Code Management -> Git -> Repositories
Select your own repository URL and admin user login credentials.
Build Environment -> Delete workspace before build starts.
Build – Execute Shell #1
Create a new build step by clicking “Add build step->Execute Shell” in the Build section. This step will login and authenticate our Jenkins pipeline to our Elastic Container Repository, so we can later push our Docker image into the repository.
Replace highlighted entries with values from your own environment. The ECR_REPO can be found in the output of the terraform apply command labelled “hello-world-repo”.
Docker Build and Publish
Create a new build step by clicking “Add build step->Docker Build and Publish” in the Build section.
This step will build our project and publish the resultant tagged image to the docker registry (repo).
Enter your own docker ECR repository, URL and registry credentials for the correct region.
Build – Execute Shell #2
Create a new build step by clicking “Add build step->Execute Shell” in the Build section.
In this custom script, we are taking the image we built in the prior step and registering it as a task definition within our Fargate Cluster. Once the task is defined with the tagged image from ECS, we build or update the Fargate service with the new task definition. Here we are specifying that we desire 2 instances of the task to be running within this service.
Replace highlighted entries with values from your own environment.
Click “Save” to save your Jenkins task definition.
Execute Jenkins Build Job
Now that you have a fully configured Jenkins Freestyle project, you can test the job by clicking on “Build Now”. Monitor the Jenkins job by clicking on the job number under the “Build History” on your project page. From there, you will click on “Console Output” to see the details. This is where any errors will be reported.
The build job will download your docker configuration from GitLab, build the docker image, upload the new docker image to the ECR repository, deploy your new docker image as a Fargate task in AWS and finally build or update the Fargate service to run and monitor the tasks.
It will take a few minutes to deploy the application. You can monitor the task deployment in the AWS Console under ECS-> Clusters -> fargate cluster -> hello-world_alb_dns_name.
See the Desired tasks and Running tasks in the Services tab on this page. The Running tasks should be at least 2, but could be as many as 4 if this is not the first time you deployed the Fargate tasks. The reason is that when deploying a new version, 2 new tasks will be deployed and verified as running, prior to deleting the old tasks. In this case you would likely see 4 tasks running for a few minutes while the tasks are updated to the new version.
When the tasks are up and running in ECS, type the DNS Name of the Elastic Load Balancer servicing the ECS tasks into your web browser and verify the application responds. The DNS Name you need can be copied from the output of the terraform apply command as: “hello-world_alb_dns_name“.
Sample application output
You should see the following output. Because we are running multiple tasks in our cluster, when you refresh your browser, the hostname IP should update to the IP address of another fargate task.
If you want to test a small change to the application and redeploy the change, you can edit your local copy of the application code that you cloned and check it into your GitLab repository, then run the Jenkins task again. It may take up to 5 minutes for the deployment to update your public web site.
A simple change can be made by changing your index.php file with a new message. The index.php file can be found in your application repository under the www folder. Make the change, run “github add .”, “github commit”, and “git push”. Then, in Jenkins, click “Build Now”. In a few minutes, you change should be visible on the public URL.