Description
You were recently hired as a Junior Devops Engineer at the startup FlyEasy. They’re building a pretty cool nodejs application with docker. Now they’re thinking about automating the process as much as possible from the integration to the deployment phase. Your challenge is to build a new pipeline with GitlabCI.
The docker image will be stored in a private repository ECR and the container executed on a ECS Cluster.
Learning Objectives
- Create an AWS Account
- Create ECS IAM Roles
- Create a private repository on ECR
- Create a new ECS Cluster
- Create a task definition
- Create a service
- Add environment variables on Gitlab
- Update the task-definition.json
- Update the gitlab-ci.yml
- Test the build pipeline
- Test your access to the application
Step 1 : Create an AWS Account
We need to create an AWS account for Gitlab with the required permissions to :
– Push the docker image to the private ECR repository
– Register the task definition and update the service
1. Open the IAM console, and select Add user
2.
Username : gitlabci-user
then select Programmatic access , click Next permissions
3. Add these 2 policies :
-AmazonEC2ContainerRegistryFullAccess
-AmazonEC2ContainerServiceFullAccess
and click on “Next“
4. Check that you have the 2 policies attached, then select “Create user”
5. Save the Access key ID and Secret access key, we will need later to set up Gitlab.
Step 2 : Create ECS IAM Roles
First things first, let’s create 2 IAM roles, these roles will be used later.
EcsInstanceRole
EcsTaskExecutionRole
1. Open the IAM console, and select Create role
2. Select AWS service > EC2 > Next Permissions
3. Select AmazonEC2ContainerServiceforEC2Role and click on “Next“
4. Name your role “EcsInstanceRole” , then click on “Create role”
Repeat the same steps (1->4) to create the second role EcsTaskExecutionRole
At step 3, select AmazonECSTaskExecutionRolePolicy
Step 3 : Create a private repository on ECR
1. Open the console Elastic Container Registry > Create repository
2.
Repository name : nodejs-demo-app-ecs
then, Create repository
Step 4 : Create a new ECS Cluster
1. Open the Elastic Container Service, then select “Create Cluster”
2. Select EC2 Linux + Networking
3.
Cluster name: “Preproduction”
Number of instances : 1
Key pair : “Select your key pair”
4. Select an existing VPC or create a new one
5. Select the role we’ve created earlier “EcsInstanceRole” , then click on “Next”
Step 5 : Create a task definition
1. Select Create new Task Definition
2. Select EC2
3.
Task Definition Name : nodejs-demo-app
4. Select the 2nd role we’ve created earlier “EcsTaskExecutionRole” , then click on “Add container”
5. In the container section:
Container name: nodejs-demo-app-ecs
Image: ikubelabs/nodejs-demo-app-ecs:1.0
Memory Limits: 200
Port mappings
Host port : 80
Container port : 8080
6. Scroll down and add this environment variable
Key: ENVAPP
Value: STAGING
Then click on “Add”
7. Set the task size to 500
8. Scroll down and select “Create” ,
Step 6 : Create a service
1. Select “Actions > Create Service“
2.
Launch Type : EC2
Cluster: Preproduction
Service name: node-demo-app-service
Number of tasks: 1
Minimum healthy percent : 0
3. Leave the options by defaulit, then “Next step”
4. Select “None” then “Next step”
5. Leave the default option, then “Next step”
6. Select “Create Service“
7. Back to the console, select “Tasks” tab, see the task in execution, it should be in “PENDING” state, wait until it shows “RUNNING”
Then click on the Task id to see more details
8. Scroll down and expand the container for more details
9. Open the URL to access the application
10. This webpage is displayed.
Step 7 : Add environment variables on Gitlab
1. On your Gitlab project repository , select Settings > CI/CD > Variables
2. Add these variables :
– AWS_ACCESS_KEY_ID (see – Step 1)
– AWS_SECRET_ACCESS_KEY (see – Step 1)
– AWS_DEFAULT_REGION (default region where the application is deployed)
Step 8 : Update the task-definition.json
1. On your Gitlab project repository , Update the file task-definition.json
2.
Replace image value with your Elasctic Container Registry URI you created earlier (Step 3)
Make sure to keep :build
3.
Replace the taskRoleArn value with the ecs task role you created earlier (Step 2)
Step 9 : Update the gitlab-ci.yml
1. On your Gitlab project repository , Update the file gitlab-ci.yml
image: docker:latest
variables:
AWS_REGION: eu-central-1
ECR_REGISTRY: 618925041282.dkr.ecr.eu-central-1.amazonaws.com
ECR_REPO: 618925041282.dkr.ecr.eu-central-1.amazonaws.com/nodejs-demo-app-ecs
IMAGE_TAG: $CI_COMMIT_SHORT_SHA
IMAGE_NAME: nodejs-demo-app-ecs
CI_AWS_ECS_CLUSTER: Preproduction
CI_AWS_ECS_SERVICE: nodejs-demo-app-service
CI_AWS_ECS_TASK_DEFINITION_FILE: task-definition.json
AWS_REGION:
the region where you set your ECR repository
ECR_REGISTRY:
the ECR default registry
ECR_REPO:
the ECR repository uri
IMAGE_TAG:
the commit id in short format
IMAGE_NAME:
the name of your docker image
CI_AWS_ECS_CLUSTER:
the ECS Cluster name
CI_AWS_ECS_SERVICE:
the service name
CI_AWS_ECS_TASK_DEFINITION_FILE:
the task definition file
In the first section we declare the 2 stages of our pipeline : Build and Deploy
We declare services (docker:dind), as we need to access the service image of Docker during the build.
Before the execution of the script, we need to install the packages (curl, jq, py3-pip and sed)
During the Build stage :
1. We log into the ecr erpository
2. We use the docker command to build the image
3. We tag the image (note that the image will be tagged with the commit short format id)
4. We push the tagged image to the ECR repo
During the Deploy stage :
1. We create a new task definition for the build
– We replace the chain of characters “build” by the “$IMAGE_TAG” in the task-definition.json file
– We register a new task definition
2. We update the service with the new task definition
– We register the new task definition, we parse the response (json ) to retrieve the task definition arn and we assign the result to the variable REV
– We update the service
2. Once completed it should this page, with the commit id and the successfull Build and Deploy stages.
3. Note that the commit id matches with the docker image tag in your ECR repo;
This might be useful if there is any issue with the deployment.
Step 10 : Test the application access
1. Back to the console, select “Tasks” tab, see the task in execution, it should be in “PENDING” state, wait until it shows “RUNNING”
Then click on the Task id to see more details
2. Scroll down and expand the container for more details
3. Open the URL to access the application
