Configuring and running Tasks in Amazon ECS

A step by step guide on how to create task definition in Amazon ECS and how to run tasks in Amazon ECS Cluster

In this article, we will walk you through defining ECS tasks and running them on ECS Cluster. To begin with, let’s understand the basics of ECS tasks.

What is ECS task?

ECS task is responsible to instantiate docker containers in ECS instances or Fargate. Tasks are defined using Task definitions. Each task definition is a collection of parameters like docker image to use, CPU, memory limits, networking mode, etc. When the task is run in the ECS Cluster, it reads Task definitions and accordingly spins up docker containers.

How to configure Amazon ECS Task definition?

• Login to Amazon ECS console.
• In the left navigation panel, click on Task Definitions
• Under the task definitions page, click on the Create new Task Definition button.

Task definitions start with defining the launch type. Choose launch type and click the Next step button.

ECS offers 2 launch types –

1. Fargate
   • Tasks will be launched on infra managed by AWS. (serverless)
   • Tasks will be billed on resources being used and usage duration.
2. EC2
   • TAsks will be launched on ECS instances registered to ECS Cluster
   • No separate bills. You will be paying for ECS instances as per normal EC2 instance bills.

For this exercise, I am using the ECS launch type since I have an ECS cluster running with 2 ECS instances registered to it.

After clicking the Next step button, the task and container definition screen should appear. Lots of things to be defined on this screen. Let’s go one by one –

• Task Definition Name: For identification purpose
• Task Role: If containers being used designed to access some AWS services then you can specify the IAM role here which to be used by containers while accessing AWS services.
• Network Mode: There are 4 modes available here –
  • <default> which is bridge mode
  • Bridge: Traffic forwards between host and container by bridge (kernel-level software)
  • Host: Container network mapped directly to host network
  • awsvpc: Each container assigned with ENI (and hence SG too) Hence each container’s networking can behave like EC2’s ENI.
  • None: No networking for containers. Containers spin up with the loopback IP address assigned.

Task execution IAM role: Needed for pulling container images and sending container logs to Cloudwatch.

Task size: Resource allocation for the task. Limiting resources to be consumed by task container. Should be defined properly when using the FARGATE launch type. For EC2 launch type, this should be calculated depending on ECS instances resources available to the task. Since I am launching a small Apache container on EC2 I left it unfilled.

Container definitions

Container Definitions: Under this section, all container-related settings can be defined. Click on the Add container button and it should take you to the container definitions screen.

There are 2 sections under container definitions.

1. Standard
2. Advanced configuration: Covered in a separate article since it’s a long list of parameters. See ECS container Advanced Configurations here.

Under standard configuration, define –

• Container name: For identification purpose
• Image: Container image. repository-url/image:tag. If you want to use an image from Dockerhub then simply specify image:tag. And for that, you should be having internet access on ECS instances to pull images either via NAT or Internet gateway.
• Private repository authentication: If you are using private container repo like ECR use this option.
• Memory limits: Its memory reserved or allowed for containers during execution.
  1. Hard limit: Max allowed memory for containers to use.
  2. Soft limit: Memory reserved for the container.
• Port mappings: Host port to container port mapping. It’s always advisable to use dynamic host port mapping by defining the host port as zero.

Click Add button to add this image definition in task definition.

• Elastic Inference: Allows you to attach low-cost GPU powered acceleration to tasks. The inference is the process of making predictions using the trained model. It requires processing power. If your container is into such stuff it makes sense to use Elastic Inference which can save you up to 75% cost.
• Constraint: Helps you to decide the placement of containers on ECS instances. Not applicable for the FARGATE launch type. Define conditions to filter and select ECS instances. Once the constraint is applied and instances are selected for task deployment, further placement strategy (Explained in the Run Task section below) will be applied and finally, tasks will be launched on the final instance.
• Service Integration: It’s a facility by AWS to manage your microservices easily. This configures proxy to communicate between microservices for better visibility and HA for services.

• Proxy configuration: Should be auto-configured once you select App Mesh and fill out the required details in that section.
• Log Router Integration: Enables routing of container logs to other AWS services or APN services for storage and analysis. It will spin up the respective container using AWS provided image.
• Volumes: Volumes to be mounted inside containers. It supports 3 types
  1. Bind mount: Mounts file or directory on the host inside the container. More info.
  2. Docker: Managed by docker and creates /var/lib/docker/volumes on the container where volume data resides. Drivers can be selected local or third-party. Can persist on task completion if declared as shared.
  3. EFS: Mount EFS volumes in containers!
• Tags: For identification.

Click Create button to create task definition.

Task definition should be created and it will be versioned as :1. Task definition can be edited using the Create new revision button and it will be versioned as :2 and so on.

How to run Amazon ECS Task?

From same page or going back to Task definitions page, select recently created task definition and click on Actions button.

Run Task screen should appear where you can provide details on how tasks should be run.

• Launch type: Fargate or EC2
• Task Definition: choose which revision to be used
• Cluster: On which cluster task should be run
• Number of tasks: How many containers need to spin up. This count is for HA, FT, or for performance.
• Task Group: Identification purpose.
• VPC and security groups: Available only if the task definition mentions the use of awsvpc networking mode. Defines ENI level networking details for containers.
• Task Placement: Supports 5 templates –
  1. AZ Balanced Spread
     • Deploy containers so that they are evenly spread across AZ
     • Make use of available ECS instances in each AZ
  2. AZ Balanced BinPack
     • Deploy containers by filling one host at a time.
     • Do not start with another host unless the current host capacity is full. Allows maintaining unused hosts.
     • Balanced it across AZ. So start with one host in each AZ and go on filling it till full capacity then move on to the next host in that AZ.
  3. BinPack
     • Same as above except AZ balancing
  4. One Task Per Host
     • Strictly one task per host.
     • If no free hosts available, the task will fail.
     • make sure you have enough available hosts for tasks you are running.
  5. Custom
     • User-defined with a combination of Spread, BinPack or random
     • Configure the order in which it needs to be evaluated.

Under advanced options, IAM roles can be overridden that are defined in Task Definitions.

Task role and Task Execution role, both the IAM role can be overridden with a new one under this section.

Lastly, tagging settings to be done. Enable ECS managed tags as ECS tag tasks with cluster and service name which is pretty much easy for identification later.

Click on Run Task button.

The task will be started and in a couple of minutes, you should be seeing them in RUNNING state. In the above screenshot, the Pending tasks count lists 2 EC2 since I captured the screen by refreshing only the Tasks tab below. I did not refresh whole cluster page 🙂

Now, the task is in a RUNNING state that means containers are instantiated on ECS instances and port 80 of container bound to host port.

Click on any single task ID and it should show task details like below –

Under container details, a host port can be obtained. In this case, the 32768 host port is bounded to port 80 of the container. To verify the functionality of the Apache container, the external link needs to be checked.

Since this cluster is running with ECS instances placed in a private subnet we need to use bastion host to open this external link. Also, since these are private instances you can see the external link is a private IP address, not the public one.

I curled to the external link from bastion host and it worked!

[ec2-user@ip-10-0-0-164 ~]$ curl http://10.0.0.118:32768/
<html><body><h1>It works!</h1></body></html>
[ec2-user@ip-10-0-0-164 ~]$

If it’s not working for you make sure security groups allow the respective traffic between hosts.

And with this, we are completing the creating and running of ECS tasks tutorials. ECS services are used to manage the ECS tasks. We will walk through it in an upcoming article.