Yearly Archives: 2020

Creating first AWS Lambda function

A quick article explaining all available options for creating a new AWS Lambda function and how to run it.

AWS Lambda is a compute service offered by AWS. It is treated as a serverless compute since the end-user did not have to worry about providing and maintaining the back-end infrastructure. Without further delay, let’s move ahead with creating the first AWS Lambda function.

Login to AWS Lambda console.
Click on the Create function button.
You should see wizard as below –

There are 4 ways you can choose to create AWS Lambda function –

Author from scratch: Select the programming language of your choice and start writing the function code
Use a blueprint: Use readymade function pre-written for you by AWS.
Container image: Use a docker container image to deploy function
Browse serverless app repository: Use complete application pre-written in Lambda.

Basic information
- Function name: For identification purpose
- Runtime: Function’s coding lanuage. Select a supported language from the dropdown.
- Permissions: Arrange the Lambda function IAM role to access AWS resources on your behalf.

You can either use an existing role or let the wizard create a default permission role. If your code requires special permissions, it’s recommended to create a new role using policy templates to ensure that function has sufficient privileges to access AWS resources.

Advanced settings
- Code signing: To ensure the integrity of the code and its source.
- Network: If function expected to have network access, then select VPC here and hence in turns select subnet, security group which will be applicable to function when it runs.

Click the Create function button.

AWS Lambda function should be created. Click on the Latest alias to edit your code and test it.

In latest alias, there are 5 tabs you can navigate.

Overview: Overall details.
Code: Edit code and code-related configurations.
Test: Invoke test event for the Lambda function
Monitor: Monitoring aspects in context to Cloudwatch
Latest configuration: Various Lambda function related configurations.

Lets have a look at all these tabs one by one.

Overview
- General configuration: Shows description, last modified, and Function ARN.
- Function visualization: Visual representation of a function.
Code
- Code Source: In browser code editor. You can upload the code in a zip file or reference it from the S3 location.
- Runtime settings: Edit the code language.
- Code signing details: Edit code signing for code integrity.
- Layers: Edit or add layers to function.
Test
- Test event: Create, format, or invoke test event.
Monitor
- View CloudWatch metrics or CloudWatch logs insights.
Latest configuration
- This is the biggest section of all. It has 5 subsections
  1. General
  2. Environment
  3. Triggers
  4. Permissions
  5. Destinations

Lets go through each section one by one:

General
- Basic settings: Edit below configurations
  - Description
  - Memory (MB): Memory allocated to function. CPU will be allocated in proportion to memory.
  - Timeout: Max execution time. Limit: 15 mins.
  - Execution role: IAM role being used by the function.
- Asynchronous invocations:
  - Maximum age of event: To keep the unprocessed event in the queue.
  - Retry attempt: Maximum number of retries after the function returns an error.
  - Dead letter queue: SQS queue for sending unprocessed events for asynchronous invocations.
- Monitoring tools
  - Amazon CloudWatch: By default enabled for AWS Lambda
  - AWS X-ray: For analysis and debug purposes.
  - CloudWatch Lambda insights: Enhanced monitoring.
- VPC
  - Edit VPC, subnet, security group for Lambda function if function required networking.
- State machines
  - Step functions to orchestrate Lambda function.
- Database proxy
  - Manage database connections if the function requires DB connectivity.
- Filesystem
  - Make EFS available to Function.
Environment
- Environment variables: Set of key-value pairs to use as environment variables in function during execution.
Triggers
- Add triggers to invoke the lambda function.
Permissions
- IAM permissions via role or resource-based policies
Destinations
- Configure destination for invocation records once function executes.

Since its a test function, I move ahead with all the defaults. Once everything is configured you are ready to test the fuction.

Click on the Test tab and then click the Invoke button to invoke the Lambda function test event.

Test invokation results presented on same screen. The important details to look for are –

Returned results by function
Duration: Actual execution time
Billed Duration: Billed execution time. Rounded to nearest 1ms.
Max memory used

The log output at the end can also be viewed in CloudWatch log group.

The bill amount is calculated using the amount of memory allocated and billed duration.

So that’s how you configure and execute simple basic Lambda functions for understanding the foundation of AWS Lambda!

Replication in Amazon S3

Leave a reply

An article to overview SRR and CRR in Amazon S3. Why, when, and how to use it.

Amazon S3 is a Simple Storage Service offered by AWS. It’s object storage with virtually unlimited in size. S3 also offered the feature of replication to replicate objects from one bucket to another bucket. The bucket is a placeholder for arranging objects in S3. It can be visualized as a root folder in the file structure.

What is replication in Amazon S3?

Replication is a process of copying objects and their metadata from a source bucket to a destination bucket in an asynchronous way in the same or different region.

Why S3 replication is needed?

There are couple of use cases where S3 replication can prove helpful –

Compliance: Some customers need to keep a copy of data in a particular territory for compliance purposes. In such cases, S3 replication can be set to copy data into the destined region.
Data copy under different regions and accounts: S3 data replication enables users to copy data in different regions and under different accounts.
Latency requirement: Keeping a copy in a region close to consumers helps to achieve low latency in accessing data stored in S3.
Objects aggregation: Data from different buckets can be aggregated into a single bucket using replication.
Storage class requirement: S3 replication can directly put objects in Glacier 9diff storage class) in the destination bucket.

Types of S3 replications

Same-Region Replication (SRR)
- Source and destination buckets lie within the same region.
- Used in case data compliance requires the same copy of data to be maintained in different accounts.
- Log aggregation can be achieved using SRR by setting up SRR from different log buckets to the same destination bucket.
Cross-Region Replication (CRR)
- Source and destination bucket lies in different regions.
- Used in case data compliance requires the same copy of data to be maintained in different regions.
- Useful in latency requirements.

Requirements and limitations of Amazon S3 replication

Versioning must be enabled at source and destination buckets.
Existing files are not replicated. Only new uploaded objects (after enabling replication) are replicated.
Delete markers created by lifecycle rules are not replicated.
You can’t replicate objects which are encrypted using SSE-C.
Can set up across accounts.
It’s one-way replication, not bidirectional. It means source bucket to destination and not vice versa.
Lifecycle policy events are not replicated.
Amazon S3 should have access to read/write on both buckets by assuming your identity.
The destination bucket can not be configured with requestor pays mode.

How to configure S3 replication?

Let’s dive into the practical aspects of it. We will be configuring replication between 2 buckets hosted in different regions under the same account for this exercise. Read here how to create an S3 bucket. Below are 2 buckets in consideration –

Log in to the S3 console.
Click on Buckets in the left navigation panel.
On the buckets page, click on the bucket name you want to create replication (source bucket).

Creating replication rule for source bucket

Click on the Management tab under bucket details.
And scroll down to the Replication rules section.
Click on the Create replication rule button.

Make sure you have versioning enabled on both source and destination buckets, or else you will see a warning like the one below –

In replication rule configuration wizard –

Replication rule configuration
- Replication rule name: Name
- Status: Enable/disable the rule on creation
- Priority: In the case of multiple replication rules, the object follows priority.

Source bucket
- Source bucket name: Prepopulated.
- Source region: Region of source bucket. Prepopulated.
- Choose a rule scope: Replication can be limited to selected objects by using a prefix filter. Or apply it for all objects in the bucket.
Destination
- Bucket name: Choose bucket from the same account or a different account here as a destination.
- Destination Region: Region of destination bucket.

IAM role
- Select the IAM role to be used by S3 for performing the copy operation. Make sure you have carefully added proper permissions to it. I choose to create new for this exercise.
Encryption
- If data is encrypted using AWS KMS, it can be replicated.
Destination storage class
- Change the storage class of replicated objects or keep it the same as the source.

Additional replication options
- RTC: Premium service with guaranteed 15 mins ETA for replication of new objects
- Replication metrics and notifications: Cloudwatch for replication!
- Delete marker replication: Enable to replicate S3 delete operation created delete markers. Remember lifecycle rule created delete markers can not be replicated.

Click the Save button. Replication rule should be created and you should be presented with rule details like below –

You can even see the IAM role created by AWS while creating a rule since we opt to create one.

Now, it’s time for the test. I had one file already in the source bucket, and it’s not replicated after creating the rule since it’s only applied to new objects uploaded to the source bucket after rule creation.

File1.text was existing before rule creation and File2.txt uploaded after rule creation. So the destination bucket should be having File2.txt replicated from the source bucket.

And its there! So replication rule worked perfectly!

How to check if object is replicated?

To verify if the object is replicated or not. Click on the object and check its replication status.

For replicated object replication status should show it as REPLICA. Whereas, for source object i.e., who uploaded to source object and then got replicated status is COMPLETED

And it would be blank for non-replicated objects. Since File1.txt was uploaded before replication rule creation, the rule was not applied to it.

And hence we see its replication status as - i.e., no replication rules applied to this object.

Configuring Visual Studio Code for Terraform to work with AWS

Leave a reply

A quick post for working on AWS using Visual Studio Code and terraform.

Install Terraform

Download the latest terraform software and install it on your machine. For windows, keep terraform.exe downloaded from this page in C:\Windows\System32 and that should be enough.

Install Visual Studio Code

Download Microsoft Visual Studio Code software from their website. Install and launch Visual Studio Code. It supports many code languages. But many of them are not supported on install. One needs to add extensions from the extension marketplace to make it available in VSC.

Install Terraform Visual Studio Code extension

The official Visual Studio Code extension for Terraform is HashiCorp Terraform. You can visit this link and click on the Install button on the webpage, which in turn opens up VSC and install the extension or follow the below procedure.

Launch Visual Studio Code
Click on the Extensions shortcut on the left navigation panel.
Search terraform in extensions marketplace
Select HashiCorp Terraform from search results
Click the Install button.

Once installed it will be available to use.

Login to AWS account for terraform

It’s programmatic access you will be configuring to AWS. So for that, you should create Access keys through the IAM AWS console. Once you are ready with a valid AWS account with programmatic access enabled and downloaded its access keys, head back to VSC.

Create a new file main.tf with the below code for login. Make sure you use your own key values in there :

provider "aws" {
    access_key = "AKIAVGESLBG6N4NLDQ7H"
    secret_key = "jyiL7OTx4glshojC6VVoVxMe94kBtsc4XiEaD+8p"
    region = "us-east-1"
}

and in terminal below, run terraform init command.

Although hard coding credentials like this is not recommended, you can define them as environment variables or other methods supported by terraform.

This should initialize Terraform to use with AWS. It will download the AWS plugin for Terraform. Sample output –

PS D:\Testing> terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v3.17.0...
- Installed hashicorp/aws v3.17.0 (signed by HashiCorp)

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.

* hashicorp/aws: version = "~> 3.17.0"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

The next step is scanning existing AWS infra to give an account to make the terraform aware of it. Execute the terraform plan command. This command now will use the access keys defined in the file, login to AWS, and check the existing infra.

PS D:\Testing> terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Creating sample user in AWS using terraform from Visual Studio Code

Now to test a small command, let’s define a user in the file. If that user existing in your AWS account, you should see the same output as above. If not, terraform should show that a new user needs to be created to match the file’s requirements.

Ensure the user you are using in terraform has permissions to create resources in AWS you are planning to.

Add below code in file, save and run terrafrom plan again.

resource "aws_iam_user" "user1" {
    name = "user1"
}

PS D:\Testing> terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_iam_user.user1 will be created
  + resource "aws_iam_user" "user1" {
      + arn           = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + name          = "user1"
      + path          = "/"
      + unique_id     = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

As you can see, since testuser is not existing in AWS infra, it’s prompting to create one! To apply the changes, run the terraform apply command, and it will execute the operation.

Finally, to verify the user created run terrafrom show command.

Finally, to delete all resources being created by terraform, you can run terrafrom destroy command.

PS D:\Testing> terraform destroy
aws_iam_user.user1: Refreshing state... [id=user1]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # aws_iam_user.user1 will be destroyed
  - resource "aws_iam_user" "user1" {
      - arn           = "arn:aws:iam::xxxxxxxxxxxx:user/user1" -> null
      - force_destroy = false -> null
      - id            = "user1" -> null
      - name          = "user1" -> null
      - path          = "/" -> null
      - tags          = {} -> null
      - unique_id     = "AIDAVGESLBG6PGBXZBT35" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:
aws_iam_user.user1: Destroying... [id=user1]
aws_iam_user.user1: Destruction complete after 2s

Destroy complete! Resources: 1 destroyed.

It should ask confirmation and then proceed with deleting resources.

How to create a user with programmatic access in AWS

Leave a reply

A quick post with step by step procedure to create a new IAM user in AWS with programmatic access. Also, learn how to re-generate access keys.

Login to AWS IAM console
On the left navigation panel, click on the Users link.
On the right-hand side Users page click on the Add user button.
Add user screen should come up –

Fill in details –

User name: User id
Access type:
- Programmatic access: No access to AWS console. Use of keys for authentication. Console access can be enabled later.
- AWS management console access: Access to AWS console Use of userid/password authentication. Programmatic access can be enabled later.
Click on the Next: Permissions button.

Set permissions
- Add user to group
  - An efficient way to manage user permissions by making them members of the group. Apply policies/permissions to the group!
- Copy permission from an existing user
  - If you want to have user same permission as another existing user in the same account.
- Attach existing policies directly
  - Attach permission policy to user either AWS managed policy from the list given or by writing your own policy by clicking Create policy button.

For this exercise, I will choose an easy way by marking AWS managed Administrator access policy to users.

Set permission boundary
- Define the maximum permissions this user can have. User’s permissions can not breach the boundary defined here. Again you can create your own or use AWS managed policy here.

Click on Next: Tags button

Add user tags for identification purpose and click on Next: Review button.

Review all the configurations and click on Create user button.

User should be created and you should be seeing above screen.

From this screen, you can copy or download the keys required for AWS programmatic access.

Click on the Download .csv button to download the key pair. Or click on Show link under the Secret access key and then copy/save both the Access key ID and Secret access key. Once you navigate away from this page, you will not be able to retrieve the secret access key from anywhere. You need to recreate the pair for this user then.

You have successfully created an IAM user with programmatic access to AWS. You have access keys with you!

How to re-create IAM secret access keys?

As I mentioned above, if you lose the secret access key there is no way to retrieve it unless you saved it somewhere. But in such unfortunate incidents, you can re-create them using an AWS root account.

Login to AWS IAM console using the root account
On the left navigation panel, click on the Users link.
On the right-hand side users page, click on the user name whose keys needs to be regenerated
On users, summary page click on the Security credentials tab

It’s good practice to keep only one key pair active at a time so click on the Make inactive link for the existing key. You can keep it unless there is such a requirement. Click the Deactivate button on the pop-up. You can even delete this key 9if it does not have any dependency) by clicking a small x next to it.

Click on Create access key button to generate new key pair.

New key pair will be generated and you will have a chance to download/copy save the secret access key again!

How to create an Amazon SQS queue and test with Amazon SNS?

Leave a reply

The step by step procedure to create an Amazon SQS queue and later test it with Amazon SNS.

What is an Amazon SQS?

Amazon SQS (Simple Queue Service) is the very first AWS service. It’s a message queueing service that helps with decoupling the architecture. It’s widely used in microservices designs. It’s a poll based service. So basically, it receives messages from some source, maintains messages in the queue for consumers to poll for it, consume it, and delete it after processing.

Using SQS, one need not keep consumer compute/infrastructure up and running all the time. SQS takes care of buffering and queueing messages, and consumers can poll them once a while, processes them, and exists/shuts them till the next poll. It also holds messages if consumers are not available or busy and scales accordingly, so you don’t lose any messages.

Amazon SQS offers two types on queue and their differences are as below –

Standard Queue	FIFO Queue
Default SQS queue	First in first out queue
Messages order is not preserved	Preserved messages order which is first in first out
message can be processed twice	Each messages processed exactly once
It can scale indefinitely	It can processes 3000 messages per seconds only
Best fit for scaling and high throughput requirement	Best fit where order is important and duplicates cant be tolerated
No naming restrictions	Queue name must ends with .fifo suffix.

Lets dive into creating our first SQS queue.

How to create Amazon SQS queue?

Log in to an Amazon SQS dashboard
Click on the Create queue button on the introduction page.
Create a queue wizard should open up –

Details
- Choose the type of queue
  - Standard
  - FIFO
- Name: Enter the queue name. For the FIFO queue, the name should have a .fifo suffix.

Configuration
- Visibility timeout: Time for which message won’t be visible to other consumers when picked up by consumer for processing. Range 0 secs to 12 hours
- Messages retention period: Duration for which messages live in the queue if not deleted by any consumers. Range 1 min to 14 days
- Delivery delay: Time for which message won’t be visible to consumers to pick once the queue receives it. Range 0 sec to 15 min. 0 means the message will be available immediately for processing once received by the queue.
- Maximum message size: Message size. Range 1KB to 256KB
- Receive message wait time: Max time for polling to wait for messages to be available. 0 means short polling (increases empty responses), and 20 means long polling—range 0 to 20 secs.

Access policy
- Basic
  - Simple bullet selections for access to send and receive messages to/from the queue. JSON will be generated accordingly.
- Advanced
  - Use your own JSON to define policies

Three optional configurations for SQS queue.

Encryption: Server-side encryption offered by AWS.
Dead-letter queue: Home for undeliverable messages.
Tags: For identification purposes.

Click on Create queue button.

SQS should create a queue and present you with queue details.

You can click on the Send and receive messages button to manually send, receive, and delete messages in this queue.

Enter the message body and click on the Send message button. The message will be sent to the SQS queue.

If you scroll down on the same screen, you can click the Poll for messages button and receive the messages from the SQS queue you sent in the earlier step.

Click on message ID to view message details. Under the message body tab, you can see the message body.

Once messages are processed (i.e., read in our case), you can delete messages from the queue by choosing a message and clicking the Delete button.

Sending messages to SQS queue from SNS

We walked through sending and receiving messages manually to and from the SQS queue. Now we will do it using SNS. SNS is a Simple notification Service offered by AWS. SNS topics are the channels where you publish messages for recipients. It’s a push-based service.

I already created an SNS topic named sns-topic-001 to which the SQS queue will be subscribed.

On an SQS queue details page click on the Subscribe to Amazon SNS topic button.

You should be able to select the existing SNS topic from the drop-down on the next page.

Click Save button.

Login to the SNS dashboard
Click on the Topic you created
In the Topic details screen, click on the Publish message button.
Make sure you see the SQS queue under Subscriptions.

Fill in message details like Subject and body and click the Publish message button.

Now the message is published to the SNS topic. We have an SQS queue that is subscribed to SNS Topic. That means this published message should appear in the SQS queue.

Lets go back to SQS queue and check for it.

You should see 1 message available to receive in SQS. And when you click the Poll for messages button, you will see that message in the messages details section below.

If you check message body you will see all SNS formatted message details.

In this way, you can check the SQS queue’s functionality by sending and receiving messages manually or using SNS Topic. You can even receive messages automatically using the Lambda function and then publish it to another SNS Topic, which can mail it to you for testing! We might cover this in another article. But this is it for now!

Know different Load Balancers in AWS

Leave a reply

A quick post about different types of load balancers in AWS and the difference between them.

AWS offers a load balancing feature under EC2 compute service. It offers basically 4 types of load balancers :

Application Load Balancer
Network Load Balancer
Gateway Load Balancer
Classic Load Balancer

We will quickly go through them one by one and finally compare them with each other.

Application Load Balancer

It’s a Layer 7 load balancer. Operates at the application layer.
Aimed to handle HTTP and HTTPS traffic
It is capable of routing based on path patterns.
SSL can be offloaded to it. Supports SNI.
Even authentications can be offloaded to it.
Targets can be EC2, Lambda, and IP addresses.
Step by step ALB creation

Network Load Balancer

It’s a layer 4 Load Balancer. Operates at the transport layer.
Aimed to handle TCP, UDP, and TLS traffic
Uninterrupted end to end encryption till target
Ultra-low latency load balancers capable of handling millions of requests per second.

Gateway Load Balancer

It’s a Layer 3 Load Balancer. Operates at the network layer.
Aimed to handle virtual appliances traffic on GENEVE protocol.
Scale virtual appliances like Firewalls, IDP, etc. using this LB

Classic Load Balancer

It’s a combination of ALB and NLB offered by AWS formerly (with reduced features).
New deployments should not be using it.
Its existence is only for compatibility for old EC2-Classics running customers.

Lets compare all 4 Elastic Load Balancers side by side –

	ALB	NLB	GLB	CLB
OSI model layer	7	4	3	7 and 4
Protocol supported	HTTP, HTTPS	TCP, UDP, TLS	GENEVE	HTTP, HTTPS, TCP
Supports static IP for ELB	No, only DNS name	Yes	No	No, only DNS name
SSL offloading	Yes	Yes (TLS termination)	No	Yes
SNI support	Yes	Yes	No	No
Authentication offloading	Yes	No	No	No
End to end encryption	No if using SSL offloading	Yes	No	Yes
Sticky sessions	Yes	Yes	Yes	Yes
Path patterns	Yes
Cross zone load balancing	Enabled by default	Yes	Yes	Disabled. Enable it manually
Type of registered targets	Instance, Lambda, IP	Instance, IP	Instance, IP	Instance,IP
Use cases	Websites, web applications	Application requiring low latency load balancing	Load balancing or scaling virtual appliances for IDP, firewall etc.	Web applications.

I mentioned the commonly used features comparison above. Amazon published a very good comparison of all load balancers on this page.

Creating Application Load Balancer in AWS

Leave a reply

A step by step procedure to create an application load balancer for a web application.

Application Load Balancer creation in AWS!

This article will walk you through the steps to create an application load balancer and then testing. ELB can be used in Amazon ECS as well, but for this exercise, we will be using the below architecture, which is running webservers on EC2, not in containers. Its a subset of our custom VPC –

We have 2 EC2 instances running Apache webserver in 2 different public subnets. Application Load Balancer will receive traffic from the internet and forward it to the back-end EC2 instances.

Lets dive into ALB creation procedure –

Log in to the EC2 dashboard.
On the left navigation panel, click Load Balancers under the Load Balancing section.
On the load balancer page, click on the Create Load Balancer button.
You should be seeing load balancer creation wizard –

Here you need to select type of load balancer to create:

Application load balancer: For load-balancing HTTP, HTTPS web traffic.
Network load balancer: For load balancing TCP, TLS, UDP network traffic.
Gateway load balancer: For load balancing virtual appliances traffic over GENEVE.
Classic load balancer: Old ELB tech.

Click on the Create button under the Application load balancer to proceed. Load balancer configuration wizard should open up.

Fill in details –

Name: Name for ALB.
Scheme: Choose internet-facing since we are configuring the web load balancer.
IP address type: Select the addressing type.
Listeners: Choose HTTP with port 80. If your application is on HTTPS, then select accordingly.

VPC: Select VPC under which ALB will be deployed.
Availability zones: Select minimum 2 for HA. If you are creating internet-facing ALB, then subnets should have a route to the internet gateway, i.e., public subnets. You can select only one subnet per AZ.
AWS Global accelerator: For performance. This is part of integrated service and can be modified later as well.
Tags: Tagging.
Click on the Next: Configure Security Settings button at the end.

Since HTTP was selected in the basic configuration, a security notice should appear.

Click again on the Next: Configure Security Settings button in the end to proceed.

Create new or select an existing security group for ALB. We are selecting here existing SG, which allows HTTP traffic.

Click on Next: Configure Routing button.

In the routing section, we are configuring the destination for ALB. Here, ALB will come to know where it needs to direct traffic once it receives the traffic. As per our design, we are going to direct traffic to 2 EC2 instances.

Target group
- Target Group: Create new or use existing. It’s a collection of resources acting as targets for ALB.
- Name: for identification.
- Target type: In our case, its instance.
- Protocol: HTTP or HTTPs
- Port: Depends on your web application listening port. I am using the default web server listening on port 80
- Protocol version: Again depends on the web application.
Health checks
- Protocol: To be used by ALB to perform health checks on the target type.
- Path: ALB will reach out to this path using the mentioned protocol to determine health or target.

Advanced health check settings
- Port: Communication port for a health check.
- Healthy threshold: Number of consecutive successful health check before marking any unhealthy target as healthy. Range 2-10.
- Unhealthy threshold: Number of consecutive failed health checks to mark a target as unhealthy. Range 2-10.
- Timeout: If no response is received within this timeframe, mark the health check as failed. Range (2-120 secs)
- Interval: Time between health checks (5-300 secs)
- Success codes: HTTP code to be received back for marking health check as a success.

Click on Next: Register Targets button.

As you can see, we have 2 EC2 instances running in different zones that are available to register as a target for ALB. Meanwhile, also verify that both instances are serving the webpage properly.

Both our webservers are serving different webpages (to test on ALB later). We verified it by using the public Ip of the EC2 instances.

Now, select instances serving web traffic and click on the Add to registered button.

Both targets should be registered and list under the registered target list. Verify and then click the Next: Review button.

Review all configurations and click on Create button.

You should be seeing success message like one above.

Click the Close button, and it will take you to the load balancers page. In here, newly created application load balancers should be listed, and mostly it’s in provisioning state. After a couple of minutes, it should go into an Active state once at least one target passes the health check.

Once ALB is active, grab the DNS name from the ALB details screen and load it in the browser. It should populate the webpages from either of the EC2 targets.

The above small GIF shows the same ALB DNS loads web page from different backend targets. I created distinct web pages to identify the difference and functionality of ALB. In the real world, it should be serving the same page as any of the backend targets.

That’s it! We created an Application load balancer that serves the webpage from different backend servers!

Amazon ECR: Creating repository and pushing first container image

Leave a reply

A quick rundown on how to create an Amazon ECR repository and push container image to it.

What is Amazon ECR?

Amazon ECR, i.e., Elastic Container Registry, is a fully managed container image registry service provided by AWS. It is integrated with Amazon ECS so that developers can have a fully managed container platform by AWS. You can visualize it as your own docker hub.

Browse through our Amazon ECS related articles here.

How to create ECR repository?

Amazon ECR has its own home under Amazon ECS dashboard.

Login to Amazon ECR dashboard; click on Get started button
Or login to the Amazon ECS dashboard
- Click on Repositories in the left navigation panel
- Click on the Create repository button on the repository page.
You should be presented with the below screen –

Different configurations are –

Repository name: It will follow the below format
- account-id.dkr.ecr.region.amazonaws.com/name
Tag immutability: Prevents overwriting images with the same tags in the subsequent push.
Image scan settings: Enable it to scan images as soon as they are pushed to ECR for vulnerabilities.
Encryption settings: Use KMS or let ECR use default encryption for images once pushed to ECR.

Click Create repository button.

A repository should be created, and the ECR dashboard should enlist the newly created repository. Please select it and click on the View push commands button to get the authentication token, login, and push commands. These are ready-made commands for you to ease your tasks.

ECR will provide you with 4 commands pre-populated with the correct repository name. The explanation is given with each command is pretty self-explanatory.

How to push container image to Amazon ECR?

Now, we will use these commands to push the test container image to Amazon ECR. I will be using the Amazon Linux EC2 instance for this exercise. Make sure you have docker and aws-cli installed, configured, and running fine on the EC2 instance. Also, make sure EC2 is having at least an AmazonEC2ContainerRegistryPowerUser role attached so that ECR push works fine if you are using EC2 roles for authentication.

Step 1.

Get the authentication token and log in using the docker command. Its taken care of by the first command we saw in the previous screenshot. It has a two-part. First, aws command gets the authentication details and passes them to the second docker command for login.

[root@ip-10-0-0-226 ~]# aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Step 2.

Tag container image. I am assuming you already have an image created using Dockerfile. For testing, I pulled the Nginx image from DockerHub and tagged it.

[root@ip-10-0-0-226 ~]# docker tag nginx:latest 3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver:latest

Verify image tags.

[root@ip-10-0-0-226 ~]# docker images
REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver   latest              daee903b4e43        9 hours ago         133MB
nginx                                                    latest              daee903b4e43        9 hours ago         133MB

Step 3.

Push image to ECR using docker push command.

[root@ip-10-0-0-226 ~]# docker push 3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver:latest
The push refers to repository [3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver]
b9e73ac5343e: Pushed
5887d03dfc3d: Pushed
e3a971c30b12: Pushed
32048dd980c7: Pushed
f5600c6330da: Pushed
latest: digest: sha256:bb84ff0786cd1dbde780d84f6bf76bfdef36fe8ffa658f7f5c48e39363b4d500 size: 1362

If you come back to the AWS console and check under the repository, you should be seeing this latest pushed image there.

As you can see, since the Image scan was enabled, ECR scanned the image for vulnerabilities as soon as it was pushed. And added a report in the same view. Click on the image tag for checking image details.

If you don’t enable the Image scan setting, then the manual scan can be done by clicking the Scan button on the Images details screen.

Another point to note here is ECR showing image size as 53.61MB, whereas it was reported as 133MB on EC2 command outputs. This is because docker compresses the image layers when pushing the image to the repository. Note about the same can be found here in AWS documentation.

How to pull container image from ECR?

It’s a simple docker pull command. Provided you have EC2 instance and docker daemon properly setup for authentication (as mentioned above).

[root@ip-10-0-0-226 ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
[root@ip-10-0-0-226 ~]# docker pull 3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver
Using default tag: latest
latest: Pulling from webserver
852e50cd189d: Pull complete
a29b129f4109: Pull complete
b3ddf1fa5595: Pull complete
c5df295936d3: Pull complete
232bf38931fc: Pull complete
Digest: sha256:bb84ff0786cd1dbde780d84f6bf76bfdef36fe8ffa658f7f5c48e39363b4d500
Status: Downloaded newer image for 3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver:latest
3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver:latest
[root@ip-10-0-0-226 ~]# docker images
REPOSITORY                                               TAG                 IMAGE ID            CREATED             SIZE
3xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/webserver   latest              daee903b4e43        10 hours ago        133MB

Assorted list of resources to ease your AWS tasks

Leave a reply

Assorted list of resources to help you with your work in AWS!

In this post, I will quickly run down the assorted list of different software, tools, or online resources that will help you with your AWS journey. So without further delay, let’s jump into it.

AWS Native tools

AWS Pricing Calculator: Estimate your design cost beforehand and budget accordingly
AWS Simple Monthly Calculator: Estimate the monthly running cost of your infra on AWS. It is going to be deprecated soon and will be replaced by the former one.
AWS service health dashboard: Live status of AWS services. You can even use the AWS personal health dashboard, which will curate health status according to the infra being used in your account.

AWS console related

AWS Helper: Google chrome extension to manage and switch roles in AWS consoles easily!
AWS Extend Switch roles: Another one!
Authy MFA tool: Best in class. It has phone and desktop apps so comfortable to use. Encrypted backups.
Firefox multi-account containers: Use Firefox containers to log into different AWS accounts in single browser window.

AWS architecting

AWS Architecture Icons: List of supported drawing and diagramming tools and download links for Icons assets and toolkits. It’s very much helpful in designing presentable diagrams with updated, latest AWS icons. Every architecture’s must-have resource!
Draw.io: Create architecture diagrams with draw.io
Visual subnet calculator: Easy way to slice up your CIDR block into the required number of subnets.
CIDR Range Visualizer: An interactive CIDR IP addressing visualizer webpage!
AWS EC2 instances info: Single place to look at all EC2 instances, filters, costing, etc.

CLI lovers

pagent authentication agent: If you are an AWS engineer and logging into EC2 Linux instances frequently then this tool is pretty much handy. It does all authentication key’s heavy lifting where you can connect to any EC2s seamlessly. Even with the use of bastion hosts.
SQLectron: An open-source lightweight utility for connecting to RDS.

Infrastructure coding

Visual Studio Code: Most favoured and loved software for all coding you do on your computer. Supports almost all languages and clouds. Allows you to connect to your cloud and run your code from the software window itself.
cfn-lint plugin: Cloudformation Linter plugin for Microsoft Visual Studio Code.
InfraCost: cloud cost estimates on the terraform pull requests!

The list can grow on and on. I just collected a few of them here to start with. Let me know your additions in the comments down below!

The Container configurations in Amazon ECS

Leave a reply

A quick post on advanced container configurations in Amazon ECS.

Container definitions are part of Task Definitions in Amazon ECS. It’s the configuration where you can customize the container’s infrastructure aspects. In this article, we will walk you through advanced configurations of containers.

In our last article about Task Definitions, we walked you through standard container configurations. Now, we will check all the parameters available in Advanced container definitions.

Checking container on ECS instance

First verify if the container is running.

[ec2-user@ip-10-0-0-122 ~]$ docker container ls
CONTAINER ID        IMAGE                            COMMAND                  CREATED              STATUS                                 PORTS                   NAMES
2c2267e6ce85        nginx:latest                     "/docker-entrypoint.…"   About a minute ago   Up About a minute (health: starting)   0.0.0.0:32768->80/tcp   ecs-webserver-nginx-8-nginx-d28beae194c4eada5b00
9bb8f8b0b6ea        amazon/amazon-ecs-agent:latest   "/agent"                 2 minutes ago        Up 2 minutes (healthy)

[ec2-user@ip-10-0-0-122 ~]$ docker exec -it 2c2267e6ce85 /bin/bash
root@kt-web-container:/usr/share/nginx/html# cat /etc/resolv.conf
search kerneltalks.com
nameserver 1.1.1.1
nameserver 8.8.8.8
options timeout:2 attempts:5
root@kt-web-container:/usr/share/nginx/html# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.2.3.4        xyz.com
172.17.0.2      kt-web-container
root@kt-web-container:/usr/share/nginx/html#

You can see container hostname is set, DNS nameservers are set, extra IP-hostname pair has been added to /etc/hosts, logged in user is the root and working directory is set to /usr/share/nginx/html! Everything is accommodated.

Last thing to verify if the container is sending logs to the CloudWatch service. Click on the link View Logs in CloudWatch under container details on the Tasks page. (can be seen in the above screenshot)

And logs are being populated in CloudWatch!

That’s all! All advanced container configuration which one can configure under Amazon ECS Task Definition.

Kernel Talks

Unix, Linux, & Cloud!

Yearly Archives: 2020

Creating first AWS Lambda function

Replication in Amazon S3

What is replication in Amazon S3?

Why S3 replication is needed?

Types of S3 replications

Requirements and limitations of Amazon S3 replication

How to configure S3 replication?

How to check if object is replicated?

How to create a user with programmatic access in AWS

How to re-create IAM secret access keys?

How to create an Amazon SQS queue and test with Amazon SNS?

What is an Amazon SQS?

How to create Amazon SQS queue?

Sending messages to SQS queue from SNS

Creating Application Load Balancer in AWS

Assorted list of resources to ease your AWS tasks

AWS Native tools

AWS console related

AWS architecting

CLI lovers

Infrastructure coding

The Container configurations in Amazon ECS

Read more about Amazon ECS –

Checking container on ECS instance