Yearly Archives: 2021

Preparing for Certified Kubernetes Administrator (CKA) exam

A small rundown on CKA preparation.

In this post, I will be sharing various resources to help you prepare for the CKA exam. In addition, feel free to add resources you know in the comments section, which may help fellow readers.

Exam details

Offered by: The Cloud Native Computing Foundation (CNCF)
Duration: 2 hours
Type: Complete tasks on Linux CLI (Practical)
Number of questions/tasks: 15-20 (I had 17)
Mode: Online proctored
Cost: $375 (that includes one free retake). Watch out over LinkedIn or internet for coupons. They got good deals on black friday as well.
Result: It will be available in 24 hours from the exam completion.
You are allowed to open one additional browser tab to access K8s docs, K8s Github or K8s blog. You should not be clicking/opening any links other than these domains that includes K8s forum as well.

Study journey

Having Linux and container background helps picking up Kubernetes world quickly. No prior cloud experience is required.
If you are not familier with containerization, its good to look into it before starting with Kubernetes. Docker containers are the best place to get started with.
The complete curriculam hosted on Github.
There are couple of online courses that will get you started :
- Certified Kubernetes Administrator (CKA) with Practice Tests by Mumshad Mannambeth. The best seller!
- Certified Kubernetes Administrator (CKA) by William Boyd

Practise course labs heavily. You may go through course quickly to understand the Kubernetes world but you need to spend more time on practising Kubernetes on CLI.
Online free labs for practising :
- Katakoda playground
- K8s labs

Once you are good with the theory and understood all aspects of Kubernetes world, Labs are the only places where you should spend all of your study time.
Once you are through all the scenarios/tasks provided by online courses, you can think of your own custom scenarios and try implementing them.

Tips

Practise! Practise!! Practise!!! The more you are familiar with the CLI and commands, the more time you will save during the exam. In addition, it helps to build your muscle memory for command arguments and gain those extra seconds during the exam.

CKA requires you to complete the given tasks in the Linux terminal (Ubuntu) CLI on a shared Kubernetes cluster setup. So, having good Linux background is added plus! Moreover, it helps you in navigating through CLI, editing files and save a lot of time.

Make use of -h frequently! If you are not sure about the command arguments, use a -h flag that lists arguments along with example commands. You can directly copy those example commands and edit them accordingly before executing. A quick way to get the job done rather than navigating through kubectl commands on Kubernetes documentation

Try to complete tasks using imperative commands rather than building spec files.

Read the question carefully and completely before creating any objects. Keep an eye on the namespaces mentioned in the questions. Assume default namespace when no specific namespace is mentioned.

Verify created objects to make sure they carry properties asked in questions. For pods, make sure they reach running state before proceeding.

Setting alias in Linux shell is one of the famous tips you will come across over the internet. Use it according to your comfort. I did not use it.

Always make sure you run the given context commands at the start of each task. It makes sure you are on the right cluster to perform the task.

Always make sure to return to the main terminal if you are doing ssh to other nodes for performing the tasks.

For Tasks mentioning sudo -i for root privileges, it’s good practice to switch to root as soon as you log in to the respective node rather than finding out you are not run after running some commands and investing time there!

If you are not familiar with Linux editors like vi, edit your spec files in the exam provided notepad and then copy the final version of the config directly on the terminal within the file rather than running around in Linux editors and wasting time.

Get familiar with copy, paste operations in the terminal. There are different key combinations depending on the operating system. Refer exam handbook for the same. Then, practise using those key combinations.

Use kubernetes.io/docs heavily during practice. If you are stuck at something, always try to search and get information from Kubernetes official documentation. This will make you comfortable navigating through the documentation site and hence saves some time during the exam. In addition, you will know exact keywords to search and exact links to click on topics you had a hard time studying.

It’s the student’s responsibility not to click/open any other sites than the allowed three. Search in K8s documentation may yield results with links to the K8s forum. You should not be clicking them. Make a habit of checking links before opening to avoid issues during the exams.

Please note that the exam simulator you get along with your exam booking has more challenging questions than the actual exam. They mentioned it explicitly there. So if your morale goes down pretty quickly, then it’s best not to check those questions just before the exam :P. They aim more at getting an in-depth understanding of how things run under the hood.

That’s all I have. All the best!

How to configure switching IAM roles in AWS CLI?

Leave a reply

A short howto on configuring AWS CLI to switch roles

Requirement:

You have one AWS account that needs to switch roles before executing things on AWS. It’s an easy method on AWS console, but how to switch roles in AWS CLI.

Solution:

Let’s consider the below setup-

AWS IAM account with programmatic access – user101
Same IAM account having sts:AsumeRole permissions.
AWS IAM role for above said IAM user to assume (same or cross-account)- role101

Start with configuring the AWS CLI in a standard way.

$ aws configure --profile user101
AWS Access Key ID [None]: AKIAQX3SNXZGUQFOSK4T
AWS Secret Access Key [None]: 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
Default region name [us-east-1]: us-east-1
Default output format [None]: json

Note: It is not a good practice to keep AWS credentials in a plain text format. Keep them in a secured encrypted way using aws-auth.

Now, at this point, you must have an AWS credentials file created in the home directory.

$ cd ~/.aws
$ cat credentials
[user101]
aws_access_key_id = AKIAQX3SNXZGUQFOSK4T
aws_secret_access_key = 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
region = us-east-1
output = json

You need to edit the above credentials file to add IAM role details. Append the below configuration in the file.

If you are working with AWS Gov Cloud make sure the ARNs has proper AWS Partition defined. E.g. arm:aws-us-gov:x:x:…..

[role101]
role_arn = arn:aws:iam::xxxxxxxxx:role/role101
output = json
source_profile = user101

where –

role101 is a Role identifier. You can choose as per your choice.
Mention the correct IAM role ARN
source_profile should use the profile identifier of the user who will assume this role. In our case, its user101.

Save the file, and you are ready to go.

Test configurations –

$ aws sts get-caller-identity
{
    "UserId": "AIDAQX3SNXZG3Z2AXNIMJ",
    "Account": "xxxxxxxxx",
    "Arn": "arn:aws:iam::xxxxxxxxx:user/user101"
}

$ aws sts get-caller-identity --profile role101
{
    "UserId": "AROAQX3SNXZG6KL4YENFZ:botocore-session-1631087792",
    "Account": "xxxxxxxxx",
    "Arn": "arn:aws:sts::xxxxxxxxx:assumed-role/role101/botocore-session-1631087792"
}

You can see this by using --profile role101 we are assuming the IAM role role101 for the user user101.

AWS CLI configuration for switching roles using MFA

Note: If you are on Windows and using GitBash, refer to configuring GitBash for MFA prompts. It works perfectly in Powershell.

In some cases, your AWS environment must have MFA restrictions in place where the user user101 must have MFA enabled to switch to the role role101. In such a scenario, your role profile in credentials files should include MFA device ARN as well like below –

[role101]
role_arn = arn:aws:iam::xxxxxxxxx:role/role101
mfa_serial = arn:aws:iam::xxxxxxxxx:mfa/user101
output = json
source_profile = user101

where –

mfa_serial is the ARN of the MFA device of user101.

You will be prompted to supply the MFA code whenever you use profile role101 in AWS CLI commands.

$ aws sts get-caller-identity --profile role101
Enter MFA code for arn:aws:iam::xxxxxxxxx:mfa/user101:
{
    "UserId": "AROAQX3SNXZG6KL4YENFZ:botocore-session-1631089277",
    "Account": "xxxxxxxxx",
    "Arn": "arn:aws:sts::xxxxxxxxx:assumed-role/role101/botocore-session-1631089277"
}

How to find AWS resources that need to be tagged

Leave a reply

A quick rundown on how to hunt AWS resources that needs tagging

Tags are the most important and equally negligible AWS entity! As AWS spread grows in an organization they start to realize the importance of tags and then comes the projects for tagging existing resources!

At this stage, the first question on the table is how to search for AWS resources that need tagging? or How can we search non-tagged AWS resources?

It’s a very short process that can be summarised in a single picture!

Breaking it down –

Login to AWS Resource groups console.
On left hand side menu, select Tag Editor under Tagging.
Now you should have seelction on right hand side.
Select perticular region or All regions from Regions drop down.
Select specific resource or All supported resource types from Resource types drop down.
Tags – Optional: You can specify key, value details to search for specific tags. Since we are searching for resources that are not tagged lets keep it blank.
Finally, click on Search resources button and you are done!
You should be presented with list of AWS resources in specified regions that needs to be tagged like below.

You can export the list to CSV as well for further data analytics.

Configuring ALB authentication with Amazon Cognito for ConsoleMe webapp

Leave a reply

A quick article on how to configure ALB Auth via Amazon Cognito for ConsoleMe webapp

In our last article, we looked at Netflix’s IAM management tool ConsoleMe. We installed it on the Ubuntu Linux machine with the Local install method, and we got it up and running with the default example configuration. In this article, we will walk you through the process of configuring ALB authentication for the ConsoleMe webapp.

With the default example configuration, ConsoleMe webapp opens up without any authentication. However, since ConsoleMe will manage your AWS account’s IAM, it’s not safe to keep your keys to kingdom open on the internet without any authentication mechanism in place. Hence, we will be protecting it with the ALB auth method.

ConsoleMe supports webapp authentication via –

ALB Auth (Recommended)
Google groups
OIDC/OAuth2
SAML
Headers

As recommended by ConsoleMe, we will move ahead with ALB Auth.

Let’s get into it.

Pre-requisites

ConsoleMe is up and running
ALB is configured to listen on HTTPS with target group configured on 8081 port with ConsoleMe instance as target
Above mentioned setup is working correctly and you are able to open up ConsoleMe webapp using ALB DNS name/DNS ALIAS you configured for ALB.
Amazon Cognito user pool is created. Read how to create Cognito user pool

Procedure

Before you proceed, you need to make few configuration changes in the Amazon Cognito user pool if you have followed the above link to create it.

Edit/ make sure you have the below configurations in the Cognito user pool’s App client settings :

Callback URLs are set to http://DNS-NAME/auth, http://DNS-NAME/oauth2/idpresponse. Where DNS-NAME is DNS name of ALB or the ALIAS defined for DNS name.
Enable Authorization code grant
Allowed Oauth scope has email, openid and profile enabled.

Now, you need to edit ALB listener rules to enable Amazon Cognito authentication. Next, follow the step by step procedure to enable Cognito authentication on ALB.

Apart from the steps in the above link, you need to add extra rules in the HTTPS listener that forwards below path patterns directly to the target group bypassing Cognito authentication.

/api/v1/get_roles*
/api/v2/mtls/roles/*
/api/v1/get_credentials*
/api/v1/myheaders/?
/api/v2/get_resource_url*
/noauth/v1/challenge_poller/*
/noauth/v1/challenge_generator/*

ConsoleMe leverages these path patterns to perform CLI actions and authentication.

After adding them, your listener should look like –

Now, the final step is to do the custom configuration on the ConsoleMe part. As you are aware from the ConsoleMe installation that it runs the default example configuration. We should override that open configuration with the one that supports ALB Auth. Copy the sample configuration file from GitHub here.

Save this file as in the installation directory. We installed ConsoleMe in /consoleme directory, so it should be saved as /consoleme/consoleme.yaml

Edit below parameters in the configuration file –

application_admin: Email that will receive the approval requests
metadata_url: Replace region and Cognito pool id.
is_example_config: false
ses: Edit if you are using SES
aws: with relevent details

Restart application

systemctl restart consolme
OR
python consoleme/__main__.py

Now, your application is reading the newly created configuration file. Next, open up ALB’s DNS/ ALIAS DNS, and you will be prompted to log in from Cognito. We discussed this part in an earlier article on Cognito authentication.

Once you are successfully authenticated, you should see the ConsoleMe console with custom config!

Notice that the example configuration notice is vanished now. Also, you can see Cognito user ID’s email as a logged-in user in ConsoleMe!

We successfully enabled ALB Auth for securing ConsoleMe webapp!

Netflix’s ConsoleMe local installation on Linux machine

1 Reply

A step by step guide to install ConsoleMe on Ubuntu Linux machine

ConsoleMe is an open-source web service published by Netflix. It is designed to make life easy for end-users and cloud administrators. Using ConsoleMe, cloud administrators can manage IAM permissions/credentials for IAM roles, S3 buckets, SQS queues, and SNS topics across multiple AWS accounts from a single interface. It also provides CLI called weep for AWS credentials management. That’s a fair introduction if you are not aware of the tool. Next, let’s get into the installation part.

ConsoleMe offers docker and local installs. We will walk you through local install in this article.

Pre-requisite:

A machine running Ubuntu 19.04+ with root access. I used Ubuntu 20.04 LTS x86.
Active and working package manager subscription to install packages
Storage requirement: 2GB of disk space
An AWS user/role for consoleme service with appropriate permissions
AWS access keys for above user if you are not using roles. I used keys (steps below)

Installation

We are installing ConsoleMe in /consoleme directory. If you want to install in another location, make the necessary changes in the commands below. Let me give you a list of commands you need to run as root –

apt-get update
apt-get install build-essential libxml2-dev libxmlsec1 libxmlsec1-dev libxmlsec1-openssl musl-dev libcurl4-nss-dev python3-dev pkg-config python3.8-venv awscli docker-compose -y
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash
apt-get install -y nodejs
npm install yarn -g
cd /
git clone https://github.com/Netflix/consoleme.git
cd consoleme
docker-compose -f docker-compose-dependencies.yaml up -d

Here, the first few commands are installing all the dependencies and related software/tools. Then, we are cloning the GitHub repo of the tool in /consoleme and lastly, we are running two containers.

These are Redis and dynamodb containers that ConsoleMe leverages for caching and aggregating the AWS accounts information. You can make use of AWS Redis and dynamodb table services, but for now, we will run these containers locally so that ConsoleMe will talk to them rather than AWS services.

I am avoiding putting up console outputs for frequently used commands like package installations etc., here.

Make sure both containers are up and running before proceeding to the next step –

root@kerneltalks:/consoleme# docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS         PORTS                              NAMES
5333cdee2202   cnadiminti/dynamodb-local         "java -jar DynamoDBL…"   10 seconds ago   Up 4 seconds   8000/tcp, 0.0.0.0:8005->8005/tcp   consoleme-dynamodb
19ac354c3d70   redis:alpine                      "docker-entrypoint.s…"   10 seconds ago   Up 4 seconds   0.0.0.0:6379->6379/tcp             consoleme-redis
4cf931d38652   aaronshaf/dynamodb-admin:latest   "node bin/dynamodb-a…"   10 seconds ago   Up 4 seconds   0.0.0.0:8001->8001/tcp             consoleme-dynamodb-admin

Now, you need to prepare the machine to talk with AWS for fetching account details in the upcoming install steps. Ensure that you have set up account and permissions perfectly in IAM (mentioned in the pre-requisite above) to avoid any issues. You can do that by configuring AWS profile –

root@kerneltalks:/consoleme# aws configure
AWS Access Key ID [None]: AKIAQX3STVKIYRO36XEC
AWS Secret Access Key [None]: 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
Default region name [None]:
Default output format [None]:

Lastly, create a new python environment and run the final install step. This will take a while to complete since at the end of make install command, it also fetches and caches the AWS account details in the local Redis cache –

python3 -m venv env
. env/bin/activate
make install

After successful installation, you should be able to start the application.

Running ConsoleMe

On a current shell, you can run the ConsoleMe with the command. If you are in another shell, activate the python environment again –

(env) root@kerneltalks:/consoleme# python consoleme/__main__.py
{"asctime": "2021-07-25T08:32:16Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "jwt.py", "funcName": "<module>", "levelname": "ERROR", "lineno": 14, "module": "jwt", "threadName": "MainThread", "message": "Configuration key `jwt.secret` is not set. Setting a random secret", "eventTime": "2021-07-25T01:32:16.286230-07:00", "hostname": "kerneltalks", "timestamp": "2021-07-25T08:32:16Z+0000"}
2021-07-25 08:32:17,322 - DEBUG - root - [constants.py:39 - <module>() ] - Leveraging the bundled IAM Definition.
2021-07-25 08:32:17,322 - INFO - root - [iam_data.py:10 - <module>() ] - Leveraging the IAM definition at /consoleme/env/lib/python3.8/site-packages/policy_sentry/shared/data/iam-definition.json
2021-07-25 08:32:17,824 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
2021-07-25 08:32:17,859 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
{"asctime": "2021-07-25T08:32:18Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "__main__.py", "funcName": "init", "levelname": "DEBUG", "lineno": 57, "module": "__main__", "threadName": "MainThread", "message": "Server started", "eventTime": "2021-07-25T01:32:16.286230-07:00", "hostname": "kerneltalks", "timestamp": "2021-07-25T08:32:18Z+0000"}

But, it will exit out when you terminate the command or shell. It’s safe to run it in the background or, even better, run it as a Linux service. For running ConsoleMe as a service, create below two files –

File /usr/bin/consoleme_start.sh

#!/bin/bash
. env/bin/activate
python consoleme/__main__.py

File /etc/systemd/system/consoleme.service


[Unit]
Description=Run consoleme service.

[Service]
Type=simple
User=root
WorkingDirectory=/consoleme
ExecStart=/usr/bin/consoleme_start.sh

[Install]
WantedBy=multi-user.target

Assign executable permissions to

chmod +x /usr/bin/consoleme_start.sh

Enable and start the service

root@kerneltalks:/consoleme# systemctl enable consoleme
Created symlink /etc/systemd/system/multi-user.target.wants/consoleme.service → /etc/systemd/system/consoleme.service.

root@kerneltalks:/consoleme# systemctl start consoleme

root@kerneltalks:/consoleme# systemctl status consoleme
● consoleme.service - Run consoleme service.
     Loaded: loaded (/etc/systemd/system/consoleme.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2021-07-25 08:35:52 UTC; 7s ago
   Main PID: 14775 (consoleme_start)
      Tasks: 5 (limit: 4706)
     Memory: 159.7M
     CGroup: /system.slice/consoleme.service
             ├─14775 /bin/bash /usr/bin/consoleme_start.sh
             └─14776 python consoleme/__main__.py

Jul 25 08:35:52 kerneltalks systemd[1]: Started Run consoleme service..
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: {"asctime": "2021-07-25T08:35:53Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "jwt.py", "funcName": "<module>", "levelname": "ERROR", "lineno": 14, "m>
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:53,954 - DEBUG - root - [constants.py:39 - <module>() ] - Leveraging the bundled IAM Definition.
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:53,955 - INFO - root - [iam_data.py:10 - <module>() ] - Leveraging the IAM definition at /consoleme/env/lib/python3.8/site-packages/policy_sentry/shared/data/i>
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:54,354 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:54,361 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: {"asctime": "2021-07-25T08:35:54Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "__main__.py", "funcName": "init", "levelname": "DEBUG", "lineno": 57, ">

ConsoleMe GUI

Now that your console service is running, you should load its GUI on a web browser. The service listens on the 8081 port, so you need to navigate the server address with port 8081. Make sure the security group is allowing 8081 traffic if you are installing on EC2.

At this point, ConsoleMe is running with the default open example configuration. It’s very well highlighted on the web app as a warning. It would be best if you were editing this configuration to make your ConsoleMe more secure. ConsoleMe recommends Application Load Balancer authentication for securing your web app GUI. Refer to our next article on how to secure the ConsoleMe web app using ALB authentication.

How to enable ALB authentication using Amazon Cognito

Leave a reply

A quick step by step process to configure Amazon Cognito authentication on AWS Application Load Balancer!

Pre-requisite or assumption:

Application Load Balancer is up and running with HTTPS listner
Load balancer has active backend serving webpage/webapp that you want to protect with Cognito authentication
Amazon Cognito user pools is created. See our article on how to create user pool

Configuration

Let’s get into configurations now. First, we need to edit the ALB’s listener rule to add Cognito authentication.

Log into EC2 console
Navigate to the Load Balancing > Load Balancers in left sidebar menu
On Load balancers page, select the Application load balancer that needs to be configured
Click on Listeners tab in below details pane.
Click on View/Edit rules against HTTPS 443 listner

You should see the editor window where you need to click on pen icons to open up the rule editor.

Now, click on + Add action button. Select Authenticate from the drop-down. The rule will be listed in the editor where you need to select from dropdowns-

Cognito user pool
Client ID

Once you select Cognito user pool it will accordingly populate values for Client ID in that pool. Choose respectively. Keep other options untouched unless you have other requirements.

If you have not created a Cognito user pool or need any clarity around it, please go through our article on how to create an Amazon Cognito user pool for ALB authentication.

New Authenticate action will be prioritized to rule 1 and existing Forward to action will be getting down to number 2. Your rule windows should look like this –

Adding Cognito authenticate action in ALB listener rule

Click on Update to save the rule.

If you are facing any issue to populate User pool ID and seeing Too many requests error in console then you can opt to re-create ALB with this configuration using AWS CLI or CloudFormation. Because using AWS CLI, you can not edit the default rule. If you have ALB created from the Cloudformation template already then you just need to tweak your template and add the below code for the ALB listener resource.

Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: <ALB ARN>
      Port: 443
      Protocol: HTTPS
      Certificates:
        - CertificateArn: <HTTPS CERT ARN>
      DefaultActions:
        - Type: authenticate-cognito
          AuthenticateCognitoConfig: 
            OnUnauthenticatedRequest: authenticate 
            UserPoolArn: <AMAZON COGNITO USER POOL ARN>
            UserPoolClientId: <USER POOL CLIENT APP ID>
            UserPoolDomain: <USER POOL DOMAIN PREFIX>
          Order: 1
        - Type: forward
          ForwardConfig: 
            TargetGroups: 
              - TargetGroupArn: <TARGET GROUP ARN>
                Weight: 1
          Order: 2

Make sure your HTTP listener rule (port 80) is set to redirect to HTTPS.

Now ALB is ready to authenticate users before their requests are forwarded to target group targets. Grab the ALB DNS from the load balancers page. If you are having CNAME/ALIAS entry for ALB then you should be using the custom DNS you have defined.

Log into EC2 console
Navigate to the Load Balancing > Load Balancers in left sidebar menu
On Load balancers page, select the Application load balancer that needs to be configured
Click on Description tab in below details pane.
Copy DNS name

Getting the Application load balancer DNS name

Paste DNS name in the browser and you should be redirected to the authentication page from Amazon Cognito. You can customise this page along with its domain as we explained in our earlier article on the Amazon Cognito user pool.

Amazon Cognito authentication in action!

Notice that we configured a custom domain for Amazon Cognito hence our request is served with a custom domain auth.kerneltalks.com. If you observe the URL carefully, you can see Client ID, Callback URL, etc. details configured in Amazon Cognito when we created the user pool.

For any Cognito user, the default behaviour is to force change password on the first login which we observed here –

Cognito force change password on first login

During change password, you can even see password policy applied while creating the Amazon Cognito user pool is being checked –

Cognito user pool password policy in action

For testing purposes, if you want to bypass all the above password change procedures then you can set the password from AWS CLI using the below command –

aws cognito-idp admin-set-user-password --user-pool-id <pool-id> --username <user> --password <password> --permanent

Once the password is changed successfully, you will be redirected to the Callback URL configured in Amazon Cognito. We configured it to the ALB DNS name. So we redirected to it and backend EC2 target served us a sample webserver page!

Thus we configured Amazon Cognito authentication on the Application Load balancer and secured our targets!

How to create an Amazon Cognito User pool for ALB authentication

1 Reply

A step by step procedure to create an Amazon Cognito user pool. All available options are explained.

One of the best features of AWS application load Balancers (ALB) is authentication! You can offload authentication to ALB that leverages Amazon Cognito in the backend. Amazon Cognito offers identity management through user pools or federated identities. This article will walk you through creating a user pool in Amazon Cognito that is used for ALB authentication. Without further delay, let’s get into it.

Login to Amazon Cognito console
Click on Manage User Pools
On the User pools page, click on Create a user pool button on top right hand corner of the page.
That should start user pool creation wizard. Lets go through it one by one –

Name

Enter the Pool name and click on the Step through settings button.

Attributes

Settings on this page can not be edited later so choose wisely!

The first thing you need to choose is the end user’s sign-in method. They should use a username or email address/phone number to signup/sign in. I am choosing a username and also, allowing them to use email addresses while logging in later once they sign up. I am also selecting case sensitive usernames because that makes more sense.

via CognitoChoose the way end-user sign in

The next section of attributes, let you choose through the list of attributes you want the end user to provide when they sign up in Cognito. You can also choose to add a custom attribute here if one is not listed in the standard list.

Policies

End-user password policies and controls are defined in this section. All the fields are pretty self-explanatory.

MFA and verifications

An extra layer of account security can be defined here. MFA and related configurations. Please note that if you are enabling MFA for end-users then you should be enabling phone number attributes in earlier settings and text messages (verification and subsequent messaging) will incur extra charges. Amazon pretty much explained each option here.

If you are opting for adding and managing phone number attributes then you need to create a role that provides access to Cognito for sending text messages on your behalf.

Messages customizations

In this section, you should be customizing the email or SMS messages being sent out by Amazon Cognito on your behalf. It’s a place if you want to have company branding in the communications! Make sure you have a verified email address in Amazon SES to set it as From email address.

In the later part of the page, you can configure how you want verification to be done using codes or clickable links. Also, you can customize the text of the message here.

Devices

Choose if Cognito should remember the user’s device. This will enhance the user experience. But, to use this feature you should have MFA enabled for end-users. Since we did not opt for it, we will simply say No and move forward.

App clients

In this section, you should create an app client which will access this user pool. On the creation of the app client, you will receive the app ID and secret key that you can configure in your applications to access this user pool.

Click on Add an app client

App client name: Add unique name
Refresh token expiration: Refresh tokens are used to retrieve new ID and access token. Control their expiration here. Read more about refresh tokens
Access token expiration: Used for autorizing the API operation. Control expiration here. Read more about access tokens
ID token expiration: It used to claim the authenticated user’s identity. Define its expiration limits here. Read more about ID tokens.
Auth Flows Configuration: Enable depends on your integration requirements. I selected ALLOW_USER_PASSWORD_AUTH and left others untouched.

Security Configuration: It allows to send back generic error. Select recommened unless you have any other reason not to!
Advanced token settings: Enable or disable token revocations.
Attributes read and write permissions: Select list of attributes which this app client can read or write.

Click on Create app client. It will be created along with the user pool when you completes the whole wizard.

Click on the Next step to move forward in the user pool creation wizard.

Triggers

On this page, you can configure lambda functions to be triggered on specific actions or workflow. You need to create Lambda functions in advance to select here from the dropdown. List of triggers available here –

Pre sign up
Pre authentication
Custom message
Post authentication
Post confirmation
Define Auth Challenge
Create auth challlenge
Verify authc challenge response
User migration
Pre token generation

All triggers are listed with descriptions for easy understanding of when they will be activated and execute related Lambda functions. For the simplicity of this article, we are not adding any.

Review

Review all the details you supplied throughout the wizard. You can make edits if necessary and then lastly click on Create pool

You should be greeted with a success message and the user pool management page. You can note the user pool ID generated for this user pool.

Amazon app clients settings

Now, that you created a user pool and app client for it. Let’s look at some of the settings those needs to be checked or changed to make sure your app client is ready to be consumed.

Configure Amazon Cognito app client’s IDP settings

Navigate to App integration > App client settings on the left sidebar menu on the user pool page.

Enable Cognito user pool under Enabled identity providers.
You should be having Callback URLs handy to fill in here. Those are URLs where app will be navigated once successful authentication happens. Your application developers should be able to help you with these details.
Sign out URLs are those where user will be redirected once its signed out from IDP session
OAuth 2.0 settings should be discussed with developer and configured as the app requirement

What is Amazon Cognito domain and how to configure it?

It’s a domain prefix with FQDN https://<prefix>.auth.<region>.amazoncognito.com where,

prefix : unique identifier of your choice
region: AWS region where user pool is hosted.

This domain is used to host sign-up and sign-in pages by Amazon Cognito. You can edit those pages for your company branding as well as explained in the next step.

Navigate to App integration > Domain name on the left sidebar menu on the user pool page.

Enter the prefix in the given text box and click Check availability. It will make sure you chose a unique prefix. Click on Save changes

You can opt to choose your own domain as well. You need to have an associated SSL certificate in ACM and permission to add the ALIAS record in the DNS hosted zone.

Once done, Cognito will create Amazon Cloudfront distribution for that domain in the backend and supply you with the alias target value to be configured in the hosted zone.

Add ALIAS record (CNAME for non-Route53) for Domain name and Alias target mentioned above. Once done and CloudFront distribution is created, your domain status will be set to ACTIVE.

How to change login UI of Amazon Cognito?

Navigate to App integration > App client settings on the left sidebar menu on the user pool page.

On the last part of the page, you can find Hosted UI settings. There you will be able to play around with CSS, logo files to create a new custom login page.

Make sure you have Amazon Cognito domain name defined and at least one OAuth scope defined (above step)

How to retrieve Amazon app client secret?

Navigate to General settings > App clients on the left sidebar menu on the user pool page. And there you can retrieve app client secret.

How to connect RDS with AWS IAM authentication

Leave a reply

A quick post listing step by step procedure to connect RDS database configured with IAM authentication.

We are considering RDS running MySQL and configured with AWS IAM authentication option throughout this post. However, if you are using a different database engine, consider editing commands/arguments whenever necessary.

Basically, we will be creating a local database user that leverages AWS IAM for authentication. Then EC2 will be configured with an IAM role or aws configure that has appropriate policies defined. And lastly, the user will generate an authentication token and log into the RDS database.

Why should we use IAM authentications for RDS?

Here is a list of reasons that are helpful to understand the benefits of the IAM authentications option for RDS.

IAM tokens used to log into the RDS database are valid for 15 minutes only. So they are more secure than permanent username/password pairs, and administrators don’t need to enforce/manage password reset policies.
IAM tokens are generated by making API calls to the AWS IAM service whenever needed. As a result, storing tokens is useless, and even if someone does, that does not pose a security threat due to its short life.
Applications can use EC2 instance profiles for generating tokens, so there is no need to store authentication information anywhere for applications to consume.

What you need?

You should be equipped with below inventory before hand –

RDS instance up and running configured with IAM DB authentication.
EC2 instance configured with AWS CLI (make sure SG allows the connectivity between EC2 and RDS on database port)
Master user login to RDS datatase
Access to AWS IAM service.

Creating user on database for the RDS access

For this step, you need to log in to the RDS database with the master user and create a new user. If you are on windows, you can use a lightweight tool like Sqlectron, or if you are already on EC2, you can use CLI as well –

For SQL CLI :

[root@kerneltalks ~]# mysql -h kerneltalks-rds.cn8uwrapea4b.us-east-1.rds.amazonaws.com -P 3306 -u admin -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 8.0.20 Source distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> CREATE USER 'kt_iam_user' IDENTIFIED WITH AWSAuthenticationPlugin as 'RDS' REQUIRE SSL;
Query OK, 0 rows affected (0.01 sec)

Make necessary changes to username and RDS endpoint!

Create IAM role for RDS access

Navigate to AWS IAM
Create the IAM policy (sample policy below)
If you intend to use the EC2 instance profile, create an IAM role for AWS service EC2 and attach the IAM policy to it.
If you intend to use only IAM users, then make sure you configure your CLI with aws configure command by supplying access key ID and secret access key.
Make necessary changes to the resource section!

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
             "rds-db:connect"
         ],
         "Resource": [
             "arn:aws:rds-db:us-east-1:986532147:dbuser:db-kerneltalks-rds/kt_iam_user"
         ]
      }
   ]
}

Download the SSL root certificate available for all regions from S3 bucket. This certificate required while making RDS connection since we enforced SSL on the database user. This ensures data is encrypted in flight.

[root@kerneltalks ~]#  wget https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem
--2021-07-03 14:44:12--  https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem
Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.205.189
Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.205.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1456 (1.4K) [binary/octet-stream]
Saving to: ‘rds-ca-2019-root.pem’

100%[===================================================================================================================================>] 1,456       --.-K/s   in 0s      

2021-07-03 14:44:12 (81.2 MB/s) - ‘rds-ca-2019-root.pem’ saved [1456/1456]

Now your EC2 or IAM user is ready to access RDS.

Connect to RDS using IAM token

It’s time for you to generate an IAM token and connect to RDS. We will save the token in the Shell variable for easy management and pass that shell variable token into RDS connect command. If the downloaded certificate is not in the PWD then use the absolute path for the pem file.

[root@kerneltalks ~]# token=`aws rds generate-db-auth-token --hostname kerneltalks-rds.cn8uwrapea4b.us-east-1.rds.amazonaws.com  --port 3306 --region us-east-1 --usernam
e kt_iam_user`
[root@kerneltalks ~]# mysql -h kerneltalks-rds.cn8uwrapea4b.us-east-1.rds.amazonaws.com -P 3306 --ssl-ca=rds-ca-2019-root.pem -u kt_iam_user --password=$token --protocol
=tcp
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 59
Server version: 8.0.20 Source distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]>

And you are connected!

Here we have used IAM token as password to connect to RDS database! Make a note that, these generated tokens are good only for 15 minutes. If you are using connect command after 15 minutes you need to generate the token again.

Just for anote, token looks like this –

[root@kerneltalks ~]# echo $token
kerneltalks-rds.cn8uwrapea4b.us-east-1.rds.amazonaws.com:3306/
NXZG4BNZMCVY%2F20210703%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Date=20210703T153110Z&X-Amz-Signature=04605698c01f3de6b2d7a7a48ddf28008d290795e11e
e4ac89587cbb3cdd9661

Understanding the authentication flow

For your understanding, lets see how authentication flows under the hood –

User runs the token generation command with database name, port, region and username for which token to be generated.
RDS sends back API token with 15 min lifespan. It requires your EC2 Instance role/IAM user to have rds connect permission. We covered this by defining IAM policy.
The user attempt to connect to RDS using the token acquired in the previous step.
A secure connection establishes, and the user logs in only if
- The root certificate is valid
- IAM permissions are in place and valid for db:connect
- Supplied username has RDS authentication set in the database
- The supplied token is generated for the same username, and currently, it’s not expired.
A user granted access and logs in. SQL prompt will be presented.

How to connect AWS RDS database from Windows

Leave a reply

A quick article for AWS beginners on connecting the AWS RDS database from Windows.

A step by step RDS database login procedure using the lightweight SQL client. This article is intended for AWS beginners who wanted to learn about RDS service with a little bit of database hands-on. Being said that, there is always a question for the non-DB guys; “How did I connect with this RDS database?”. So here we will walk you through step by step procedure for doing that.

For connecting to RDS database you should have –

RDS database up and running
RDS database endpoint
A SQL client
Connectivity between your machine and RDS database

Throughout this article, I am considering the MySQL database on RDS. If you are using a different database engine on RDS, then the connection port may vary. Also, we are considering ‘Password authentication’ of authentications options here –

Password authentication: RDS configured with this option has users managed at the database level.
AWS IAM database authentication: RDS configured with this option authenticates users by leveraging AWS IAM service. As a result, users are managed outside the database.

Let’s get started!

Identify RDS Endpoint

Navigate to the RDS console
Go to Databases and select your database
Database details screen should open up where you can copy your RDS endpoint.

Prepare SQL client

Download lightweight SQL client Sqlectron
Install and open the Sqlectron client.

Click on Add button to enter RDS connection details. We are testing RDS with ‘Password authentication’ here hence user and password needs to be supplied.

Click on Test to verify the connectivity.

If you encounter the below error, try to connect from a machine with direct connectivity to RDS. You can analyse RDS security groups to determine allowed subnets.

Verify below things –

There are no firewall rules between your machine and RDS blocking the port 3306 traffic.
RDS database is publicly accessible. (Obv. It applies to testing POC databases). If not, you can configure RDS to be publicly accessible.
If you don’t want your RDS to be publicly accessible, you need to connect RDS from the allowed subnets. That means your machine needs to be in the same VPC as RDS (e.g. Windows EC2 in the same VPC)

How to make RDS publicly accessible

Not recommended for production RDS instances or RDS carrying sensitive data.

Navigate to RDS console and respective database
Select Modify from the action menu for that particular database
Goto Connectivity section and expand Additional configuration
Here choose the radio button against Publicly accessible and Apply the changes

RDS connection using ‘Password authentication’ via SQL client

Once you sort out RDS connectivity issues, go back to the SQL client and try Test again. Now you should be able to succeed.

Now, click on Save to save this configuration in SQL client. And then hit Connect to connect to your RDS database.

After connecting, you can see schema on the left-hand side and a command box on the right-hand side to execute commands on the database.

Command outputs or messages will be shown in the blank space below the command text area. You will be able to download outputs in JSON, CSV format or even copy them directly.

That’s all! You can use this light weight SQL client to get started with RDS immediately.

Preparing for Hashicorp Certified Terraform Associate Exam

Leave a reply

A quick article that helps you preparing for Hashicorp Certified Terraform Associate Exam

In this quick post, I would like to share some of the resources that help you clear the terraform associate exam. Feel free to add resources you know in the comments section, which may help fellow readers.

The terraform associate exam is designed to test the candidate’s readiness towards IaC, i.e. Infrastructure as code. IaC concepts, terraform CLI hands-on (a lot of it) and knowledge on terraform’s paid offerings through Cloud or Enterprise should get you through this exam. It’s a practitioner level exam, so it shouldn’t be hard to beat if you have IaC and cloud background.

You must have researched already about the exam on its official page, but here are quick facts for your reference.

Approx 57-60 questions
Duration 1 hour
Validity: 2 years
Cost: 70.50 USD plus local taxes
Type: Multiple choice
It’s online proctored
Official study guide
Sample questions

Topics to study

I suggest you have good hands-on with terraform CLI before taking this exam. It will help you cover the majority of topics, and you don’t have to learn them during preparation. That leaves you with minimal topics to prepare for actual certification.

Hashicorp’s study guide is the best resource to follow along for preparation. Let me quickly list down a couple of topics you should not miss during preparation –

IaC concepts
- Traditional infra provisioning v/s IaC
Terraform basic workflow
- Write, plan and apply.
Different types of blocks in terraform code
Terraform CLI commands (a huge list of them!)
Terraform Modules, functions, state files
- At least go through all functions once.
- Lots of hands-on to understand how modules works
- State management (a big topic!)
Debugging and variables
- Different ways to handle variables
- Debugging levels, ways to set them, logging in files
Detailed understanding of Terraform cloud and enterprise
- Free and paid offerings in each type
- Sentinal, workspaces, remote runs etc. understanding
- Clustering, OS availability in each type

Resources for preparation

Assorted list of online resources you can leverage to follow along your preparation journey.

Official study guide
HashiCorp Certified Terraform Associate by A Cloud Guru
Learning terraform by Linkedin Learning
Terraform study guide by Ned Bellavance

I am linking here my own last day revision notes as well that I prepared during my certification preparation.

Practice tests

Here is a list of practice tests you can take online before going in for an actual exam. It will test the understanding of your topic and concretes your decision for exam booking.

HashiCorp Certified: Terraform Associate Practice Exam 2021 on Udemy (Check coupons on author’s Github here. I missed grabbing one 🙁 )
HashiCorp Certified Terraform Associate by Whizlabs. There is a small free test available too.
250 Practice Questions For Terraform Associate Certification on Medium. This list covers a lot of ground.

That’s all I have to share. All the best!