A step by step guide to install ConsoleMe on Ubuntu Linux machine
ConsoleMe is an open-source web service published by Netflix. It is designed to make life easy for end-users and cloud administrators. Using ConsoleMe, cloud administrators can manage IAM permissions/credentials for IAM roles, S3 buckets, SQS queues, and SNS topics across multiple AWS accounts from a single interface. It also provides CLI called weep for AWS credentials management. That’s a fair introduction if you are not aware of the tool. Next, let’s get into the installation part.
ConsoleMe offers docker and local installs. We will walk you through local install in this article.
Pre-requisite:
- A machine running Ubuntu 19.04+ with root access. I used Ubuntu 20.04 LTS x86.
- Active and working package manager subscription to install packages
- Storage requirement: 2GB of disk space
- An AWS user/role for consoleme service with appropriate permissions
- AWS access keys for above user if you are not using roles. I used keys (steps below)
Installation
We are installing ConsoleMe in /consoleme directory. If you want to install in another location, make the necessary changes in the commands below. Let me give you a list of commands you need to run as root –
apt-get update
apt-get install build-essential libxml2-dev libxmlsec1 libxmlsec1-dev libxmlsec1-openssl musl-dev libcurl4-nss-dev python3-dev pkg-config python3.8-venv awscli docker-compose -y
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash
apt-get install -y nodejs
npm install yarn -g
cd /
git clone https://github.com/Netflix/consoleme.git
cd consoleme
docker-compose -f docker-compose-dependencies.yaml up -d
Here, the first few commands are installing all the dependencies and related software/tools. Then, we are cloning the GitHub repo of the tool in /consoleme and lastly, we are running two containers.
These are Redis and dynamodb containers that ConsoleMe leverages for caching and aggregating the AWS accounts information. You can make use of AWS Redis and dynamodb table services, but for now, we will run these containers locally so that ConsoleMe will talk to them rather than AWS services.
I am avoiding putting up console outputs for frequently used commands like package installations etc., here.
Make sure both containers are up and running before proceeding to the next step –
root@kerneltalks:/consoleme# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5333cdee2202 cnadiminti/dynamodb-local "java -jar DynamoDBL…" 10 seconds ago Up 4 seconds 8000/tcp, 0.0.0.0:8005->8005/tcp consoleme-dynamodb
19ac354c3d70 redis:alpine "docker-entrypoint.s…" 10 seconds ago Up 4 seconds 0.0.0.0:6379->6379/tcp consoleme-redis
4cf931d38652 aaronshaf/dynamodb-admin:latest "node bin/dynamodb-a…" 10 seconds ago Up 4 seconds 0.0.0.0:8001->8001/tcp consoleme-dynamodb-admin
Now, you need to prepare the machine to talk with AWS for fetching account details in the upcoming install steps. Ensure that you have set up account and permissions perfectly in IAM (mentioned in the pre-requisite above) to avoid any issues. You can do that by configuring AWS profile –
root@kerneltalks:/consoleme# aws configure
AWS Access Key ID [None]: AKIAQX3STVKIYRO36XEC
AWS Secret Access Key [None]: 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
Default region name [None]:
Default output format [None]:
Lastly, create a new python environment and run the final install step. This will take a while to complete since at the end of make install command, it also fetches and caches the AWS account details in the local Redis cache –
python3 -m venv env
. env/bin/activate
make install
After successful installation, you should be able to start the application.
Running ConsoleMe
On a current shell, you can run the ConsoleMe with the command. If you are in another shell, activate the python environment again –
(env) root@kerneltalks:/consoleme# python consoleme/__main__.py
{"asctime": "2021-07-25T08:32:16Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "jwt.py", "funcName": "<module>", "levelname": "ERROR", "lineno": 14, "module": "jwt", "threadName": "MainThread", "message": "Configuration key `jwt.secret` is not set. Setting a random secret", "eventTime": "2021-07-25T01:32:16.286230-07:00", "hostname": "kerneltalks", "timestamp": "2021-07-25T08:32:16Z+0000"}
2021-07-25 08:32:17,322 - DEBUG - root - [constants.py:39 - <module>() ] - Leveraging the bundled IAM Definition.
2021-07-25 08:32:17,322 - INFO - root - [iam_data.py:10 - <module>() ] - Leveraging the IAM definition at /consoleme/env/lib/python3.8/site-packages/policy_sentry/shared/data/iam-definition.json
2021-07-25 08:32:17,824 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
2021-07-25 08:32:17,859 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
{"asctime": "2021-07-25T08:32:18Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "__main__.py", "funcName": "init", "levelname": "DEBUG", "lineno": 57, "module": "__main__", "threadName": "MainThread", "message": "Server started", "eventTime": "2021-07-25T01:32:16.286230-07:00", "hostname": "kerneltalks", "timestamp": "2021-07-25T08:32:18Z+0000"}
But, it will exit out when you terminate the command or shell. It’s safe to run it in the background or, even better, run it as a Linux service. For running ConsoleMe as a service, create below two files –
File /usr/bin/consoleme_start.sh
#!/bin/bash
. env/bin/activate
python consoleme/__main__.py
File /etc/systemd/system/consoleme.service
[Unit]
Description=Run consoleme service.
[Service]
Type=simple
User=root
WorkingDirectory=/consoleme
ExecStart=/usr/bin/consoleme_start.sh
[Install]
WantedBy=multi-user.target
Assign executable permissions to
chmod +x /usr/bin/consoleme_start.sh
Enable and start the service
root@kerneltalks:/consoleme# systemctl enable consoleme
Created symlink /etc/systemd/system/multi-user.target.wants/consoleme.service → /etc/systemd/system/consoleme.service.
root@kerneltalks:/consoleme# systemctl start consoleme
root@kerneltalks:/consoleme# systemctl status consoleme
● consoleme.service - Run consoleme service.
Loaded: loaded (/etc/systemd/system/consoleme.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2021-07-25 08:35:52 UTC; 7s ago
Main PID: 14775 (consoleme_start)
Tasks: 5 (limit: 4706)
Memory: 159.7M
CGroup: /system.slice/consoleme.service
├─14775 /bin/bash /usr/bin/consoleme_start.sh
└─14776 python consoleme/__main__.py
Jul 25 08:35:52 kerneltalks systemd[1]: Started Run consoleme service..
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: {"asctime": "2021-07-25T08:35:53Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "jwt.py", "funcName": "<module>", "levelname": "ERROR", "lineno": 14, "m>
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:53,954 - DEBUG - root - [constants.py:39 - <module>() ] - Leveraging the bundled IAM Definition.
Jul 25 08:35:53 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:53,955 - INFO - root - [iam_data.py:10 - <module>() ] - Leveraging the IAM definition at /consoleme/env/lib/python3.8/site-packages/policy_sentry/shared/data/i>
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:54,354 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: 2021-07-25 08:35:54,361 - DEBUG - git.cmd - [cmd.py:817 - execute() ] - Popen(['git', 'version'], cwd=/consoleme, universal_newlines=False, shell=None, istream=None)
Jul 25 08:35:54 kerneltalks consoleme_start.sh[14776]: {"asctime": "2021-07-25T08:35:54Z+0000", "name": "consoleme", "processName": "MainProcess", "filename": "__main__.py", "funcName": "init", "levelname": "DEBUG", "lineno": 57, ">
ConsoleMe GUI
Now that your console service is running, you should load its GUI on a web browser. The service listens on the 8081 port, so you need to navigate the server address with port 8081. Make sure the security group is allowing 8081 traffic if you are installing on EC2.
At this point, ConsoleMe is running with the default open example configuration. It’s very well highlighted on the web app as a warning. It would be best if you were editing this configuration to make your ConsoleMe more secure. ConsoleMe recommends Application Load Balancer authentication for securing your web app GUI. Refer to our next article on how to secure the ConsoleMe web app using ALB authentication.