Quick revision on topics AWS SWF, Beanstalk, EMR, Cloudfomation before appearing AWS Certified Solutions Architect – Associate exam.
This article notes down a few important points about AWS (Amazon Web Services) SWF, Beanstalk, EMR, Cloudfomation. This can be helpful in last-minute revision before appearing for the AWS Certified Solutions Architect – Associate level certification exam.
This is forth part of AWS CSA revision series. Rest of the series listed below :
In this article we are checking out key points about SWF (Simple Work Flow), Beanstalk (App deployment Service), EMR (Elastic MAp Reduce), Cloudfomation (Infrastructure as code).
Quick revision on topics AWS CloudFront, SNS, SQS before appearing AWS Certified Solutions Architect – Associate exam.
CloudFront, SNS, SQS revision!
This article notes down a few important points about AWS (Amazon Web Services) CloudFront, SNS, and SQS. This can be helpful in last-minute revision before appearing for the AWS Certified Solutions Architect – Associate level certification exam.
This is third part of AWS CSA revision series. Rest of the series listed below :
In this article, we are checking out key points about CloudFront(CDN Content Delivery Network), SNS (Simple Notification Service), and SQS (Simple Queue Service).
Origin can be S3 bucket or CNAME of Elastic Load Balancer ELB
S3 bucket as the origin. URL will be bucket_name.s3-reagion.cloudfront.net
Private content sharing with signed URL with an expiration time limit
To serve a new object version, create a new distribution, or create invalidation of the old objects. Since invalidation costs, creating new distribution always helps.
Limits :
1,00,000 Requests per second per distribution
200 distributions per account
40Gbps speed per distribution
25 origins per distribution
20 GB max file size to serve
By default, object expiration is 24 hours. The minimum TTL is 0.
Amazon SNS
The latest addition to SNS is Lambda
SNS has two clients: Publishers and subscribers
Publishers communicate with subscribers by sending messages to the topic.
Protocol supported :
HTTP
HTTPS
SMS
email
email-JSON
Amazon SQS
AWS Lambda
SNS Topic of the same name can be created after 30-60 seconds the previous topic deleted.
Amazon SQS
The default visibility timeout is 30 secs. The maximum is 12 hours.
Mainly used to decouple your application
The default period message stays in queue is 4 days. Min-Max periods are 1 min to 2 weeks.
The maximum SQS message size is 256KB.
Supports an unlimited number of queues and unlimited messages per queue.
Different ways to find the MAC address of LAN card in HPUX. Learn how to use lanscan, lanadmin, print_manifest, SAM to check MAC.
MAC addresses also known as station addresses can be found physically on LAN cards which are mostly PCI cards on your HP server. Obviously being hardware, it’s not always feasible to open up just to get MAC address! Another way is to get these details from the OS command. You can use lanscan,lanadmin, sam, print_manifest command to get the MAC address of the LAN card in HPUX.
First, you need to get a LAN number on which your expected IP is configured. You can use netstat -nvr to check all IP configured on the system and their respective LAN number.
Look at the interface column to get lanX number. For example, we will try to get the MAC of lan1 interface.
lanscan command
lanscan command without any argument will give you station address i.e. MAC addresses of all available LAN on the system.
# /usr/sbin/lanscan
Hardware Station Crd Hdw Net-Interface NM MAC HP-DLPI DLPI
Path Address In# State NamePPA ID Type Support Mjr#
0/1/2/0 0x001A3B08C4A0 0 UP lan0 snap0 1 ETHER Yes 119
0/1/2/1 0x001A3B08C4A1 1 UP lan1 snap1 2 ETHER Yes 119
Look station address and column and check the value against lan1! lan1 has MAC of 0x001A3B08C4A1.
lanadmin command
This is not straight forward as lanscan command. After issuing lanadmin command you will be presented with lanadmin console prompt where you can use lanadmin commands. Example below.
# /usr/sbin/lanadmin
LOCAL AREA NETWORK ONLINE ADMINISTRATION, Version 1.0
Mon, Apr 17,2017 18:10:09
Copyright 1994 Hewlett Packard Company.
All rights are reserved.
Test Selection mode.
lan = LAN Interface Administration
menu = Display this menu
quit = Terminate the Administration
terse = Do not display command menu
verbose = Display command menu
Enter command: lan
Here type command lan. You will be greeted with the LAN interface mode prompt like below.
LAN Interface test mode. LAN Interface PPA Number = 0
clear = Clear statistics registers
display = Display LAN Interface status and statistics registers
end = End LAN Interface Administration, return to Test Selection
menu = Display this menu
ppa = PPA Number of the LAN Interface
quit = Terminate the Administration, return to shell
reset = Reset LAN Interface to execute its selftest
specific = Go to Driver specific menu
Enter command: ppa
Enter command ppa and change your number to 1 since we are checking lan1 in our example. Default is set to lan0
Enter command: ppa
Enter PPA Number. Currently 0: 1
LAN Interface test mode. LAN Interface PPA Number = 1
Once LAN interface PPA changed to 1 hit command display and you will be shown all details of that lan card including station address!
Enter command: display
LAN INTERFACE STATUS DISPLAY
Mon, Apr 17,2017 18:10:26
PPA Number = 1
Description = lan1 HP PCI-X 1000Base-T Release PHNE_36237 B.11.11.15
Type (value) = ethernet-csmacd(6)
MTU Size = 1500
Speed = 1000000000
Station Address = 0x1a3b08c4a1
Administration Status (value) = up(1)
Operation Status (value) = up(1)
Last Change = 185
Inbound Octets = 1362884960
Inbound Unicast Packets = 1309204600
----- output clipped -----
Here you can pad two zeros in from of station address to make it perfect 12 alphanumeric MAC. Means 1a3b08c4a1 becomes 001a3b08c4a1.
Using SAM
You can even use SAM (text based GUI tool) to get these details. Go to,
SAM -> Networking and communications -> Network Interface Cards
Select your lan (in our case lan1) using a space bar (it will be highlighted). Then choose Actions from the menu bar to get details.
Using print_manifest
If you have Ignite installed on the server then you can try print_manifest command to get all system details. Those details also include MAC of all lan cards. The only issue is your LAN PPA number won’t be available here in output to match MAC with lan id.
Quick revision on topics AWS VPC, Route53, IAM before appearing AWS Certified Solutions Architect – Associate exam.
VPC, Route53, IAM revision!
This article notes down a few important points about AWS (Amazon Web Services) VPC, Route53, and IAM. This can be helpful in last-minute revision before appearing for the AWS Certified Solutions Architect – Associate level certification exam.
This is the second part of the AWS CSA revision series. Rest of the series listed below :
NACL (Network Access Control List) controls traffic security at the subnet level
Security groups control traffic security at the instance level
NACL is stateless (i.e. all traffic need to exclusively allow) while Security groups are stateful (i.e. response traffic is automatically allowed)
Only 1 Internet gateway per VPC is allowed.
VPC peering can be done between two AWS accounts or other VPS within the same region.
VPC peering is a direct network route between two VPC enabling sharing resources in different subnets.
Limits :
5 VPC per region
50 customer gateways per region
200 route table per region
50 entries per route table
5 elastic IP
5 security group per network interface
500 security groups per VPC
50 rules per security group
First 4 and last 1 IP of each subnet is reserved by AWS as below :
x.x.x.0: Network IP
x.x.x.1 : VPC router IP
x.x.x.2: For VPC DNS
x.x.x.3: For future use
x.x.x.255: Broadcast IP
Route 53
Can register domain, act as DNS, Check health of resources.
Port 53 used to serve request by DNS hence the name route 53!
Primarily TCP used to serve DNS request but if the response is more than 512 bytes it will use TCP.
Currently supported records :
A (address record)
AAAA (IPv6 address record)
CNAME (canonical name record)
MX (mail exchange record)
NAPTR (name authority pointer record)
NS (name server record)
PTR (pointer record)
SOA (start of authority record)
SPF (sender policy framework)
SRV (service locator)
TXT (text record)
Routing policies :
Simple routing: Single resource serving traffic
Weighted routing: Divert proportion wise traffic to multiple resources
Latency routing: Returns result with the lowest latency to requestor origin
Failover routing: Active-passive. One resource takes traffic when the other one is failed
Geolocation routing: Returns DNS queries based on the geolocation of the user
Limits :
500 hosted zones per AWS account
50 domains per AWS account
Ideal TTL values for CNAME to the existing domain are 24 hours and CNAM to S3 or ELB is 1 hour.
There is no default TTL for any record type in Route 53. You have to specify TTL for your records.
Weights can be assigned as integer 0 to 255. 0 means no weight i.e. don’t route to that record. The probability of routing to be done to a particular record equals to the weight of that record/Sum of all record weights.
IAM (Identity and Access Management)
Never use the root account for login. Create an admin user and use it for administrative tasks
Created users, groups and roles are global and available across all regions in the same AWS account
Prebuilt policy for :
Administrator – All access
Power-user – Everything administrator has except IAM management access
Read-only – Only view access (accounting purpose)
By default, the newly created user has normal deny on all AWS resources. Explicit allow will override normal deny.
Cross account roles can be defined. It assumes access of other users granted to another user.
The public key can be viewed in the account settings anytime. The private key visible only at the time of creation. If lost can not be retrieved and need to create fresh key pair to use.
Quick revision on topics AWS EC2, S3, RDS before appearing AWS Certified Solutions Architect – Associate exam.
EC2, S3, RDS revision!
This article notes down a few important points about AWS (Amazon Web Services) EC2, S3, and RDS. This can be helpful in last-minute revision before appearing for the AWS Certified Solutions Architect – Associate level certification exam.
This is first part of AWS CSA revision series. Rest of the series listed below :
List of all online Linux yum or apt package repositories, mirrors, download sites. Updated continuously whenever a new link is discovered
List of online resources for linux packages
This article aimed at listing all available mirrors, repositories, package download sites for Linux distros. Let us know broken/new links in comments and we will update this post accordingly.
Step by step Nginx installation on YUM or APT based Linux servers. Know about configuration files, log files, and directory listing in Nginx.
nginx installation in linux
Nginx is a popular web server and preferred over Apache. Many websites and blogs today run on Nginx webserver. In this article, we are going to see how to install Nginx in Linux.
# service nginx start (Redhat systems)
Starting nginx: [ OK ]
OR
# systemctl start nginx (RHEL7 & debian)
Starting nginx: [ OK ]
Verify service is running.
# service nginx status
nginx (pid 1689) is running...
OR
# systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2017-04-04 10:24:19 UTC; 2min 15s ago
Main PID: 6734 (nginx)
CGroup: /system.slice/nginx.service
├─6734 nginx: master process /usr/sbin/nginx -g daemon on; master_process on
└─6735 nginx: worker process
That’s it. Your installation is complete and your webserver is up and running.
Nginx server test
Now you can enter your server IP in the browser (http://x.x.x.x/) to check if your Nginx webserver is serving page or not. If you are getting a connection timed out, you may want to check firewall settings on your server to allow port 80.
You will be served with test page like below :
nginx test page on YUM based systems like Red Hat
nginx test page on APT based systems like Ubuntu
This confirms that your Nginx web server installation is successful and it’s ready for customization.
nginx configurations
The default server root directory is /usr/share/nginx/html. Files that are placed in there will be served on your web server. You can place your HTML files here to serve a static website! The test page you seen above is file index.html under /usr/share/nginx/html.This default directory can be changed under Nginx configuration file /etc/nginx/conf.d/default.conf.
nginx global configuration file is/etc/nginx/nginx.conf. In this conf file, you can set the user to run Nginx, its number of processes to fork, etc.
Default log files error.log (for errors) and access.log (logging accesses) are located under /var/log/nginx
Directory listing in nginx
If you place a few files or directories in root directory then you will not see a listing of them in your browser as we saw in Apache configuration. To enable this you need to add autoindex on; in config file after root directive. Look at my config below :
In the above config, I defined /tmp as root and turned on autoindex (highlighted lines). Restart Nginx services after saving the above changes. Now if I check my webpage (http://x.x.x.x/) it will list all files/directories under /tmp
Directory listing in nginx
You can see in above screenshot, /tmp directory listing is presented by Nginx. If you don’t put autoindex on then it will serve you 404 forbidden error!
Learn to configure the free, open-source, secure Let’s Encrypt SSL certificate on Apache webserver running on YUM based Linux server.
Lets Encrypt installation on Apache
What is Let’s Encrypt
Let’s Encrypt is free, open-source, and automatic SSL CA (Certificate Authority). Its managed by ISRG (Internet Security Research Group). SSL certificate always involved a cost which is recurring every year for renewal. Let’s encrypt aimed at open source and free SSL. This is an ideal choice for small websites, businesses which have less or no critical data on their websites and looking for SSL certificates.
If you are running a personal blog then SSL is essential for having a good search engine reputation. But before you dive into you need one dedicated IP for your domain name. If you are on shared hosting you are likely not having a dedicated IP. So for Let’s Encrypt SSL, you need to buy IP for your domain name.
Lets Encrypt SSL certificate
Let’s encrypt provides you fee SSL after your domain name validity which lasts for 3 months. You have to manually renew it every 3 months. The renewal process can be automatized too. Certbot is currently serving you deploying https on your server and configuring Let’s encrypt certs for you.
Before run into installation you should have these pre-requisite completed :
Now goto letsencrypt directory which is created in your present directory by clone command. Under that directory run below command :
# ./letsencrypt-auto certonly --standalone
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Please enter in your domain name(s) (comma and/or space separated) (Enter 'c'
to cancel):ktwebtest.ddns.net
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for ktwebtest.ddns.net
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0002_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0002_csr-certbot.pem
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/ktwebtest.ddns.net/fullchain.pem. Your cert
will expire on 2017-07-02. To obtain a new or tweaked version of
this certificate in the future, simply run letsencrypt-auto again.
To non-interactively renew *all* of your certificates, run
"letsencrypt-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
You can see above, after supplying domain name, Certbot creates a CSR file on its own, generates a key file, and fetch SSL certificate too. All paths are visible in the output. All files path are :
It also shows you when your certificate going to expire. And command you can use to renew your certificate. Now you can follow the tutorial of how to install SSL which you obtained in the above step.
You can even automatize this SSL installation steps by using command :
With this command, it will create key, CSR, fetch SSL, install SSL on your domain webserver!
Certificate renewal
You can renew certificate manually using :
# ./letsencrypt-auto renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log
-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/ktwebtest.ddns.net.conf
-------------------------------------------------------------------------------
Cert not yet due for renewal
The following certs are not due for renewal yet:
/etc/letsencrypt/live/ktwebtest.ddns.net/fullchain.pem (skipped)
No renewals were attempted.
As you can see, it will scan through all fetched let’s encrypt certificates on the server and their due date. If found due, those certificates will be renewed in no time!
If you want to renew certificate regardless of the due date then you can use force renew as below :
# ./letsencrypt-auto renew --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log
-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/ktwebtest.ddns.net.conf
-------------------------------------------------------------------------------
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for ktwebtest.ddns.net
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0003_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0003_csr-certbot.pem
-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/ktwebtest.ddns.net/fullchain.pem
-------------------------------------------------------------------------------
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/ktwebtest.ddns.net/fullchain.pem (success)
You can even schedule crontab with a little bit of scripting to have an automatic renewal of certificates.
Learn how to install EPEL repository (Extra Packages for Enterprise Linux) in YUM based Linux. It enables you to install open source packages.
EPEL repository installation
What is EPEL
EPEL stands for Extra Packages for Enterprise Linux. EPEL repository is helpful and rich with packages that are normally used/needed for commonly used Linux software. This repository is managed by the EPEL group. This repository contains all 100% free open source packages which may not available in the core
EPEL repos should be used in production with extra caution.
As you know EPEL group is the Fedora project, EPEL repository can be enabled for YUM based systems like RHEL, Oracle Linux, Fedora, CentOS, etc distributions.
How to configure EPEL
Adding EPEL repository can take a different route than adding https, FTP repositories in YUM. For EPFL, you need to download related RPM from link using any Linux downloaders. Choose correct distro and architecture while choosing a download link for the EPEL package. Or you can choose noarch rpm as I did below.
# yum install ./epel-release-latest-*.noarch.rpm
Loaded plugins: amazon-id, rhui-lb, security
Setting up Install Process
Examining ./epel-release-latest-6.noarch.rpm: epel-release-6-8.noarch
Marking ./epel-release-latest-6.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:6-8 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=============================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================
Installing:
epel-release noarch 6-8 /epel-release-latest-6.noarch 22 k
Transaction Summary
=============================================================================================================================================================
Install 1 Package(s)
Total size: 22 k
Installed size: 22 k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : epel-release-6-8.noarch 1/1
Verifying : epel-release-6-8.noarch 1/1
Installed:
epel-release.noarch 0:6-8
Complete!
Thats it. EPEL repository is accommodated by your YUM configuration now.
EPEL repository test
Let’s test the EPEL repository if it’s configured properly. You should see below the new repo in directory /etc/yum.repos.dThis is the first indication EPEL is configured.
# pwd
/etc/yum.repos.d
# ll
total 32
-rw-r--r--. 1 root root 957 Nov 4 2012 epel.repo
-rw-r--r--. 1 root root 1056 Nov 4 2012 epel-testing.repo
Now extract repolist from yum to verify configuration.
# yum repolist
Loaded plugins: amazon-id, rhui-lb, security
repo id repo name status
epel Extra Packages for Enterprise Linux 6 - x86_64 12,361
rhui-REGION-client-config-server-6 Red Hat Update Infrastructure 2.0 Client Configuration Server 6 0
rhui-REGION-rhel-server-releases Red Hat Enterprise Linux Server 6 (RPMs) 19,422
rhui-REGION-rhel-server-releases-optional Red Hat Enterprise Linux Server 6 Optional (RPMs) 11,057
rhui-REGION-rhel-server-rh-common Red Hat Enterprise Linux Server 6 RH Common (RPMs) 129
repolist: 48,031
You can see EPEL is configured and identified properly by YUM (highlighted line).
Step by step procedure to install and configure Apache webserver in Linux. It also includes troubleshooting steps for errors during install.
Apache webserver installation in Linux
In this tutorial, we are going to see how to install and configure Apache webserver in Linux. Apache is the most widely used web server that is developed and maintained by Apache Software Foundation. Its an open-source software and available for free to download and use.
What is webserver
The webserver is aimed to serve web pages to its clients. In our daily life websites, we visit from browsers that are hosted on webservers! Webserver commonly runs on port 80 (HTTP protocol) and 443 (https protocol, more secured). Communication between the web server and its client over https protocol is encrypted and hence secured.
Apache webserver
Apache webserver is an open-source web server developed by Apache Software Foundation. It is named as Apache HTTP Server Project and details can be viewed here. Apache webserver is also a part of the LAMP stack. Read more about LAMP here. Let’s see how to set up the Apache web server on the Linux box.
Error: configure: error: APR not found. Solution: Install apr and apr-utils packages (download here if you don’t have yum)
Error: configure: error: no acceptable C compiler found in $PATH Solution: Install GCC package
Error: configure: error: pcre-config for libpcre not found. Solution: Install pcre package (download here, unzip, /configure --prefix=/usr/local/pcre, make, make install)
Finally, you should be able to install it using : ./configure --prefix=/apache, make, make install (you can change /apache directory to diff where it will be installed)
Apache configuration
Post-installation you will have /var/www/html/ directory in which you should place your HTML pages which can be fetched from browser. httpd service will come to existence.
If you keep files and directories in/var/www/html/ then its listing will be shown in a browser like this example we saw in another post. We will put below HTML file in /var/www/html/ for test :
# cat test.html
<html>
<body>
<h4>This webserver is installed by kerneltalks.com</h4>
</body>
</html>
# service httpd status
httpd (pid 27920) is running...
Now, try to fetch that page from browser. Use your server IP as website address i.e. http://IP/test.html
Testing webserver
You will be greeted with such a page! Your webserver is up and running. Accepting and serving requests from browsers.
Apache config files
Apache configuration file located at /etc/httpd/conf/httpd.confIn this configuration file you can set your error file path, SSL certificate path, SSL cert key file path, port, IP (virtual hosts), access log path, etc. There is a lot more customization that can be done using this file but I stated only general tweaking parameters.
Apache log files can be found at :
/etc/httpd/logs/access_log: all access logs logged here.
/etc/httpd/logs/error_log: All errors logged here. They also include start-stop service messages.
How to start / stop Apache webserver
To start Apache webserver
# service httpd start
# service apache2 start #If you are running Apache2
To stop Apache webserver
# service httpd stop
# service apache2 stop #If you are running Apache2
How to load Apache2 modules
One of the common issues found is when the PHP module is not loaded into Apache webserver, .php links on webserver will download files in the browser instead of displaying them. To correct this you need to load the PHP module in Apache.
To load PHP or any other module in Apache, you need to add it in file /etc/sysconfig/apache2 in below line :
APACHE_MODULES="actions alias auth_basic authn_file authz_host authz_groupfile authz_core authz_user autoindex cgi dir env expires include log_config mime negotiation setenvif ssl socache_shmcb userdir reqtimeout authn_core php5"
Here you can see we are loading the php5 module in our Apache2 webserver. Then you need to restart Apache2 webserver. Once done, check if the PHP module is loaded or not. Use command apache2ctl -M to check loaded modules in Apache.
What is difference between Server Root and Document Root in Apache ?
Server root is where Apache executable binaries reside rather its an installation path for Apache. Whereas document root is where website pages reside i.e. from where webserver loads web pages. So, you can change Document Root if you don’t want to use default /srv/www/htdocs directory. You can change it to any directory where you keep your website pages.
