Monthly Archives: February 2017

How to configure telnet server in Linux

Step by step guide to configure telnet server on Linux. Generally, SSH is preferred over telnet since its more secure, and hence telnet is not available out of the box.

Telnet (TELetype NETwork) is a network protocol used on the Internet or local area networks. It uses a virtual terminal connection and provides bidirectional interactive text-oriented communication. One can use telnet to log in remotely to another system locally or over the internet.

Caution: telnet open un-encrypted communication channel to your machine over the network. Avoid using telnet and opt SSH for connectivity.

SSH i.e. Secure SHell is more secure than telnet. Hence, all Linux Unix servers use SSH for user connectivity. Even many installations don’t have telnet available out of the box.

This tutorial walks you through the process to configure telnet on your Linux machine but SSH is always advisable for server connectivity than telnet for being more secure. 

telnet server configuration :

Step 1:

As I said above, many installations don’t have telnet out of the box. You need to install the telnet package as a first step. Install telnet, telnet-server, and xinetd packages.

Use apt-get install telnetd for debian, ubuntu distro.

# yum install telnet telnet-server xinetd
Loaded plugins: amazon-id, rhui-lb, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package telnet.x86_64 1:0.17-48.el6 will be installed
---> Package telnet-server.x86_64 1:0.17-48.el6 will be installed
---> Package xinetd.x86_64 2:2.3.14-40.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================
 Package                           Arch                       Version                             Repository                                            Size
=============================================================================================================================================================
Installing:
 telnet                            x86_64                     1:0.17-48.el6                       rhui-REGION-rhel-server-releases                      58 k
 telnet-server                     x86_64                     1:0.17-48.el6                       rhui-REGION-rhel-server-releases                      37 k
 xinetd                            x86_64                     2:2.3.14-40.el6                     rhui-REGION-rhel-server-releases                     122 k

Transaction Summary
=============================================================================================================================================================
Install       3 Package(s)

Total download size: 217 k
Installed size: 423 k
Is this ok [y/N]: y
Downloading Packages:
(1/3): telnet-0.17-48.el6.x86_64.rpm                                                                                                  |  58 kB     00:00
(2/3): telnet-server-0.17-48.el6.x86_64.rpm                                                                                           |  37 kB     00:00
(3/3): xinetd-2.3.14-40.el6.x86_64.rpm                                                                                                | 122 kB     00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                        335 kB/s | 217 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 2:xinetd-2.3.14-40.el6.x86_64                                                                                                             1/3
  Installing : 1:telnet-server-0.17-48.el6.x86_64                                                                                                        2/3
  Installing : 1:telnet-0.17-48.el6.x86_64                                                                                                               3/3
  Verifying  : 1:telnet-server-0.17-48.el6.x86_64                                                                                                        1/3
  Verifying  : 1:telnet-0.17-48.el6.x86_64                                                                                                               2/3
  Verifying  : 2:xinetd-2.3.14-40.el6.x86_64                                                                                                             3/3

Installed:
  telnet.x86_64 1:0.17-48.el6                      telnet-server.x86_64 1:0.17-48.el6                      xinetd.x86_64 2:2.3.14-40.el6

Complete!

Step 2:

Set services to start on boot.

# chkconfig telnet on
# chkconfig  xinetd  on

Restart services. inetd in case of Debian.

# service xinetd restart
Stopping xinetd:                                           [FAILED]
Starting xinetd:                                           [  OK  ]

Verify service is listening on your server.

# netstat -lptu|grep telnet
tcp        0      0 *:telnet                    *:*                         LISTEN      1618/xinetd

# lsof -i |grep telnet
xinetd    1618     root    5u  IPv6  13908      0t0  TCP *:telnet (LISTEN)

Step 3:

Connect your server from a windows machine with the telnet protocol. Open a command prompt and type telnet IP-address. You will be greeted with a login prompt and will be able to login with an existing user.

If you are not able to connect via telnet make sure there are no firewalls are blocking communication between your Windows machine and telnet server for port 23 TCP.

How to list open ports on Linux/Unix server

Learn to list all open ports on Linux or Unix system. Also, view associated processes with them. A handy tip to troubleshoot service connectivity issues.

Many times in the life of sysadmin, you need to check which all ports open on your system. Sometimes you need to check if a particular port is listening on the server or not. If a particular service is communicating on a configured port or not. If a particular port has established connection or not.

All these things can be analysed with below commands.

netstat command :

Obviously first command is none other than netstat command. Use netstat with 4 options :

  • -a : Shows all sockets
  • -p : Show related PID
  • -t : TCP
  • -u : UDP
# netstat -ptau
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 *:58573                     *:*                         LISTEN      1013/rpc.statd
tcp        0      0 *:sunrpc                    *:*                         LISTEN      991/rpcbind
tcp        0      0 *:ssh                       *:*                         LISTEN      1208/sshd
tcp        0      0 localhost:ipp               *:*                         LISTEN      1069/cupsd
tcp        0      0 localhost:smtp              *:*                         LISTEN      1287/master
tcp        0      0 ip-12-31-28-246.ap-:telnet 112.197.214.169:49648       ESTABLISHED 3213/in.telnetd
tcp        0      0 ip-12-31-28-246.ap-:telnet 200-163-187-49.scrce2:53440 ESTABLISHED 3215/in.telnetd
tcp        0      0 ip-12-31-28-246.ap-sou:ssh 59.182.17:49413             ESTABLISHED 1441/sshd
tcp        0    288 ip-12-31-28-246.ap-sou:ssh 59.182.17:50729             ESTABLISHED 1694/sshd
tcp        0      0 *:sunrpc                    *:*                         LISTEN      991/rpcbind
tcp        0      0 *:ssh                       *:*                         LISTEN      1208/sshd
tcp        0      0 *:telnet                    *:*                         LISTEN      1618/xinetd
tcp        0      0 localhost:ipp               *:*                         LISTEN      1069/cupsd
tcp        0      0 localhost:smtp              *:*                         LISTEN      1287/master
tcp        0      0 *:56954                     *:*                         LISTEN      1013/rpc.statd
udp        0      0 localhost:766               *:*                                     1013/rpc.statd
udp        0      0 *:39730                     *:*                                     1013/rpc.statd
udp        0      0 *:bootpc                    *:*                                     884/dhclient
udp        0      0 *:netrcs                    *:*                                     991/rpcbind
udp        0      0 *:sunrpc                    *:*                                     991/rpcbind
udp        0      0 *:ipp                       *:*                                     1069/cupsd
udp        0      0 *:60991                     *:*                                     1013/rpc.statd
udp        0      0 *:netrcs                    *:*                                     991/rpcbind
udp        0      0 *:sunrpc                    *:*                                     991/rpcbind

In above output you can see :

  • The first column is a protocol
  • Fourth column local address includes local IP, port, service
  • Fifth column destination IP, port, etc
  • The sixth column is the current state
  • The last column is PID and process name which owns that socket

Another way is to use netstat with an option:

# netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 0.0.0.0:8001                0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:9633              0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:7937                0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:9634              0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:7938                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9443                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9444                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9060                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9061                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:14502               0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:14503               0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:8359                0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:199               0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:810                 0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9100                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9101                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9133                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:8080                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:8880                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:6000                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:8881                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9043                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9044                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:21                  0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9080                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:9081                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:2809                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:2810                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:6556                0.0.0.0:*                   LISTEN
tcp        0      0 10.10.5.160:42124         10.10.5.199:35994         ESTABLISHED
tcp        0      0 10.10.5.160:42376         10.10.5.199:39154         ESTABLISHED
tcp        0      0 10.10.5.160:8881          10.10.5.160:55117         TIME_WAIT
tcp        0      0 10.10.5.160:44367         10.10.5.199:48181         ESTABLISHED
tcp        0      0 10.10.5.160:36671         10.10.5.199:58137         ESTABLISHED
tcp        1      0 10.10.5.160:56253         10.10.5.160:9081          CLOSE_WAIT
tcp        0      0 10.10.5.160:57168         10.10.5.52:1521           ESTABLISHED
tcp        0      0 10.10.5.160:8880          10.10.5.160:55035         TIME_WAIT
tcp        0      0 10.10.5.160:6556          10.100.22.173:51544         ESTABLISHED
udp        0      0 0.0.0.0:7938                0.0.0.0:*
udp        0      0 0.0.0.0:37909               0.0.0.0:*
udp        0      0 0.0.0.0:161                 0.0.0.0:*
udp        0      0 0.0.0.0:804                 0.0.0.0:*
udp        0      0 0.0.0.0:807                 0.0.0.0:*
udp        0      0 0.0.0.0:177                 0.0.0.0:*
udp        0      0 0.0.0.0:5353                0.0.0.0:*
udp        0      0 0.0.0.0:111                 0.0.0.0:*
udp        0      0 203.127.98.134:123          0.0.0.0:*
udp        0      0 10.10.5.160:123           0.0.0.0:*
udp        0      0 127.0.0.1:123               0.0.0.0:*
udp        0      0 0.0.0.0:123                 0.0.0.0:*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     7690   @ISCSIADM_ABSTRACT_NAMESPACE
unix  2      [ ACC ]     STREAM     LISTENING     11363  @/tmp/fam-root-
unix  2      [ ACC ]     STREAM     LISTENING     11003  /var/run/avahi-daemon/socket
unix  2      [ ACC ]     STREAM     LISTENING     8701   @/var/run/hald/dbus-RL64SjEdUd
unix  2      [ ACC ]     STREAM     LISTENING     247409367 /jas/HTTPServer/logs/cgisock.14073
unix  2      [ ACC ]     STREAM     LISTENING     247409370 /jas/HTTPServer/logs/siddport
unix  2      [ ACC ]     STREAM     LISTENING     11221  /tmp/.gdm_socket
unix  14     [ ]         DGRAM                    8008   /dev/log
unix  2      [ ACC ]     STREAM     LISTENING     7668   @ISCSID_UIP_ABSTRACT_NAMESPACE
unix  2      [ ]         DGRAM                    1797   @/org/kernel/udev/udevd
unix  2      [ ]         DGRAM                    8709   @/org/freedesktop/hal/udev_event
unix  2      [ ACC ]     STREAM     LISTENING     8700   @/var/run/hald/dbus-mO28j2Fpoe
unix  2      [ ACC ]     STREAM     LISTENING     7963   /var/run/audispd_events
unix  2      [ ACC ]     STREAM     LISTENING     10761  /tmp/.font-unix/fs7100
unix  2      [ ACC ]     STREAM     LISTENING     10709  /dev/gpmctl
unix  2      [ ACC ]     STREAM     LISTENING     8608   /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     8675   /var/run/acpid.socket
unix  2      [ ACC ]     STREAM     LISTENING     11248  /tmp/.X11-unix/X0
unix  2      [ ]         DGRAM                    322339652
unix  2      [ ]         STREAM     CONNECTED     284330078
unix  2      [ ]         STREAM     CONNECTED     83187468
unix  2      [ ]         DGRAM                    11482214
unix  2      [ ]         STREAM     CONNECTED     98040
unix  3      [ ]         STREAM     CONNECTED     11366  @/tmp/fam-root-
unix  3      [ ]         STREAM     CONNECTED     11365
unix  3      [ ]         STREAM     CONNECTED     11356  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     11355
unix  3      [ ]         STREAM     CONNECTED     11333  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     11332
unix  3      [ ]         STREAM     CONNECTED     11306  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     11305
unix  3      [ ]         STREAM     CONNECTED     11283  /tmp/.font-unix/fs7100
unix  3      [ ]         STREAM     CONNECTED     11282
unix  2      [ ]         DGRAM                    11254
unix  3      [ ]         STREAM     CONNECTED     11286  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     11253
unix  3      [ ]         STREAM     CONNECTED     11252  /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     11251
unix  3      [ ]         STREAM     CONNECTED     11006  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     11005
unix  3      [ ]         STREAM     CONNECTED     11000
unix  3      [ ]         STREAM     CONNECTED     10999
unix  2      [ ]         DGRAM                    10997
unix  2      [ ]         DGRAM                    10851
unix  2      [ ]         DGRAM                    10686
unix  2      [ ]         DGRAM                    10661
unix  2      [ ]         DGRAM                    10594
unix  2      [ ]         DGRAM                    10569
unix  2      [ ]         DGRAM                    10453
unix  3      [ ]         STREAM     CONNECTED     10344  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10343
unix  3      [ ]         STREAM     CONNECTED     10298  @/var/run/hald/dbus-mO28j2Fpoe
unix  3      [ ]         STREAM     CONNECTED     10297
unix  3      [ ]         STREAM     CONNECTED     10117  @/var/run/hald/dbus-mO28j2Fpoe
unix  3      [ ]         STREAM     CONNECTED     10113
unix  3      [ ]         STREAM     CONNECTED     10096  /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     10093
unix  3      [ ]         STREAM     CONNECTED     10063  @/var/run/hald/dbus-mO28j2Fpoe
unix  3      [ ]         STREAM     CONNECTED     10059
unix  3      [ ]         STREAM     CONNECTED     8704   @/var/run/hald/dbus-RL64SjEdUd
unix  3      [ ]         STREAM     CONNECTED     8703
unix  3      [ ]         STREAM     CONNECTED     8612
unix  3      [ ]         STREAM     CONNECTED     8611
unix  3      [ ]         STREAM     CONNECTED     8466
unix  3      [ ]         STREAM     CONNECTED     8465
unix  2      [ ]         DGRAM                    8343
unix  2      [ ]         DGRAM                    8016
unix  3      [ ]         STREAM     CONNECTED     7954
unix  3      [ ]         STREAM     CONNECTED     7953

lsof command :

Using lsof command also you can trace current open ports on the system. lsof mainly lists open files. Since sockets are treated as network files at the kernel level and they are treated as open when communicating, sockets can be listed using lsof!

lsof has -i option specifically to list network files.

# lsof -i
COMMAND    PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dhclient   884     root    5u  IPv4  10791      0t0  UDP *:bootpc
rpcbind    991      rpc    6u  IPv4  11109      0t0  UDP *:sunrpc
rpcbind    991      rpc    7u  IPv4  11111      0t0  UDP *:netrcs
rpcbind    991      rpc    8u  IPv4  11112      0t0  TCP *:sunrpc (LISTEN)
rpcbind    991      rpc    9u  IPv6  11114      0t0  UDP *:sunrpc
rpcbind    991      rpc   10u  IPv6  11116      0t0  UDP *:netrcs
rpcbind    991      rpc   11u  IPv6  11117      0t0  TCP *:sunrpc (LISTEN)
rpc.statd 1013  rpcuser    6u  IPv4  11206      0t0  UDP localhost:766
rpc.statd 1013  rpcuser    7u  IPv4  11210      0t0  UDP *:39730
rpc.statd 1013  rpcuser    8u  IPv4  11214      0t0  TCP *:58573 (LISTEN)
rpc.statd 1013  rpcuser    9u  IPv6  11218      0t0  UDP *:60991
rpc.statd 1013  rpcuser   10u  IPv6  11222      0t0  TCP *:56954 (LISTEN)
cupsd     1069     root    6u  IPv6  11412      0t0  TCP localhost:ipp (LISTEN)
cupsd     1069     root    7u  IPv4  11413      0t0  TCP localhost:ipp (LISTEN)
cupsd     1069     root    9u  IPv4  11416      0t0  UDP *:ipp
sshd      1208     root    3u  IPv4  11960      0t0  TCP *:ssh (LISTEN)
sshd      1208     root    4u  IPv6  11962      0t0  TCP *:ssh (LISTEN)
master    1287     root   11u  IPv4  12158      0t0  TCP localhost:smtp (LISTEN)
master    1287     root   12u  IPv6  12160      0t0  TCP localhost:smtp (LISTEN)
sshd      1441     root    3r  IPv4  12964      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:ssh-59.184.179.68:49413 (ESTABLISHED)
sshd      1444 ec2-user    3u  IPv4  12964      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:ssh-59.184.179.68:49413 (ESTABLISHED)
xinetd    1618     root    5u  IPv6  13908      0t0  TCP *:telnet (LISTEN)
sshd      1694     root    3r  IPv4  14812      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:ssh-59.184.179.68:50729 (ESTABLISHED)
sshd      1697 ec2-user    3u  IPv4  14812      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:ssh-59.184.179.68:50729 (ESTABLISHED)
in.telnet 3420     root    0u  IPv4  35294      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->5ec3c900.skybroadband.com:39192 (ESTABLISHED)
in.telnet 3420     root    1u  IPv4  35294      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->5ec3c900.skybroadband.com:39192 (ESTABLISHED)
in.telnet 3420     root    2u  IPv4  35294      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->5ec3c900.skybroadband.com:39192 (ESTABLISHED)
in.telnet 3422     root    0u  IPv4  35326      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->188-24-133-29.dynamic.brasov.rdsnet.ro:apwi-rxserver (ESTABLISHED)
in.telnet 3422     root    1u  IPv4  35326      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->188-24-133-29.dynamic.brasov.rdsnet.ro:apwi-rxserver (ESTABLISHED)
in.telnet 3422     root    2u  IPv4  35326      0t0  TCP ip-12-31-28-246.ap-south-1.compute.internal:telnet->188-24-133-29.dynamic.brasov.rdsnet.ro:apwi-rxserver (ESTABLISHED)

In the above output, you can see which command being run by which user using which socket and state of the port at the end!

If you have any other trick to list open ports on the system, please let us know in the comments. We will add it to this post.

The complete guide: logrotate utility on Linux

Learn everything about logrotate utility in Linux. This post explains what is it, why to use it, and its configuration steps.

What is logrotate?

First of all, but an obvious introduction to logrotate tool. It is a utility which rotates, compress, purge, email logs once they match specified criteria like size, an age defined in the configuration. In short, its an automated log management tool.

It offers service (log specific) policies that can be set and it will handle logs accordingly. Conditions to trigger logrotate can be set on a size basis or time basis like daily, weekly, or monthly.

Why to rotate logs ?

Another obvious question is why to rotate logs? Logs are files that grow over time. Since they keep on filling over time. If you don’t maintain them they will end up filling your mount point which you never want to see in production!

Means, to save your disk space logs should be rotated. Logs are always useful for troubleshooting but yes you don’t need pretty old logs keeping your disk space on toss!  Logs can be purged, compressed, or moved manually. But to save your time from manual tasks you can automate log management using this tool.

Logrotate Installation:

Let’s walk through logrotate configuration. How to install it, how to configure it as per your requirement etc.

First of all, you need to install logrotate utility on your machine like below (Red Hat, CentOS, Fedora):

For Debian, ubuntu use : apt-get install logrotate

# yum install logrotate
Loaded plugins: amazon-id, rhui-lb, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package logrotate.x86_64 0:3.7.8-26.el6_7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================
 Package                        Arch                        Version                              Repository                                             Size
=============================================================================================================================================================
Installing:
 logrotate                      x86_64                      3.7.8-26.el6_7                       rhui-REGION-rhel-server-releases                       58 k

Transaction Summary
=============================================================================================================================================================
Install       1 Package(s)

Total download size: 58 k
Installed size: 87 k
Is this ok [y/N]: y
Downloading Packages:
logrotate-3.7.8-26.el6_7.x86_64.rpm                                                                                                   |  58 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : logrotate-3.7.8-26.el6_7.x86_64                                                                                                           1/1
  Verifying  : logrotate-3.7.8-26.el6_7.x86_64                                                                                                           1/1

Installed:
  logrotate.x86_64 0:3.7.8-26.el6_7

Complete!

Confirm if its installed

# rpm -q logrotate
logrotate-3.7.8-26.el6_7.x86_64

Once logrotate is confirmed installed move on to configurations.

Logrotate configuration :

/etc/logrotate.conf is the main configuration file. Individual service-specific configuration files can be kept under directory /etc/logrotate.d

To use these service-specific individual config files you must include below the line in /etc/logrotate.conf

include /etc/logrotate.d

Let’s configure logrotate to manage /var/log/messages (Syslog in Linux). To keep things clean, its recommended to create individual configs under /etc/logrotate.d. We will create rotate_syslog.conf file under this directory and add the below code.

/var/log/messages {
daily
rotate 3
size 10M
compress
delaycompress
}

Now understand each filed specified in conf file –

  1. daily: Rotate logs on daily basis
  2. rotate 3: Means keep last 3 rotated logs. Older copies to be purged.
  3. size 10M: Min size for rotation is 10MB i.e. logs won’t be rotated unless they grow more than 10MB
  4. compress: Compress rotated logs
  5. delaycompress: Do not compress current log and last rotated log.

Above all options in config file tells logrotate to rotate /var/log/messages logs when they grow over 10MB. After rotation keeps any extra logs than the last 3 one. Also, compress logs except current and last rotated one. Check for size daily and if found >10MB then rotate.

logroate testing :

We will dry run the above config file to check how it will work in the actual run. -d option dry runs utility but doesn’t rotate logs in actual. We have 21MB /var/log/messages file on the machine.

# logrotate -d /etc/logrotate.d/rotate_syslog.conf
reading config file /etc/logrotate.d/rotate_syslog.conf
reading config info for /var/log/messages

Handling 1 logs

rotating pattern: /var/log/messages  10485760 bytes (3 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/messages
  log needs rotating
rotating log /var/log/messages, log->rotateCount is 3
dateext suffix '-20170224'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/messages.1 does not exist
renaming /var/log/messages.3.gz to /var/log/messages.4.gz (rotatecount 3, logstart 1, i 3),
renaming /var/log/messages.2.gz to /var/log/messages.3.gz (rotatecount 3, logstart 1, i 2),
renaming /var/log/messages.1.gz to /var/log/messages.2.gz (rotatecount 3, logstart 1, i 1),
renaming /var/log/messages.0.gz to /var/log/messages.1.gz (rotatecount 3, logstart 1, i 0),
renaming /var/log/messages to /var/log/messages.1
removing old log /var/log/messages.4.gz
error: error opening /var/log/messages.4.gz: No such file or directory

In above output you can see, it rotate logs, compress them (gz extensions), renames them to maintain iterations, try to purge older than 3 logs

When I dry run on file with 9MB size, it gave me below output :

# logrotate -d /etc/logrotate.d/rotate_syslog.conf
reading config file /etc/logrotate.d/rotate_syslog.conf
reading config info for /var/log/messages

Handling 1 logs

rotating pattern: /var/log/messages  10485760 bytes (3 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/messages
  log does not need rotating

Since we specified 10MB size to rotate log and our file is less than 10MB, logrotate reported no need to rotate logs!

You can observe logrotate entry being added to /etc/cron.daily directory too.

# cd /etc/cron.daily
# ll
total 32
-rwx------. 1 root root  118 Jun 10  2015 cups
-rwx------. 1 root root  180 Jul  9  2015 logrotate
-rwxr-xr-x. 1 root root  905 Nov 16  2012 makewhatis.cron
-rwx------. 1 root root  189 Jan 26  2015 mlocate.cron
-rwxr-xr-x. 1 root root 2126 Jul 19  2013 prelink
-rwxr-xr-x. 1 root root  563 Oct 21  2013 readahead.cron
-rwx------. 1 root root  256 Jun  9  2015 rhsmd
-rwxr-xr-x. 1 root root  416 Oct 14  2015 tmpwatch

logroatete available options :

Below is a list of logrotate options that can be used in configuration files along with their meaning.

  1. daily,monthly, weekly: when to rotate logs (if conditions meet)
  2. rotate N: Purge all older logs keeping the last N rotated logs on the server
  3. size X: Rotate logs when they grow more than X size
  4. compress: Compress rotated logs using gzip (there are more options to choose compression commands, extensions, etc if you want)
  5. delaycompress: Don’t compress current and last rotated log file
  6. copy: Make a copy of the current log file
  7. create mode user group: After rotation create an empty log file with permission (mode), user and group specified
  8. dateext: Append the date to rotated file names
  9. mail: Mail rotated logs to email before deletion (more options available with this)
  10. ifempty: Rotate log even if it’s empty
  11. maxage N: Rotate logs older than N days
  12. maxsize N: Rotate when they have grown beyond N size even before run-time daily, weekly or monthly.

There are many other options available that can be used. I have listed here only frequently used ones. A detailed list can be obtained on the man page of logrotate.

All the above outputs are from the RHEL6 box. Let me know if you have any queries in the comments or if you are looking for specific options to rotate logs with your requirement.

4 step Network bonding / teaming configuration in Linux

This article explains what is network bonding in Linux. Quick 4 step guide helps you to set up network bonding in your server in minutes.

Network bonding or network teaming is binding two physical NIC (Network Interface Card) together to create once virtual NIC. This virtual NIC serves the purpose of redundancy, fault tolerance, and load balancing.

For an application running on system its a one NIC they are talking to but on bare metal, their requests are being served by two physical cards. Hence in case, one physical card is failed or unplugged, another one still serves beneath virtual NIC, and applications don’t even know about failure. It’s the same as Auto Port Aggregation (APA) in HPUX.

As of now with RHEL7, there are 7 types of NIC bond available :

  1. Bond 0: Load balancing (round-robin)
  2. Bond 1: Active backup
  3. Bond 2: Balance XOR
  4. Bond 3: Broadcast
  5. Bond 4: 802.3ad
  6. Bond 5: Balance TLB
  7. Bond 6: Balance ALB

We will see in detail about these types in another post. More commonly used are type 0 and type 1 bond. Let’s see step by step procedure to configure a network bond in Linux.

For this tutorial, we will consider two ethernet cards eth1 and eth2 to configure bond. It is assumed that both are configured/connected to the same network VLAN.

Step 1:

Configure both eths with master bond0 and slave as themselves. For that, open NIC configuration file located in /etc/sysconfig/network-scripts/ifcfg-eth1 & ifcfg-eth2 in vi and edit entries as highlighted below :

DEVICE=eth1
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes
NM_CONTROLLED=no

For eth2 file, DEVICE name will be eth2.

Step 2:

Create bond0 device file under /etc/sysconfig/network-scripts/ifcfg-bond0Add the below details in it.

DEVICE=bond0
ONBOOT=yes
IPADDR=10.10.2.5
NETMASK=255.255.255.0
BONDING_OPTS="mode=1 miimon=100"

Under bonding options, we choose mode 1. If you choose to select any other mode out of 7 mentioned above, you need to specify here against mode=

Step 3:

Make sure the bonding module is loaded into the kernel. Add append lines in /etc/modprobe.conf file.

alias bond0 bonding
options bond0 mode=balance-alb miimon=100

Execute module with below command.

# modprobe bonding

Step 4:

That’s it. You are done with configuration. You need to restart networking service and you are good to go. Make sure your network manager service is not running.

# service network restart

Shutting down interface bond0:                             [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface bond0:                               [  OK  ]

You can confirm your bond0 is up with the mentioned IP in ip addr command output. Bonding mode can be verified with below command :

# cat /proc/net/bonding/bond0

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 100
Down Delay (ms): 100

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:b6:be:32

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:b6:be:56

Even ifconfig command output will show you bond0 is up with the mentioned IP address.

Complete guide: Transfer Of Control (TOC) in HP servers

Everything you need to know about TOC i.e. Transfer Of Control reset in HP servers. It’s a way to initialize system halt and memory dump in an emergency.

What is TOC?

TOC stands for Transfer Of Control! Its a way out for sysadmin when their system stops responding or hung or not taking any inputs and they need to take memory dump before resetting system. This memory dump is helpful for investigating the cause of system abnormality.

Whenever TOC order (hardware signal) has been issued to the system, it stops all current work and starts dumping current memory information in the dump device specified in configurations. Once dumping completes, the system resets.

Why to invoke TOC?

There are many reasons like utilization being high, the disk is getting full, some process going in a loop, many processes forked (Error like sh: The fork function failed. Too many processes already exist.), etc which could bring the system down to its knees. In such a situation there is no way than resetting system since these issues make the system unusable or not responding. So why TOC? Even the normal reset will do the job.

But if you are interested in the root cause of what has happened on the system which brought it down then you will need a memory dump for analysis. This memory dump can be generated when TOC is issued. Since the system doesn’t respond to the user, you can not check what’s happening and then the memory dump is only hoped for investigation after reboot. Hence, TOC reset is always recommended in case of system hung issues.

How to do TOC reset ?

  1. TOC can be invoked by using the TOC switch on the back of your HP server.
  2. Using TC command in the GSP menu.
  3. Using vparreset with option (for vPars)
TOC switch :

Its located in the back of your HP server normally a push button. Sometimes it also accompanied by GSP reset switch too. You need to use it to activate TOC.

TC command in GSP :

Login to GSP or MP. Goto command menu using CM. Then use TC command there to reset with TOC.

MP MAIN MENU:

         CO: Consoles
        VFP: Virtual Front Panel
         CM: Command Menu
         CL: Console Logs
         SL: Show Logs
         FW: Firmware Update
         HE: Help
          X: Exit Connection

[Server-mp] MP> cm
                Enter HE to get a list of available commands

                      (Use ^B to return to main menu.)

[Server-mp] MP:CM> TC
vparreset command :

Using -t option with vparreset command reset vPars with TOC.

# vparreset -p <vpar_name> -t

					

How to get boot path of vpmon in HPUX

Learn to identify the boot path of vpmon vPar Monitor. It’s important to know vpmon path when you are planning activities on virtual partitions in HP hardware

What is vpmon?

vpmon is vPars Monitor. It’s a daemon that monitors vPars in the background. It also provides a shell MON> through which various operations can be performed on vPars. Hence vpmon is a very crucial component when it comes to deal with vPars. Also, unless specified, all operations by vpmon are performed on boot disk from which it was spawned. So boot disk of vpmon is an important aspect while planning any activity on vPars.

The only vparload is the command which has the facility to specify different disk on which operation to be performed. Or else all commands of vpmon runs on boot disk it was booted from.

Boot path of vpmon

To get boot path of vpmon you need to run below command from one of the vPar running HPUX.

testsvr# vparstatus -m

Console path: No path as console is virtual
Monitor boot disk path: 0.0.4.1.0.1.0
Monitor boot filename: /stand/vpmon
Database filename: /stand/vpdb
Memory ranges used: 0x0/349224960 monitor
0x14d0c000/237568 firmware
0x14d46000/581632 monitor
----- output truncated -----

You can see boot path against Monitor boot disk path (highlighted above). This is the hardware address of the disk which you need to decode to get disk name in kernel/OS. IT can be decoded as below from left to right :

  1. This is cabinet number
  2. This is I/O chassis (0 is front, 1 is back)
  3. Its I/O bay
  4. Its slot number
  5. Rest is ctd

Normally, the first disk of first vPar people set as vpmon boot path.

RHEL6 boot process

Understand step by step how RHEL6 system boots. Walkthrough of  RHEL6 boot process which lists all the tasks, activities happen during boot.

Anyone starting to learn Linux must know the boot process of Linux. Here is this post I will be explaining the boot process of Red Hat Enterprise Linux 6 i.e. RHEL6. In brief RHEL6 boot process can be summarized as below :

  1.  Powered on system loads boot loader once it completes POST. The boot loader in turn loads GRUB.
  2. GRUB loads kernel into memory which further loads necessary modules and mount root partition as read-only.
  3. Kernel invokes /sbin/init program and hands it over the boot process.
  4. Init program loads all services as per run level and mounts mount points
  5. The user is presented with a login screen.

Lets see each point in detail to understand RHEL6 boot process properly.

1. Power on and boot loaders:

Whenever the system turned on, it runs POST (Power on self-test) to check all hardware and its operating state. Once POST is cleared, the system runs BIOS. BIOS is a basic input-output system which is the lowest level interface for hardware. BIOS gets loaded in memory and checks system, connected peripherals, boot device path. Lastly, BIOS will load the first sector of the bootable disk in memory which is the MBR master boot record. Once MBR loaded, BIOS hand over boot control to it.

MBR is a small machine code that has a first stage boot loader. The first stage or stage-1 boot loader exists to locate the second stage boot loader and load it in memory only. The second stage or stage-2 boot loader is GRUB. Now boot control is with GRUB.

In UEFI based systems, BIOS is replaced by UEFI. It’s much powerful than BIOS. It has its own architecture, CPU, device drivers. It can mount and read file systems. Such systems have EFI partitions that have EFI’s own boot loaders which can load operating systems or stage-2 boot loaders.

2. GRUB:

GRUB displays a list of available kernels to the user in the graphical interface (like below). Its configuration file is /boot/grub/grub.conf (for BIOS) or /boot/efi/EFI/redhat/grub.conf (for UEFI). Here user can select its kernel to boot using arrow keys and press enter. If not then it will boot default selected kernel when selection time passes out. We can even reset the forgotten root password on this screen.

Once GRUB destined to load the kernel, it searches the kernel binary of it under /boot partition. The boot loader then places one or more appropriate initramfs (Initial RAM file system) images into memory (as seen in the above screenshot). The initramfs is used by the kernel to load drivers and modules necessary to boot the system. Once kernel and initramfs are loaded into memory, boot control is taken by the kernel.

3. Kernel:

Once kernel gets boot control, it quickly run though below tasks:

  • Initialize and configure memory, hardware, and attached peripherals.
  • Decompress initramfs into /sysroot and loads necessary drivers from it
  • Loads virtual devices related to file systems like LVM etc.
  • Free up memory by removing initramfs image
  • Create a root device, mount root partition (read-only)

Now, the kernel is fully loaded and operational. But no services loaded in the system yet so the system is not usable for humans. To load rest of the services kernel calls /sbin/init program and hand it over boot process to him.

4. Init program :

/sbin/init i.e. init process spawns very first in a system with PID 1 and it will be parent process for many system processes or zombie/defunct processes all the time. Init executes and calls various scripts as below :

  • Runs /etc/rc.d/rc.sysinit to start swap, set environment, FS checks, and some system initialization steps.
  • Process jobs in /etc/event.d directory which has run level specific settings
  • Set function library /etc/rc.d/init.d/functions
  • Runs background processes from their respective rc directories. Default specified in /etc/inittabe.g. for run level 3, it will execute /etc/rc.d/rc3.d/ . Mostly rc directories are having symbolic links of start/stop services.
  • Once all processes started in the specified run level, init finishes, and spawns login screen.

5. Login screen:

Once init completes loading RC directories, it forks Upstart which in turns call /sbin/mingettymingetty will be forked for each virtual console. Run level 1 i.e. single user mode has 1 while run level 2 to 5 has 6 virtual consoles. /sbin/mingetty starts communication with tty devices, sets terminal modes, prints login screen (with messages if any), and prompt username to the user!

This completes RHEL6 boot process from power up to login prompt!

How to open the file in the read-only mode under vi or vim

Learn how to open the file in the read-only mode under vi or vim editor. Opening it as read-only prevents any accidental edits in the file and maintain file integrity. 

“vi editor” is sysadmin’s and programmer’s daily text editor in Linux Unix systems. Opening a file to view its content can be achieved by many commands like cat, more, less etc. But many prefer to open a file in vi editors to view. Especially when the file is long and one needs to search particular terms in it. Vi editor makes it easy to search content, line numbering while viewing files.

One of the disadvantages is you are prone to accidentally alter file content and end up saving it in the file. This is a threat to file integrity and hence needs to be avoided. The option is to view files in vi editors carefully or open them in read-only mode!

Lets see different ways to view file in read only mode under vi :

1. Use of view command

One of the widely used ways to view files in vi editors. Simple open file with view command. It will open in vi editors and any attempt to save/alter data will result in failure, securing your file from accidental edits.

# view test.txt

This is test file
Test text
File ends here.
~
~
~
~
~
"test.txt" [readonly] 3L, 44C

You can see above it shows readonly at the bottom and open file in vi editors. If you get into INSERT mode to edit any content of the file you will see below warning.

W10: Warning: Changing a readonly file

If you try to save edited content then you will see below warning

E45: 'readonly' option is set (add ! to override)

So this is 2 level warnings before you could actually save the file using override option w!. There is no way you could *accidentally* ignores two warnings and make accidental edits in the file!

2. vi or vim command with -R option

Another way is to open the file in vi editors with -R option. It functions the same way as above and also shows the same double-layered warning messages when you enter INSERT mode and try to save the file. This option still lets you save the edits made in the buffer by using override w!.

You can open file using vi -R filename

3. vi modifications not allowed mode

Modification not allowed mode can be called using -M option. From the man page, The ’modifiable’ and ’write’ options will be unset, so that changes are not allowed and files can not be written.

Unlike the above two options, this mode won’t let you edit files at all. It will show below error at the bottom when you enter INSERT mode.

E21: Cannot make changes, 'modifiable' is off

So you won’t be able to type in anything even you try INSERT mode. If you try to save the file with :w then it will show you below error :

E142: File not written: Writing is disabled by 'write' option

This makes it the most secure way to open files in the read-only mode under vi to avoid accidental content alteration!

Let us know which way you use most in your daily operations in comments. Happy viewing! 🙂

Everything you need to know about the zombie process

Understand what is zombie process in Linux Unix, how the process goes to a zombie state, how to handle the zombie process, what is the impact of zombie processes.

In very lame terms, the Zombie process is a process that is present in the process table even if it’s already dead! The zombie process is also known as a defunct process. To understand how the zombie process gets created let see how to process exiting takes place from memory.

How the zombie gets created

Whenever the process completes its execution, it exits and notifies its parent process that his child has died. At this time, the parent process supposes to execute the WAIT system call which reads dead child process status and other details. Once the wait system call completes, a dead child will be removed from memory. At this point, if the parent process is not coded properly or unable to read this status from a child for some reason then it won’t fork wait system call. This in turn keeps the dead child process in memory & process table.

This whole operation completes very fast. Zombie takes a very tiny amount of memory to live in so a couple of zombies on the server are harmless. Many zombie processes have parent PID as 1 which is the init process. When a child is dead but not cleared from memory and parent process exists then those child zombies will be taken over by init. Init usually runs its child zombie clearance periodically. So its possible those zombies get cleared out in it.


How to check the zombie process

You can list zombie processes on your server with below command :

# ps aux |grep Z
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     16479  0.0  0.0 103304   812 pts/0    S+   10:44   0:00 grep Z
oracle9  14523  0.0  0.0  52367   702 pts/1    Z    09:15   1:00 asldjkeh

In the above output watch out for STAT column. Z indicates the zombie process. Even top command shows a total number of zombie processes on the system. Check highlighted field in yellow :

You can get parent PID of the zombie process using ps or pstree command.


How to kill the zombie process

You simply can’t! Because of its Zombie! It’s already dead! The maximum you can do is to inform its parent process that its child is dead and now you can initiate a wait system call. This can be achieved by sending SIGCHLD a signal to parent PID using below command :

# kill -s SIGCHLD <ppid>

Here, ppid is the parent process id.


Impact of the zombie process on the system

As we already discussed, a couple of zombie processes are harmless. But if they are growing rapidly then there are two areas which may be bottlenecked and your system prone to panic :

  1. Each zombie holds its PID hence rapidly growing zombies can exhaust all available PIDs on the system and then no new process can be forked by the kernel.
  2. Even zombie holds a very tiny amount of memory, huge numbers can make a difference. A large number of zombies can hog a considerable amount of memory contributing to high memory utilization of the machine.

If your system is flooded with zombies that are not being cleared even by init, system reboot can be tried for a quick refresh. Obviously this would be the last resort you should be looking at.

Dynamic Root Disk DRD configuration in HPUX

Learn how to configure Dynamic Root Disk (DRD) in HPUX. Understand how to clone root disk, view files in it, activate, and deactivate DRD disk.

Dynamic Root Disk aka DRD is a root disk cloning tool from HP. This tool aims to provide system integrity solutions at maintenance activities performed on the root disk. On DRD cloned disk you can perform any maintenance activity which you planned to do on actual live disk without worrying about disturbing the running system. You can activate cloned disk and reboot server which then boots from altered cloned disk. If you observe your changes are not perfect, you can re-activate your old live root disk hence getting back to the original state within minutes!

Proposed normal DRD clone disk life cycle is :

  1. Clone live root disk
  2. Mount cloned disk
  3. Make any changes you want on the cloned disk
  4. Activate cloned disk and reboot server
  5. Now system boots from cloned disk (Your old live disk is intact!)
  6. If you want to go back to the old state, set the old live disk as the primary boot disk
  7. Reboot system and your old live disk will be booted as it is.

Let’s see different operations which can be done through dynamic root disk commands.

1. How to clone root disk using DRD

DRD has its own set of commands to perform operations on clone disk. To clone your live root disk, attach/identify unused disk with the same or more capacity than live root disk with the same technology/model. Once identified, use below command :

# /opt/drd/bin/drd clone -v -x overwrite=true -t /dev/dsk/c0t2d0

=======  04/22/16 16:42:47 IST  BEGIN Clone System Image (user=root)  (jobid=testsrv)

       * Reading Current System Information
       * Selecting System Image To Clone
       * Selecting Target Disk
       * Converting legacy DSF "/dev/dsk/c0t2d0" to "/dev/disk/disk6"
       * Selecting Volume Manager For New System Image
       * Analyzing For System Image Cloning
       * Creating New File Systems
       * Copying File Systems To New System Image
       * Making New System Image Bootable
       * Unmounting New System Image Clone
       * System image: "sysimage_001" on disk "/dev/disk/disk6"

=======  04/22/16 17:14:18 IST  END Clone System Image succeeded. (user=root)  (jobid=testsrv)

DRD binary resides in /opt/drd/bin. Use clone argument todrd command and supply target disk path with -t option (which will be final cloned disk). There are a few options which can be used with -x. We used here to overwrite disk if any data resides in it. This command execution takes 30 mins to hours time depending on your root VG size.

In the end, you can see system image has been cloned on disk /dev/dsk/c0t2d0 i.e. /dev/disk/disk6. You can check the status of DRD using the below command which lists all details about the cloned disk.

# /opt/drd/bin/drd status

=======  04/22/16 17:24:21 IST  BEGIN Displaying DRD Clone Image Information (user=root)  (jobid=testsrv)

       * Clone Disk:               /dev/disk/disk6
       * Clone EFI Partition:      AUTO file present, Boot loader present, SYSINFO.TXT not present
       * Clone Creation Date:      04/22/16 16:43:00 IST
       * Clone Mirror Disk:        None
       * Mirror EFI Partition:     None
       * Original Disk:            /dev/disk/disk3
       * Original EFI Partition:   AUTO file present, Boot loader present, SYSINFO.TXT not present
       * Booted Disk:              Original Disk (/dev/disk/disk3)
       * Activated Disk:           Original Disk (/dev/disk/disk3)

=======  04/22/16 17:24:32 IST  END Displaying DRD Clone Image Information succeeded. (user=root)  (jobid=testsrv)

2. How to mount the cloned disk

Once the disk is cloned, you can view data within it by mounting it. Use mountargument with drd command.

# /opt/drd/bin/drd mount

=======  04/22/16 17:30:20 EDT  BEGIN Mount Inactive System Image (user=root)  (jobid=testsrv)

 * Checking for Valid Inactive System Image
 * Locating Inactive System Image
 * Mounting Inactive System Image

=======  04/22/16 17:30:31 EDT  END Mount Inactive System Image succeeded. (user=root)  (jobid=testsrv)

This will create a new VG on your system named drd00 and mounts clone disk within it. All you root disk mount points in the cloned disk will be mounted on /var/opt/drd/mnts/sysimage_000 e.g. /tmp in the cloned disk will be available on /var/opt/drd/mnts/sysimage_000/tmp mount point. See below output for your understanding:

# bdf
Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    4194304  176968 3985984    4% /
/dev/vg00/lvol1    2097152  158344 1923696    8% /stand
/dev/vg00/lvol8    12582912  846184 11645064    7% /var
/dev/vg00/lvol7    10485760 3128368 7299968   30% /usr
/dev/vg00/lvol6    10485760  456552 9950912    4% /tmp
/dev/vg00/lvol5    10485760 4320288 6117352   41% /opt
/dev/vg00/lvol4    4194304   21304 4140408    1% /home
/dev/drd00/lvol3   4194304  176816 3986136    4% /var/opt/drd/mnts/sysimage_000
/dev/drd00/lvol4   4194304   21304 4140408    1% /var/opt/drd/mnts/sysimage_000/home
/dev/drd00/lvol5   10485760 4329696 6108024   41% /var/opt/drd/mnts/sysimage_000/opt
/dev/drd00/lvol1   2097152  158408 1923696    8% /var/opt/drd/mnts/sysimage_000/stand
/dev/drd00/lvol6   10485760  456536 9950928    4% /var/opt/drd/mnts/sysimage_000/tmp
/dev/drd00/lvol7   10485760 3196640 7232232   31% /var/opt/drd/mnts/sysimage_000/usr
/dev/drd00/lvol8   12582912  876016 11615544    7% /var/opt/drd/mnts/sysimage_000/var

You can even un-mount DRD cloned disk using drd unmount command.

# /opt/drd/bin/drd umount -v 

=======  04/22/16 17:30:45 IST  BEGIN Unmount Inactive System Image (user=root)  (jobid=testsrv)

       * Checking for Valid Inactive System Image
       * Locating Inactive System Image
       * Preparing To Unmount Inactive System Image
       * Unmounting Inactive System Image
       * System image: "sysimage_001" on disk "/dev/disk/disk6"

=======  04/22/16 17:30:58 IST  END Unmount Inactive System Image succeeded. (user=root)  (jobid=testsrv)

3. Different tasks which can be performed on cloned DRD disk

There are different maintenance activities that you can perform on this cloned DRD disk. To name a few: patch installation, editing some system files manually, tuning static kernel parameters, etc.

To execute tasks on the cloned disk you need to supply commands as an argument to drd runcmd option. For example, if you want to view /etc/hosts file in the cloned image,  use drd runcmd cat /etc/hosts

# /opt/drd/bin/drd runcmd kctune -B nproc+=100

=======  04/22/16 18:15:54 IST  BEGIN Executing Command On Inactive System Image (user=root)  (jobid=testsrv)

       * Checking for Valid Inactive System Image
       * Analyzing Command To Be Run On Inactive System Image
       * Locating Inactive System Image
       * Accessing Inactive System Image for Command Execution
       * Setting Up Environment For Command Execution
       * Executing Command On Inactive System Image
       * Executing command: "/usr/sbin/kctune -B nproc+=100"
WARNING: The backup behavior 'yes' is not supported in alternate root
         environments.  The behavior 'once' will be used instead.
       * The automatic 'backup' configuration has been updated.
       * Future operations will ask whether to update the backup.
       * The requested changes have been applied to the currently
         running configuration.
Tunable            Value  Expression  
nproc    (before)   4200  Default     
         (now)      4300  4300        
       * Command "/usr/sbin/kctune -B nproc+=100" completed with the return code "0".
       * Cleaning Up After Command Execution On Inactive System Image

=======  04/22/16 18:16:23 IST  END Executing Command On Inactive System Image succeeded. (user=root)  (jobid=testsrv)

See above example where I tune kernel parameters within the cloned disk.

You can even install patches using command drd runcmd swinstall -s /tmp/patch123.depot. Even if patch which needs a reboot can be installed. Since you are installing it on cloned (nonlive) root disk, the server won’t be rebooted. To make these changes live on your server, you need to boot the server with this cloned disk.

4. How to activate DRD cloned disk

To activate the dynamic root disk, you need to run drd activate command. Actually, this command sets your cloned disk path as a primary boot path which you can do by setboot command too!

# /opt/drd/bin/drd activate -x reboot=true

=======  04/22/16 18:20:21 IST  BEGIN Activate Inactive System Image (user=root)  (jobid=vm19)

       * Checking for Valid Inactive System Image
       * Reading Current System Information
       * Locating Inactive System Image
       * Determining Bootpath Status
       * Primary bootpath : 0/0/0/0.0x0.0x0 before activate.
       * Primary bootpath : 0/0/0/0.0x2.0x0 after activate.
       * Alternate bootpath : 0/0/0/0.0x1.0x0 before activate.
       * Alternate bootpath : 0/0/0/0.0x1.0x0 after activate.
       * HA Alternate bootpath :  before activate.
       * HA Alternate bootpath :  after activate.
       * Activating Inactive System Image
       * Rebooting System

If you set reboot to false, it will just set the primary boot disk path and exists. After that when you manually reboot the system, it will boot from cloned disk.

If you don’t choose auto-reboot then you will have a chance to reverse activate operation using deactivate command argument.

5. After booting cloned disk

If you boot your system from dynamic root disk, below things will be changed :

  1. Root VG mirroring will be missing
  2. Past live root disk will be intact
  3. Past live root disk will be removed from setboot primary/alternate boot path settings
  4. You have to restore the root mirror
  5. You have to check and set the alternate boot path
  6. Your system will be having all changes (patch install, kernel tuning) you made on the cloned disk

Dynamic Root Disk is a very powerful tool when it comes to chopping down your downtime. If you have little downtime window and need to perform a large number of patching which requires the reboot. Patch cloned disk and just reboot server during your short downtime!