How to do safe and graceful Measureware service restart in HPUX

A how-to guide for safe and graceful measureware service restart on HPUX machines. Learn how to preserve old log files during service restart and avoid overwriting them.

Measureware service is a native utility to HPUX for performance measurement. It is responsible to collect system utilization data in the background. Measureware agent mwa runs in background and stores data in logfiles called datafiles. If you attempt measureware service restart without moving logfiles then it will overwrite current files and all historic data is on the toss. Hence you need to stop it then move data files to another location and then start it. In this sequence, you prompt agents to create new blank data files to save data.

You can view the current status of all measureware services using below command :

# mwa status all
 Perf Agent status:
    Running scopeux               (Perf Agent data collector) pid 2814
    Running midaemon              (Measurement Interface daemon) pid 2842
    Running ttd                   (ARM registration daemon) pid 2703

 Perf Agent Server status:

    Running ovcd                  (OV control component) pid 3483
    Running ovbbccb               (BBC5 communication broker) pid 3484
    Running coda                  (perf component) pid(s) 3485
       Configured DataSources(1)
                  SCOPE

    Running perfalarm             (alarm generator) pid(s) 2845

If any of the components are not running or having issues then it may call for measureware service restart. Let’s see the process of the graceful shutdown and the start of measureware services in HPUX.

Read also another performance measurement tool System Activity Report (SAR) in the below series :

1. Stop mwa

Stop all measureware services with single command as below :

# mwa stop all

Shutting down Perf Agent collection software
         Shutting down scopeux, pid(s) 2814
         The Perf Agent collector, scopeux has been shut down successfully.
NOTE:   The ARM registration daemon ttd will be left running.

Shutting down the alarm generator perfalarm, pid(s) 2845
         The perfalarm process has terminated

OVOA is running. Not shutting down coda

As you can see in the above output ttd is left running by command. You need to kill it using below command :

# ttd -k

Also, mideamon still runs after the above command. You can terminate it using :

#  midaemon -T

These three commands collectively shut off everything related to measureware services. You can confirm if midaemon, ttd and scopeux are down with status command again :

#  mwa status all
 Perf Agent status:
WARNING: scopeux    is not active (Perf Agent data collector)
WARNING: midaemon   is not active (Measurement Interface daemon)
WARNING: ttd        is not active (ARM registration daemon)

 Perf Agent Server status:

    Running ovcd                  (OV control component) pid 3483
    Running ovbbccb               (BBC5 communication broker) pid 3484
    Running coda                  (perf component) pid(s) 3485
       Configured DataSources(1)
                  SCOPE

WARNING: perfalarm is not active (alarm generator)

This ensures you can proceed with log movement before starting mwa again.

2. Log movement

Datafiles (all starts with log) resides in /var/opt/perf/datafiles directory. List of datafiles is as below :

# ll /var/opt/perf/datafiles/log*
-rw-r--r--   1 root       users      11064908 Jan  1 03:05 /var/opt/perf/datafiles/logappl
-rw-r--r--   1 root       root       43951620 Jan  1 03:05 /var/opt/perf/datafiles/logdev
-rw-r--r--   1 root       users      9556384 Jan  1 03:05 /var/opt/perf/datafiles/logglob
-rw-r--r--   1 root       root         15716 Jan  1 03:01 /var/opt/perf/datafiles/logindx
-rw-r--r--   1 root       users           15 Nov  4  2009 /var/opt/perf/datafiles/logpcmd0
-rw-r--r--   1 root       root       76492020 Jan  1 03:05 /var/opt/perf/datafiles/logproc
-rw-r--r--   1 root       root       96153856 Jan  1 03:05 /var/opt/perf/datafiles/logtran

Now move current data files to a different directory. You can use below small inline scripts to do this or you can manually move them one by one.

# cd /var/opt/perf/datafiles
# nowis=`date +%d%b%y-%H:%M`
# mkdir /var/opt/perf/datafiles.old.`echo $nowis`
# cp log* /var/opt/perf/datafiles.old.`echo $nowis`

Make sure you copied datafiles to the destination correctly and proceed to start services again.

3. Start mwa

Start it using below command :

#  mwa start all

The Perf Agent scope collector is being started.
         The ARM registration daemon
         /opt/perf/bin/ttd has been started.

         The Performance collection daemon
         /opt/perf/bin/scopeux has been started.

         The coda daemon /opt/OV/lbin/perf/coda is already running.
The Perf Agent alarm generator is being started.
         The alarm generator /opt/perf/bin/perfalarm
         has been started.

Observe while shutting down we used three commands for shutting different components but while starting up it came up with the single command. You can check the status with mwa status all command to make sure all components are started. This pretty much sums up how to do a safe and graceful measureware service restart.

All examples on this post are from the machine running HPUX 11.31. Let us know if you have any queries, suggestions, corrections in comments.

chage command in Linux for password aging control

Leave a reply

Learn chage command in Linux with several examples. View, edit password aging parameters using chage command to secure your Linux accounts.

Controlling password aging of user accounts is very much important for the security of the server. This ensures users are always updated with passwords and there are no old passwords or accounts living on the server which are vulnerable to compromise.

1. Last change date

This is the number of days from Unix date i.e. 1 Jan 1970 when the password was last changed. Normally this date changes automatically when the user changes his password. But, if you want to change it manually you can use chage command with -d switch like below :

# chage -d 2016-03-12 user4  << YYYY-MM-DD format

You can view change in date by comparing before and after the output of chage -l <user> command. This date is displayed against the “Last password change” attribute in the output. We will see this output in detail in the last part of this post.

2. Expiry date

This is the date on which account will expire and the user won’t be able to log in until he changes his account password. It can also be set as YYYY-MM-DD format with -E option as below :

# change -E 2016-12-05 user4

This date changes automatically whenever the user changes his password. It checks the maximum days attribute and adds those many days to the current date (date of password change); the resulting date will be an expiry date.

Setting this to -1 removes the account password expiry. That account will have a non-expiry password and never need to change the password in the future.

3. Minimum days

These are a number of days a user must wait to make another password change on his account. For example, if this is set to 7 then once a user changes the password, he can not change the password again until 7 days. This can be set using -m option.

# chage -m 7 user4

Setting this parameter to 0 enables the user to change his password at any time (no restriction).

4. Maximum days

These are a number of days users can use the same password. For example, if this is set to 20 days then the user must change the password after 20 days. This value decides the password expiration date we saw above. This can be set using -M option

# chage -M 30 user4

If you want to remove this restriction and want to use the same password forever then you need to set the expiration date to -1 which we saw earlier.

5. Warning days

These are a number of days before the password expiry date, the user starts seeing a warning on his login screen about password expiry. User warning will be shown post-login like below :

login as: user4
user4@10.10.2.5's password:
Warning: your password will expire in 6 days
Last login: Thu Dec 29 17:17:32 2016 from 10.10.2.10
#

You can set this attribute using -W option

# chage -W 7 user4

6. Inactivity period

These are a number of days the account can remain inactive after the password is expired. After which account will be locked for security reasons since idle accounts vulnerable to compromise. This can be set using -I option.

# chage -I 10 user4

If you set this to -1 then this restriction will be waived off from that account.

7. Viewing all the above attributes

To view all the above attributes you can use -l option :

# chage -l user4
Last password change                                    : Mar 12, 2016
Password expires                                        : Jun 10, 2016
Password inactive                                       : never
Account expires                                         : Nov 30, 2016
Minimum number of days between password change          : 0
Maximum number of days between password change          : 90
Number of days of warning before password expires       : 7

In above output:

Last password change is Last change date -d
Account expires is the expiry date -E
Password inactive is Inactivity period -I
Account expires is the expiry date for the account. Last change date plus maximum days.
Minimum number of days……. is minimum days -m
Maximum number of days……. is maximum days -M
Number of days of warning …… is warning days -W

You can check this output before changing any attribute using the above commands. Check change in attribute post command execution again!

Let us know queries, suggestions, feedback, corrections in the comments below.

9 Linux account password policies explained

1 Reply

Learn 9 different account password policies in Linux. Understand how to view them, how to change them and what is their impact on user management.

User management is one of the important aspects of Linux system administration. Restricting unauthorized access to systems can be prohibited by implementing strong password policies on accounts. That’s why this is a mandatory task in system hardening.

In this post, we will be seeing below nine different password policies that can be implemented in Linux.

Password Max days
Password Min days
Password warning days
Password history depth
Password minimum length
Minimum upper case characters
Minimum lower case characters
Minimum digits in password
Wrong password retry

In the above list first 3 parameters are password aging-related whereas rest decides password strength.

1. Password Max days

This parameter decides how many days the maximum a password can be used. Once account password ages for these many days, it’s mandatory for the user to change his/her account password. This forbids users from using the same password for a long duration. In short, this is a maximum number of days password is valid on the system. This value can be set under file /etc/login.defs against parameter PASS_MAX_DAYS as shown below:

# cat  /etc/login.defs

----- file clipped -----
#       PASS_MAX_DAYS   Maximum number of days a password may be used.
PASS_MAX_DAYS   90
----- file clipped -----

File parameter values affect only newly created accounts after the file has been edited. But for existing accounts, you need to change this value manually by using chage command with -M option. You can check the current set value by using -l option.

# chage -l user4

----- output clipped -----
Maximum number of days between password change          : 30
Number of days of warning before password expires       : 7

# chage -M 45 user4

# chage -l user4

----- output clipped -----
Maximum number of days between password change          : 45
Number of days of warning before password expires       : 7

Observe in the above example, max days for an existing account have been changed from 30 to 45 days using chage command.

2. Password min days

These attributes control a minimum number of days before a password can be changed. This forbids users from changing passwords too frequently. For example, if this parameter is set to 7 days & user changed password today. Then he will be able to change it again only after 7 days from now. This value can be set under file /etc/login.defs against parameter PASS_MIN_DAYS as shown below:

# cat  /etc/login.defs

----- file clipped -----
# PASS_MIN_DAYS Minimum number of days allowed between password changes.
PASS_MIN_DAYS 1
----- file clipped -----

File parameter values affect only newly created accounts after the file has been edited. But for existing accounts, you need to change this value manually by using chage command with -M option.

# chage -l user4

----- output clipped -----
Minimum number of days between password change          : 3

# chage -m 1 user4

# chage -l user4

----- output clipped -----
Minimum number of days between password change          : 1

3. Password warning days

This attribute controls a number of days before the password expires, the user starts seeing a warning about password change after login. This gives sysadmins a chance to educate and made aware of their system users about password expiry. So that users can change their password well before its expiry time. This is not really adding any security to the system but helping users to avoid unwanted service impacts due to password expiry. Its value can be defined under /etc/login.defs file against PASS_WARN_AGE parameter.

# cat  /etc/login.defs 

----- file clipped -----
#       PASS_WARN_AGE   Number of days warning given before a password expires.
PASS_WARN_AGE   7
----- file clipped -----

Same as the last two parameters, this file parameter values affect only newly created accounts after the file has been edited. But for existing accounts, you need to change this value manually by using chage command with -W option.

# chage -l user4

----- output clipped -----
Number of days of warning before password expires       : 7

# chage -W 10 user4

# chage -l user4

----- output clipped -----
Number of days of warning before password expires       : 10

4. Password history depth

When the user sets a new password, it will be checked against historical passwords. If the user tries to set the same old password then the system will forbid the user to use that password. This password history depth is defined by this attribute. If it is set to 3 then the user won’t be able to use any password which matches his last 3 passwords used.

This depth can be set in /etc/pam.d/system-auth file against the remember parameter.

# cat /etc/pam.d/system-auth |grep -i pass

----- file clipped -----
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok remember=2
----- file clipped -----

In the above example, the last 2 passwords will be kept in history to check against the new one since remember is set to 2.

5. Password minimum length

Minimum characters needed in the password are defined by this attribute. This ensures the enforcement of strong passwords to be used by users. It can be defined in /etc/pam.d/system-auth file against minlen parameter.

# cat /etc/pam.d/system-auth |grep -i pass

----- file clipped -----
password    requisite     pam_cracklib.so try_first_pass retry=3 minlen=8 dcredit=-1 ucredit=-1 lcredit=-1
----- file clipped -----

This will be used whenever a new password is being set.

6. Minimum upper case characters

Another password strengthening attribute like the previous one. This ensures the enforcement of the use of uppercase characters in the password. It can be defined in /etc/pam.d/system-auth file against ucredit parameter.
Example in point 5.

7. Minimum lower case characters

This ensures the enforcement of the use of lowercase characters in the password. It can be defined in /etc/pam.d/system-auth file against lcredit parameter.
Example in point 5.

8. Minimum digits in password

This ensures the enforcement of the use of digits in passwords. It can be defined in /etc/pam.d/system-auth file against dcredit parameter.
Example in point 5.

9. Wrong password retry

This is a number of tries users get to try passwords without locking the account. As universally accepted, this is always set to be 3. Its value can be defined in retry parameter in /etc/pam.d/system-auth file.

Example in point 5.

Please make a note that all the above configurations files are taken into account from the RHEL flavor. If you have any questions, queries, suggestions, corrections please let us know in comments.

How to install patch/software in HPUX

Leave a reply

A how-to guide for the installation of patch or software on the HPUX operating environment. This includes steps to install, configure, and verify.

This post is a how-to guide that has steps to install, configure, and verify patch or software on HPUX operating system. Before we begin the patching process, one needs to finalize the downtime window for patching with the concerned team. Also, if individual patches are planned to install then patch ID needs to be known to download from the HP portal. HP Passport ID required to sign-in on the HP portal.

Also read: HPUX patch naming conventions

HPUX patching enables a system with new features, enhancements and removes bugs if any. HP releases patch bundles for HPUX every 6 months i.e twice in a year. These bundles can be downloaded and applied to the system. Also, individual patches can be downloaded. For individual patches, one should know the patch ID first.

Prerequisite :

Log in to the HP portal and download required patches. HP provides the facility to download your required patch along with its all dependencies so you need not to again search for dependencies separately. When prompted for download, select the gzip bundle format.
Upload the patch on the server using FTP. Unzip patch file which will have create_depot script inside. When executed, this script will create $PWD/depot directory which includes all patches to be installed. Make sure depot is created properly using:

# /usr/sbin/swlist  -s /tmp/PACPTHP_00001.depot

# Initializing...
# Contacting target "testsrv2"...
#
# Target:  testsrv2:/tmp/PACPTHP_00001.depot
#

#
# No Bundle(s) on testsrv2:/tmp/PACPTHP_00001.depot
# Product(s):
#

  HPOvLcore     6.00.000       HP Software Core Functionality
  HPOvLcore     6.00.000       HP Software Core Functionality
  HPOvPerf      4.70.000       HP OpenView Performance
  HPOvPerf      4.70.000       HP OpenView Performance

A server backup is an important thing to be done before any activity. Take ignite backup over network or tape. Also collect nickel, sysinfo outputs from the server. Ensure the downtime window and other related ITIL processes are completed and approved.

Installation & configuration :

Before starting activity make sure all applications on the server are brought down by respective parties. Once confirmed, proceed with patch installation. Install the patch using the command:

# /usr/sbin/swinstall -s /tmp/PACPTHP_00001.depot

NOTE:    The interactive UI was invoked, since no software was
         specified.

Starting the terminal version of swinstall...

To move around in swinstall:

- use the "Tab" key to move between screen elements
- use the arrow keys to move within screen elements
- use "Ctrl-F" for context-sensitive help anywhere in swinstall

On screens with a menubar at the top like this:

        ------------------------------------------------------
       |File View Options Actions                         Help|
       | ---- ---- ------- ------------------------------- ---|

- use "Tab" to move from the list to the menubar
- use the arrow keys to move around
- use "Return" to pull down a menu or select a menu item
- use "Tab" to move from the menubar to the list without selecting a menu item
- use the spacebar to select an item in the list

On any screen, press "CTRL-K" for more information on how to use the keyboard.

This will start the text-based GUI. You can navigate in this GUI menu using, arrow keys and SPACE key. Select the all listed patches using arrow key and space bar to highlight them.

Once marked, select Install option from the Actions menu.

This will take time to install and configure the patch in the system depending upon the size of the depot. This configuration happens in the background with this command execution only.

If the required system will be rebooted post-installation. To check if the patch needs a reboot after install, you can run above swinstall command with -p preview option. Which will just check dependencies, depot health, and show you report. This report includes if a reboot is required or not. Even while downloading the patch from the HP portal, you can view its details which also includes information about reboot requirements.

Verification :

Once the installation is complete, or after reboot, check if all installed patches are configured in the kernel or not using the command :

# swlist -l fileset -a state | grep –v –e “#” –e “configured”

The output of the above command should be blank. If there are any file sets left in an unconfigured/installed state then the above command’s output will reveal it. So, if the output is not blank, you need to configure these unconfigured/installed state file sets manually. To configure all patches again use below command:

# /usr/sbin/swconfig \ *

=======  12/27/16 10:17:15 SST  BEGIN swconfig SESSION
         (non-interactive) (jobid=testsrv2-0396)

       * Session started for user "root@testsrv2".

       * Beginning Selection
       * Target connection succeeded for "testsrv2:/".

----- output clipped -----
 * Selection succeeded.


       * Beginning Analysis
       * Session selections have been saved in the file
         "/home/user3/.sw/sessions/swconfig.last".
       * "testsrv2:/":  2831 filesets have already been configured.
       * Analysis succeeded.


       * Beginning Execution
       * "testsrv2:/":  2831 software objects were determined to be
         skipped in the analysis phase.
       * Execution succeeded.


NOTE:    More information may be found in the agent logfile using the
         command "swjob -a log testsrv2-0396 @ testsrv2:/".

=======  12/27/16 10:17:59 SST  END swconfig SESSION (non-interactive)
         (jobid=testsrv2-0396)

This command will try to configure the remaining patches. After completion of this command, once again verify your installation.

5 different examples to send email through Linux terminal

Leave a reply

Five different examples to show you how to send an email using the Linux terminal. Learn how to send an email with the subject, attachment, and body from the terminal.

We have seen how to configure SMTP in Linux, in this post we will see different examples to send a mail through terminal. This post will elaborate on how to attach a file to mail, how to add a subject line, how to add body content to mail-in command-line interface.

1. An email with the subject line :

Sendmail is low-level utility hence it doesn’t support -s option for a subject line like other email utilities do. We have to stdin subject to sendmail like below :

# echo "Subject : test" |sendmail -v user4@xyz.com
user4@xyz.com... Connecting to mailserver.xyz.com via relay...
220 mailserver.xyz.com ESMTP Postfix
>>> EHLO server2.xyz.com
250-mailserver.xyz.com
250-PIPELINING
250-SIZE 25600000
---- output clipped -----

Text following “Subject:” will be treated as the subject line for mail.

You can also use mailx command with -s option to specify the subject line.

# mailx -s "test email subject line"  user4@xyz.com
Your email body goes here... type and hit ctrl+d
Cc:

2. An email with mail body :

If you have a long email body then you can save it in a file. Then you can pass that file to sendmail command. Sendmail will parse the file and the content of files will be seen in the body of the email.

# cat /tmp/mail_body.txt | sendmail -v user4@xyz.com
user4@xyz.com... Connecting to mailserver.xyz.com via relay...
220 mailserver.xyz.com ESMTP Postfix
>>> EHLO server2.xyz.com
250-mailserver.xyz.com
250-PIPELINING
250-SIZE 25600000
250-VRFY

The same format works with mailx command too. You can even source file using redirector for mail body like below:

# mailx user4.xyz.com < /tmp/mail_body.txt

3. Email with attachment

Sending attachment with sendmail/mail is a bit tricky. We need to encode files before sending as an attachment using uuencode.

# uuencode /tmp/attachement.txt | mail -s "Subject line" user4.xyz.com

4. An email with different FROM field

You can even change from the field of the email so that you can send an email with nicer names in the FROM field rather than ugly account names which mail utility picks up from your account details. There are few options when one is not available on your system to try another.

# mailx -r from_sender_id@xyz.com -s "subject" user4@xyz.com

# mailx -s "subject" user4.xyz.com -- -f from_sender_id@xyz.com 

# mailx -s "subject" -a "From: Sender <from_sender_id@xyz.com>" user4.xyz.com

5. Adding CC and BCC fields:

You can add BC and BCC fields too. Using options -c cc and -b for bcc:

# mail -s "test subject" -c abc@xyz.com -b pqr@xyz.com user4@xyz.com

How to add a subject line in sendmail command

You can add an email subject line in sendmail by sending “Subject: xxxxx” to sendmail command. See below examples –

(echo "Subject: Test email"; cat mailbody.txt) | sendmail -v info@kerneltalks.com

Errors and logfile for email in Linux

Once you send an email check /var/mail/maillog file for errors or success messages. If your configuration is correct and everything is working as expected you can see below message in there.

Sep 17 23:44:38 testsrv postfix/qmgr[10290]: 1086212AA0C: from=<root@testsrv>, size=470, nrcpt=1 (queue active)
Sep 17 23:44:38 testsrv postfix/smtp[17001]: 1086212AA0C: to=<info@kerneltalks.com>, relay=smtp.kerneltalks.com[10.10.1.1]:25, delay=4193, delays=4193/0.02/0.03/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 66AE24043)
Sep 17 23:44:38 testsrv postfix/qmgr[10290]: 1086212AA0C: removed

It means your message is successfully sent to the SMTP server. If you still did not receive the messages in your mailbox then you should check with the SMTP server owner that if they have valid email address restriction. Since if you don’t configure sender address then command by default sent an email with TO address as username@hostname

You can use -r option in mailx command to define TO email address as well which matches the policy of the SMTP server.

# echo test | mailx -s "Test from `hostname`" -r root@kerneltalks.com -S smtp="smtp.kerneltalks.com" info@kerneltalks.com

Like we have defined the sender address as root@kerneltalks.com in the above test command. Now, this email will be accepted by the SMTP server and will be delivered to the recipient.

Another error can be seen in maillog is as below –

Sep 17 23:07:33 testsrv postfix/smtp[9918]: 1086212AA0C: to=<info@kerneltalks.com>, relay=none, delay=1968, delays=1968/0.01/0.26/0, dsn=4.3.5, status=deferred (Host or domain name not found. Name service error for name=smtp.kerneltalks.com type=A: Host not found)

Solution: type A record, the host is not found means server is unable to resolve SMTP server name from DNS. Make sure you have DNS configured in /etc/resolv.conf and SMTP name is registered in DNS. You can verify DNS resolution and DNS server being used using nslookup smtp.kerneltalks.com command.

One more same error but for record type AAAA can be seen as below :

Sep 15 22:41:04 testsrv postfix/smtp[7833]: 97E9C12A7D9: to=<info@kerneltalks.com>, relay=none, delay=349010, delays=349010/0.01/0.26/0, dsn=4.3.5, status=deferred (Host or domain name not found. Name service error for name=smtp.kerneltalks.com type=AAAA: Host not found)

Solution: This means the server is looking for an IPv6 record for the SMTP server. Edit /etc/postfix/main.cf file and remove all protocols and define only ipv4 and you will be good.

# vi /etc/postfix/main.cf
inet_protocols = ipv4

4 steps guide for SMTP configuration in HPUX

Leave a reply

Learn how to configure SMTP in HPUX machines. Know configuration file locations, parameters in config files, and testing commands.

SMTP is the short name for the Simple Mail Transfer Protocol. SMTP is an Internet standard for electronic mail (e-mail) transmission. We have already seen the SMTP configuration in Linux. In this post, we will be seeing SMTP in HPUX.

Pre-requisite

Hostname and IP address of SMTP server should be known (e.g. we will use mailserver.xyz.com 10.10.2.5)
The client should be able to reach the SMTP server over the network

Configurations

Step 1.

Add SMTP server entry in /etc/hosts file using vi editor or concatenate using echo command below.

# echo "10.10.2.5   mailserver.xyz.com" >> /etc/hosts

# cat /etc/hosts |grep -i mail
10.10.2.5   mailserver.xyz.com

Step 2.

Update daemon configuration file /etc/rc.config.d/mailservs to start the SMTP services with server startup. Edit the file with vi editors and update below entry:

SENDMAIL_SERVER=1

Add the SMTP server detail into SMTP configuration file /etc/mail/sendmail.cf. Edit below two entities

DMxyz.com
Dj<hostname>.com <<Add clients hostname
 DSmailserver.xyz.com
#C{E}root <<hash this entry

Step 3.

Now restart the SMTP daemon to take up this new configuration:

# /sbin/init.d/sendmail stop
Shutting down sendmail          [Done]
Shutting down sm-client         [Done]

# /sbin/init.d/sendmail start
Starting sendmail               [Done]
Starting sm-client              [Done]

Now check if sendmail is running using below command:

# ps -ef |grep -i sendmail
    root  1185     1  0  Nov 11  ?        61:33 sendmail: accepting connections on port 25
    root 21945 21815  0 03:18:28 pts/0     0:00 grep -i sendmail

Step 4.

Send a test mail and verify it.

# echo test | /usr/sbin/sendmail -v info@xyz.com
info@xyz.com... Connecting to mailserver.xyz.com via relay...
220 mailserver.xyz.com  ESMTP Postfix
>>> EHLO xyz.com
250-mailserver.xyz.com 
250-PIPELINING
250-SIZE 25600000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
>>> MAIL From:<root@xyz.com> SIZE=5
250 2.1.0 Ok
>>> RCPT To:<info@xyz.com>
250 2.1.5 Ok
>>> DATA
354 End data with <CR><LF>.<CR><LF>
>>> .
250 2.0.0 Ok: queued as 715C48203C
info@xyz.com... Sent (2.0.0 Ok: queued as 715C48203C)
Closing connection to mailserver.xyz.com
>>> QUIT
221 2.0.0 Bye

If you don’t receive an email, check logs under /var/adm/syslog/mail.log for troubleshooting.

3 tricks to get multiple commands output in the same row

Leave a reply

Three tricks to get two commands output in a single row. Normally two commands outputs will be always separated by a carriage return in Linux terminal.

One of the major hurdles in scripting is to get your outputs formatted well according to requirements. In Linux/Unix each command stdout its output always in a new line. This is a hurdle in many situations where the coder wants two outputs, two terms, two variables in a single row for example CSV format. In this post, we will see how to present two or more commands output in a single row.

Trick 1:

Using the translate function. AS I said earlier each command output is accompanied by a carriage return. We will be translating carriage return with tab character hence getting two or more commands output in a single row.

# (date; hostname) |tr '\n' '\t'
Sat Dec 24 01:36:22 EDT 2016    testsrv2

# (echo text1; echo text2; echo text3) |tr '\n' '\t'
text1   text2   text3

Bypassing outputs to translate function we achieved single row output. Here we instructed tr function to translate all carriage returns (\n) to tab (\t).

If you want to output in CSV format (comma-separated value) then replace tab with a comma and you will get it.

# (date; hostname) |tr '\n' ','
Sat Dec 24 01:43:09 EDT 2016,testsrv2,

# (echo text1; echo text2; echo text3) |tr '\n' ','
text1,text2,text3,

You can observe one trailing comma which shows every command output ends with \n and hence it gets replaced by ,

Trick 2:

By defining each command output as a variable. Here we will store the output of each command in one variable and then we will echo those variables in a single line.

# mydate=`date`

# myname=`hostname`

# echo $myname $mydate
testsrv2 Sat Dec 24 01:46:04 EDT 2016

Here we stored date and hostname command’s output in mydate and myname variables. In the last command, we echoed both variables with space in between. Note that commands output can be stored in a variable by executing the command using backticks (in-line execution)!

Trick 3:

By echoing in-line execution outputs. In-line execution i.e. using backticks to execute commands within another command. We will be using this trick to echo outputs in a single row.

# echo "`date` `hostname`"
Sat Dec 24 01:50:36 EDT 2016 testsrv2

In the above example, first, we have executed commands and got outputs. But those outputs are fed into echo command. Hence echo command stdout them in a single row.

# echo "`echo text1` `echo text2` `echo text3`"
text1 text2 text3

Make a note that we executed each command in separate back tick quotes.

5 steps guide for SMTP configuration in Linux

1 Reply

Learn how to configure SMTP in Linux machines. Know configuration file locations, parameters in config files, and testing commands.

Simple Mail Transfer Protocol. SMTP is one of the important services in any production system. Nowadays, mail notifications became important to monitor systems in real-time. Hence, it becomes essential to know how to configure mail utilities on servers. To enable any mail program on the system, SMTP is a basic component. It’s a very easy and quick procedure to setup SMTP on servers.

Pre-requisite :

Hostname and IP address of SMTP server should be known (e.g. we will use mailserver.xyz.com 10.10.2.5)
The client should be able to reach SMTP server over a network

Configurations :

Its 5 steps procedure to setup SMTP client on Linux.

Step 1.

Add SMTP server entry in /etc/hosts file using vi editor or concatenate using echo command below.

# echo "10.10.2.5   mailserver.xyz.com" >> /etc/hosts
# cat /etc/hosts |grep -i mail
10.10.2.5   mailserver.xyz.com

Step 2.

We need to edit the macro configuration file /etc/mail/sendmail.mc. Search and edit below parameters in this file :

define(`SMART_HOST',`mailserver.xyz.com'): Smart relay server name
define(`confDOMAIN_NAME',`xyz.com')dnl: Valid domain name
FEATURE(always_add_domain)dnl: always masquerades email address
FEATURE(`allmasquerade')dnl: rewrite both from/to relative to the local machine.
MASQUERADE_DOMAIN(`xyz.com.')dnl: Domain for masquerading emails
MASQUERADE_AS(`xyz.com')dnl: All mails masquerades as came from the defined domain

Step 3.

Once the above changes are done, you need to compile the mc file so that all these changes of the macro (mc) file will be transferred to the config ( cf) file using m4 utility.

# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Step 4.

Edit below parameters in /etc/mail/submit.cf so that mails will be submitted to defined domain.

D{MTAHost} mailserver.xyz.com
Djxyz.com: Define this in case of sendmail won’t able to determine your domain.
DSmailserver.xyz.com: Smart relay host

Step 5.

Finally, restart mail services to take up this new configurations :

# service sendmail restart
# service postfix restart

You are done! Send a test email with the below command and check if you receive it.

# echo test |sendmail -v info@xyz.com
[<-] 220 mailserver.xyz.com ESMTP Postfix
[->] HELO testsrv2
[<-] 250 mailserver.xyz.com
[->] MAIL FROM:<root@xyz.com>
[<-] 250 2.1.0 Ok
[->] RCPT TO:<info@xyz.com>
[<-] 250 2.1.5 Ok
[->] DATA
[<-] 354 End data with <CR><LF>.<CR><LF>
[->] Received: by testsrv2 (sSMTP sendmail emulation); Fri, 23 Dec 2016 02:29:07 +0800
[->] From: "root" <root@xyz.com>
[->] Date: Fri, 23 Dec 2016 02:29:07 +0800
[->] test
[->]
[->] .
[<-] 250 2.0.0 Ok: queued as 19F75822B8
[->] QUIT
[<-] 221 2.0.0 Bye

If you don’t receive an email, check logs under /var/log/maillog for troubleshooting.

How to save top command output in file

Leave a reply

A how-to guide to save top command output in a simple text file. A useful tip to share real-time, dynamic command output in a static log file.

TOP is the first command that came to mind whenever someone asks about system resource utilization monitoring. CPU, memory utilization can be easily tracked in this tool. This tool is available to almost every UNIX and Linux flavor.

The top command outputs in a real-time updating ASCII window. Data within this ASCII GUI screen updates every 2 seconds in real-time. So all bits in it change every time and its difficult to share its output to anyone who doesn’t have access to the system. Also, if you want to collect the current output and save for future use then that’s also not possible due to updating data.

For such a situation where you want to save data being shown in the output window, top command gives you options of iterations and output files. You can define how many iterations you need and the time difference between them. It works like iterations and intervals in SAR command. Use the below options to save the output in a file.

-b: batch operation mode
-n: number of iterations
-d: Delay i.e. intervals between iterations (not needed for single iteration)
-f file: Output file in which command saves data (supports in few versions only)

We are using a batch operation mode of the top. By defining, iterations you ask top command to stop execution after that many counts. Normally it runs continuously with a delay of 2 sec. This makes it easy to capture the output in a file using -o option. If -o is not supported then you can simply redirect output to file using redirection operator.

# top -n 1 -b >/tmp/top_output.txt
# cat  /tmp/top_output.txt
top - 02:58:22 up 278 days,  7:19,  1 user,  load average: 0.56, 0.52, 0.56
Tasks: 189 total,   1 running, 188 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.8%us,  4.9%sy,  0.0%ni, 84.0%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  12199956k total,  7774572k used,  4425384k free,   856212k buffers
Swap:  8388604k total,        0k used,  8388604k free,  4787268k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2994 root      RT   0  688m  84m  55m S  7.9  0.7   2998:17 osysmond.bin
 4227 grid      20   0  501m  29m  17m S  2.0  0.3 458:12.36 oracle
 4229 grid      -2   0  500m  29m  17m S  2.0  0.3   1009:43 oracle
 4251 grid      20   0  485m  27m  25m S  2.0  0.2 112:07.76 oracle
 4266 root      20   0 2206m  73m  25m S  2.0  0.6   2519:45 crsd.bin
    1 root      20   0 19400 1552 1228 S  0.0  0.0   0:03.23 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.41 kthreadd
    3 root      20   0     0    0    0 S  0.0  0.0  56:25.42 ksoftirqd/0
----- output clipped -----

In the above example, I ran the top command with a single iteration only. Since its single iteration, I didn’t use a delay option. Also -o option was not supported so I redirected output to a file. You can see the content of the file is the top command output screen!

If you use more than 1 iteration lets say 2. Then there will 2 top output screens concatenated one after another in a log file. So it will be like you have a snapshot of all values for a given period of time for future reference. See example below:

# top -n 2 -d 4 -b >/tmp/top_out.txt
# cat /tmp/top_out.txt
top - 03:07:39 up 278 days,  7:28,  1 user,  load average: 0.43, 0.51, 0.54
Tasks: 191 total,   1 running, 190 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.8%us,  4.9%sy,  0.0%ni, 84.0%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  12199956k total,  7762352k used,  4437604k free,   856212k buffers
Swap:  8388604k total,        0k used,  8388604k free,  4774904k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4213 grid      -2   0  485m  14m  12m S  2.0  0.1   5601:55 oracle
    1 root      20   0 19400 1552 1228 S  0.0  0.0   0:03.23 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.41 kthreadd
    3 root      20   0     0    0    0 S  0.0  0.0  56:25.58 ksoftirqd/0
    6 root      RT   0     0    0    0 S  0.0  0.0 399995:22 migration/0
    7 root      RT   0     0    0    0 S  0.0  0.0   1:17.98 watchdog/0
    8 root      RT   0     0    0    0 S  0.0  0.0 399911:47 migration/1
   10 root      20   0     0    0    0 S  0.0  0.0  31:40.41 ksoftirqd/1
   11 root      20   0     0    0    0 S  0.0  0.0  19:42.29 kworker/0:1
  
----- output clipped -----

top - 03:07:44 up 278 days,  7:28,  1 user,  load average: 0.48, 0.51, 0.55
Tasks: 193 total,   4 running, 189 sleeping,   0 stopped,   0 zombie
Cpu(s): 24.5%us, 10.1%sy,  0.0%ni, 65.1%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  12199956k total,  7803692k used,  4396264k free,   856212k buffers
Swap:  8388604k total,        0k used,  8388604k free,  4774968k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2466 root      20   0  129m  25m 1288 S  6.6  0.2   8235:08 OSWatcher.sh
15104 root      20   0  129m  24m  344 R  1.8  0.2   0:00.09 OSWatcher.sh
 4213 grid      -2   0  485m  14m  12m S  1.4  0.1   5601:55 oracle
 2994 root      RT   0  688m  84m  55m S  1.0  0.7   2998:22 osysmond.bin
 3038 grid      RT   0 1608m 116m  53m S  0.8  1.0   2379:27 ocssd.bin
 4385 root      20   0 2543m  42m  13m S  0.8  0.4   3013:44 orarootagent.bi
----- output clipped -----

You can even put up a cron to save the output of top commands to some log file. But this is not recommended since it will pile up logfile in no time and hence fill mount point rapidly. Anyway, we have in-depth utilization data being collected by OS which can be viewed using SAR utility.

-f option which is available in HPUX native top utility has few limitations. It stores only the first 16 lines of output to file and it supports only one iteration. 16 lines limitations make logfile quite short and it shows mainly header values CPU, memory, etc. Most of the processes related to data get lost. See example below :

$ top -f /tmp/test
$ cat /tmp/test
System: testsrv3 Wed Dec 21 03:18:19 2016
Load averages: 0.03, 0.04, 0.04
246 processes: 168 sleeping, 78 running
Cpu states:
CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS
 0    0.04   0.0%   0.0%   0.2%  99.8%   0.0%   0.0%   0.0%   0.0%
14    0.03   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%
15    0.03   0.0%   0.0%   0.2%  99.8%   0.0%   0.0%   0.0%   0.0%
16    0.04   0.0%   0.0%   0.4%  99.6%   0.0%   0.0%   0.0%   0.0%
17    0.02   0.2%   0.0%   0.4%  99.4%   0.0%   0.0%   0.0%   0.0%
18    0.04   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%
---   ----  -----  -----  -----  -----  -----  -----  -----  -----
avg   0.03   0.0%   0.0%   0.2%  99.8%   0.0%   0.0%   0.0%   0.0%

System Page Size: 4Kbytes
Memory: 1679640K (1411160K) real, 3170312K (2643480K) virtual, 30532328K free  P
CPU TTY  PID USERNAME PRI NI   SIZE    RES STATE    TIME %WCPU  %CPU COMMAND

sar command (Part III) : Disk, Network reporting

Leave a reply

Learn System Activity Report sar command with real-world scenario examples. Understand how to do disk, network reporting using this command.

In the last two parts of the sar command, we have seen time formats to be used with command, its data files, CPU & Memory reporting. In this last part, we will be seeing disk, network reporting using sar command.

Read last two parts of this tutorial here :

Disk IO reporting

sar provides disk (block devices) report with -d option. Normally, it shows below parameters values (highlighted values are more commonly observed for performance monitoring) :

DEV: Block device name. It follows the dev m-n format. M is major and n is a minor number of the block devices.
tps: Transfers per second
rd_sec/s: Sector reads per second (sector is 512 byte)
wr_sec/s: Sector writes per second
avgrq-sz: average size (in sectors) of the requests that were issued to the device
avgqu-sz: average queue length of the requests that were issued to the device
await: The average time (in milliseconds) for I/O requests
svctm: The average service time (in milliseconds) for I/O requests
%util: Percentage of CPU time during which I/O requests were issued to the device

# sar -d 2 3
Linux 2.6.39-200.24.1.el6uek.x86_64 (testsrv2)         12/21/2016      _x86_64_        (4 CPU)

01:20:19 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
01:20:21 AM    dev8-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:21 AM   dev8-64      3.00      2.00      1.00      1.00      0.00      0.50      0.50      0.15
01:20:21 AM   dev8-32      3.00      2.00      1.00      1.00      0.00      0.50      0.33      0.10
01:20:21 AM  dev252-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:21 AM  dev252-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:21 AM  dev252-2      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

In the above output, device names are not so user friendly. So to identify devices easily, -p option is available. This option prints pretty device names in the DEV column and it should always be used with -d option.

# sar -d -p 2 3
Linux 2.6.39-200.24.1.el6uek.x86_64 (testsrv2)         12/21/2016      _x86_64_        (4 CPU)

01:20:38 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
01:20:40 AM       sda      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:40 AM       sdb      4.98      0.00     91.54     18.40      0.00      0.80      0.20      0.10
01:20:40 AM       sdc      2.99      1.99      1.00      1.00      0.00      0.67      0.67      0.20
01:20:40 AM      dm-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:40 AM      dm-1      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:40 AM      dm-2      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Now see above output where device names are easily identifiable. sda means disk /dev/sda and so on.

Network utilization reporting

Option -n gives all network stats. There are a total of 18 different keywords (like NFS, IP, DEV, TCP, etc.) which can be used with -n option to get related parameters. Each keyword has almost 8-10 parameters to display. So in the call, if you are using ALL keyword, then the output will be a huge list of parameters which is difficult to understand.

To keep it short here we will see only one example of keyword DEV i.e. device. This will show the device i.e. network card’s parameter values. Most of the time NIC performance is checked hence we are using this keyword example.

# sar -n DEV 2 1
Linux 2.6.39-200.24.1.el6uek.x86_64 (textsrv2)         12/21/2016      _x86_64_        (4 CPU)

01:35:22 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
01:35:24 AM        lo      6.00      6.00      0.29      0.29      0.00      0.00      0.00
01:35:24 AM      eth0     15.50      0.50      0.91      0.04      0.00      0.00      0.00
01:35:24 AM      eth1      6.50      4.50      0.97      0.77      0.00      0.00      0.00
01:35:24 AM      eth3      0.00      0.00      0.00      0.00      0.00      0.00      0.00

Average:        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
Average:           lo      6.00      6.00      0.29      0.29      0.00      0.00      0.00
Average:         eth0     15.50      0.50      0.91      0.04      0.00      0.00      0.00
Average:         eth1      6.50      4.50      0.97      0.77      0.00      0.00      0.00
Average:         eth3      0.00      0.00      0.00      0.00      0.00      0.00      0.00

In the above example, I used the DEV keyword along with -n option and took only one iteration in output. Parameters displayed for device keyword are :

IFACE: Its interface name. You can easily see eth0, eth1, loopback (lo) interfaces here.
rxpck/s: Packets received per seconds
txpck/s: packets transmitted per second
rxkB/s: kilobytes received per second
txkB/s : kilobytes transmitted per second
rxcmp/s: compressed packets received per second
txcmp/s: compressed packets transmitted per second
rxmcst/s: Number of multicast packets received per second

This concludes sar command tutorial’s part III about the disk, network reporting. This is a three-part tutorial with example outputs included. Put your queries, suggestions, feedback in the comments below. You can also reach us using our Contact form.