Category Archives: HPUX

Run levels in HPUX at a glance

Learn the list of different run levels in HPUX and their roles. Also, see how to check the current run level in which the system is running.

A run level is the state of a system depending on which system services will spawn. Normally lower run levels are having fewer services available for the user and mainly used for administrative purposes. Higher levels have more services available and targets end user’s use. In HPUX highest run levels like 5 and 6 are kept reserved for future purposes. We will see the list of run levels and their offerings in the following article.

Current run level in HPUX can be identified using the below command :

# who -r
   .       run-level 3  Jan 19 21:14    3    0    S

The output fields of the above commands are as below:
1. A dot . indicates that the terminal has seen activity in the last minute and is therefore its. i.e. current.
2. Current run-level
3. Timestamp
4. The current state of init
5. The number of times that state has been previously entered
6. The previous state

Read also: Different usage of ‘who’ command.

List of run levels in HPUX

0 indicates shutdown state
S indicates single user mode booted to local console only with root FC (RO) mounted
s indicates the same as S only current terminal acts as system console.
1 indicates the single-user mode with local FS (RW) mounted
2 indicates multi-user state with CDE launched
3 indicates the same as 2 but with NFS
4 indicates GUI (here VUE started instead of CDE)
5,6 indicates reserved to state and not yet defined in kernel code.

How to restart NFS in HPUX

Step by step procedure to restart NFS services in HPUX. Follow this procedure with a given sequence to stop and start NFS gracefully.

Requirement :

To restart NFS server in HPUX

How to do it :

Please make a note that all exported NFS mount points will be unavailable to all clients during this restart.

Stop NFS

# /sbin/init.d/nfs.server stop
killing nfsd
killing rpc.mountd
# /sbin/init.d/nfs.client stop
killing nfs4cbd
# /sbin/init.d/nfs.core stop
killing nfsmapid
killing rpcbind

Read also :

Start NFS

# /sbin/init.d/nfs.core start
    Starting NFS CORE networking

    Starting up the rpcbind
        /usr/sbin/rpcbind
# /sbin/init.d/nfs.client start
    Starting NFS CLIENT subsystem

    Starting up nfs4cbd daemon
        /usr/sbin/nfs4cbd
      Starting up nfsmapid daemon
        /usr/sbin/nfsmapid
    Mounting remote NFS file systems ...
    Mounting remote CacheFS file systems ...
# /sbin/init.d/nfs.server start
    Starting NFS SERVER subsystem

    Reading in /etc/dfs/dfstab
    Starting up the mount daemon
        /usr/sbin/rpc.mountd
    Starting up the NFS server daemon
        /usr/sbin/nfsd
      Starting up nfsmapid daemon

Make sure you follow the sequence while stopping and starting as mentioned above.

bdf command formatted output in hpux

Learn to get the neat, clean and tabbed output of bdf.  This left aligned and properly formatted bdf output is helpful for easier data processing.

Requirement :

bdf command output normally looks scattered especially when VG names are long. It will be difficult to grep out a proper pattern out of such output. Also, it’s not convenient to share this output over email/document when extra lines break exists.

In such scenarios, we need to have a properly formatted output of bdf. Also sometimes we require output with all its columns left-aligned.

Solution:

To remove line breaks from bdf output and get single row per entry output

See below normal bdf output. Note that the last 2 mount points have two-line entry since the filesystem column has long entry.

# bdf

Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    2097152  737416 1349304   35% /
/dev/vg00/lvol1    1048576  206160  835928   20% /stand
/dev/vg00/lvol8    8388608 5475640 2902568   65% /var
/dev/vg00/lvol7    8388608 4655256 3713000   56% /usr
/dev/vg00/lvol4    2097152 1052368 1036888   50% /tmp
/dev/vg00/lvol6    8388608 6675168 1700112   80% /opt
/dev/vg00/lvol5     524288   49360  471256    9% /home
testserver01:/data
                   50574008 4541896 43463104    9% /data
/dev/vgdata/lvol1
                   918421504 591931608 306084338   66% /datastore

Now with inline awk we format the output to have one entry per row. Check below command output.

# bdf | awk '{if (NF==1) {line=$0;getline;sub(" *"," ");print line$0} else {print}}'

Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    2097152  737408 1349312   35% /
/dev/vg00/lvol1    1048576  206160  835928   20% /stand
/dev/vg00/lvol8    8388608 5475640 2902568   65% /var
/dev/vg00/lvol7    8388608 4655256 3713000   56% /usr
/dev/vg00/lvol4    2097152 1052368 1036880   50% /tmp
/dev/vg00/lvol6    8388608 6675168 1700112   80% /opt
/dev/vg00/lvol5     524288   49360  471256    9% /home
testserver01:/data 50574008 4541896 43463104    9% /data
/dev/vgdata/lvol1 918421504 591931608 306084338   66% /datastore

To get left aligned bdf output

In the above output, columns are not aligned properly. We can even do that with the below argument.

# bdf | awk '///{printf("%-30s%-10s%-10s%-10s%-5s%-10sn",$1,$2,$3,$4,$5,$6)}'

/dev/vg00/lvol3               2097152   737408    1349312   35%  /
/dev/vg00/lvol1               1048576   206160    835928    20%  /stand
/dev/vg00/lvol8               8388608   5472792   2905392   65%  /var
/dev/vg00/lvol7               8388608   4655256   3713000   56%  /usr
/dev/vg00/lvol4               2097152   1052368   1036888   50%  /tmp
/dev/vg00/lvol6               8388608   6675168   1700112   80%  /opt
/dev/vg00/lvol5               524288    49360     471256    9%   /home

Please make a note that this awk won’t remove any line breaks from the output. So one can combine (with pipe |) both awk to get left aligned output with line breaks removed.

Left-aligned output with line breaks removed!

# bdf | awk '{if (NF==1) {line=$0;getline;sub(" *"," ");print line$0} else {print}}' |awk '///{printf("%-30s%-10s%-10s%-10s%-5s%-10sn",$1,$2,$3,$4,$5,$6)}'
/dev/vg00/lvol3               2097152   737408    1349312   35%  /
/dev/vg00/lvol1               1048576   206160    835928    20%  /stand
/dev/vg00/lvol8               8388608   5481008   2897240   65%  /var
/dev/vg00/lvol7               8388608   4655256   3713000   56%  /usr
/dev/vg00/lvol4               2097152   1052368   1036888   50%  /tmp
/dev/vg00/lvol6               8388608   6675168   1700112   80%  /opt
/dev/vg00/lvol5               524288    49360     471256    9%   /home
testserver01:/data            50574008  4541896   43463104  9%   /data
/dev/vgdata/lvol1             918421504 591931608 306084338 66%  /datastore

					

How to change sender’s email id in EMS HPUX

EMS doesn’t allow us to edit the sender email address. All events are always sent out with sender id as root@hostname. Here is a small script to change it.

Requirement :

Normally in the Event monitoring system ( EMS ) on HPUX send an email with sender id as root@hostname. Many organization’s email servers don’t allow such an email address in the sender field. We need generic email id in sender field when EMS shoots an alert email something like notification@xyz.com

Workaround :

There is no provision to change this email id anywhere in HPUX or EMS configurations. You can use the below workaround which works perfectly without any issues.

Step 1 :

Make sure you have a valid email address (like notification@xyz.com) for your logged-in account which works. Send a test email from the server to verify using below command

# echo test |/usr/sbin/sendmail -v receiver_id@xyz.com
receiver_id@xyz.com... Connecting to smtpserver.xyz.com via relay...
220 smtpserver.xyz.com ESMTP Postfix
>>> EHLO xyz.com
250-smtpserver.xyz.com
250-PIPELINING
250-SIZE 25600000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
>>> MAIL From:<root@xyz.com> SIZE=5
250 2.1.0 Ok
>>> RCPT To:<receiver_id@xyz.com>
>>> DATA
250 2.1.5 Ok
354 End data with <CR><LF>.<CR><LF>
>>> .
250 2.0.0 Ok: queued as 46466822F2
receiver_id@xyz.com... Sent (Ok: queued as 46466822F2)
Closing connection to smtpserver.xyz.com
>>> QUIT
221 2.0.0 Bye

Step 2 :

Setup crontab for above logged in the account (for which email tested) which will execute the EMS log scanner script every 30 minutes. As per your convenience, you can even schedule it to run every 10 mins or even lower.

00,30 * * * * /scripts/ems_monitor.sh

Step 3:

The script code is as below.

# Script to scan EMS log file and email alert if any
# Author : Shrikant Lavhate
#! /bin/bash
if [ -f "https://z5.kerneltalks.com/logs/event_monitor.log" ]
then
:
else
cp -p /var/opt/resmon/log/event.log /logs/event_monitor.log
fi
diff /logs/event_monitor.log /var/opt/resmon/log/event.log /logs/logfile_difference
if [ -s "/logs/logfile_difference" ]
then
cat /logs/logfile_difference | grep '^'|cut -c 2- | mailx -s "EMS monitor alert from `hostname`" receiver_id@xyz.com
fi
cp -p /var/opt/resmon/log/event.log /logs/event_monitor.log

Step 4:

Now you can test the script by generating test events in EMS. Generate test event with send_test_event command.

# send_test_event -v -a disk_em

Finding resource name associated with monitor disk_em.

Found resource name /storage/events/disks/default
associated with monitor disk_em.

Creating test file /var/stm/config/tools/monitor/disk_em.test
for monitor disk_em.

Making test file /var/stm/config/tools/monitor/disk_em.test
for monitor disk_em
indicate send test event for all resources.

Performing resls on resource name /storage/events/disks/default
for monitor disk_em to cause generation of test event.
Contacting Registrar on aprss006

NAME:   /storage/events/disks/default
DESCRIPTION:    Disk Event Monitor

This resource monitors events for stand-alone disk drives.  Event 
monitoring requests are created using the Monitoring Request 
Manager.  Monitoring requests to detect changes in device status are 
created using the Peripheral Status Monitor (psmmon(1m)) and Event 
Monitoring Service (EMS). 

For more information see the monitor man page, (disk_em(1m)).

TYPE:   /storage/events/disks/default is a Resource Class.

There are 2 resources configured below /storage/events/disks/default:
Resource Class
        /storage/events/disks/default/64000_0xfa00_0x0
        /storage/events/disks/default/64000_0xfa00_0x35

You will receive test events from admin@hostname email id which is normal. Now run above script and you will receive the same email with sender id as notification@xyz.com !!

I have tested this script and it works flawlessly. Please leave comments below if it’s helpful to you too.

Get list of desired LUN id from powermt output

Small script to get LUN id from powermt output when supplied with a list of disks. It’s a tedious job to extract LUN id from the output when you have a list of disks to wok upon.

Requirement :

You have a list of disk names from OS end and you need to get their respective LUN ids from powermt output.

This requires manual work of searching each disk name in output and then copying its respective LUN id. Typically these three lines you are interested in powermt output.

Pseudo name=emcpoweraa
Symmetrix ID=000549754319
Logical device ID=03C4

If you have a list of disks to search its a tedious task. Because you have to search each and every disk name in the output.

Solution :

Get the output of powermt command in a file

# powermt display dev=all > powermt.old

Get all disk names in one file e.g. test.txt
Run a for loop which will get LUN id of each disk described in the file. In this loop, we are taking each disk and searching it in the above file. Then we are extracting 2 below lines from disk name (since 2nd line below disk name contains LUN id). And then extracting LUN id from it with some data filtering.

# for i in `cat test`
do
cat powermt.old |grep -A 2 $i|grep Logical|awk '{print $3}'|cut -d= -f2
done

You will be presented with the list of LUN ids for respective disks in the test.txt file! You can even echo disk name before LUN id by inserting echo $i just above cat command in the above code.

Vice versa:

Get disk names by giving LUN ids in text.txt file. Its same logic, the only difference is we will be extracting above 2 lines rather than below ones. Rest all procedure remains the same.

# for i in `cat test`
do
cat powermt.old |grep -B 2 $i|grep Pseudo|awk '{print $2}'|cut -d= -f2
done

Processes using high memory, cpu using unix95

Learn how to get a list of processes consuming high memory or CPU on the kernel in sorted order using unix95. Helpful for troubleshooting high system utilization.


Listing of processes who are using high memory :

# UNIX95=1 ps -ef -o vsz= -o pid= -o comm= | sort -rnk1 | awk '{ print $1/1024" MB "$2" "$3; }'
749.156 MB 2025 cimprovagt
60.1875 MB 2842 midaemon
56.8672 MB 3616 opcmona
49.3633 MB 6905 cmcld
43.8906 MB 3485 coda
42.9375 MB 2035 cimprovagt
41.043 MB 2372 java
22.875 MB 2039 cimprovagt
21.2578 MB 3414 nsrexecd
18.2617 MB 2617 vxpal
17.2188 MB 2431 vxsvc
16.9297 MB 3264 postgres:

----- output truncated -----

or

# UNIX95= ps -eo vsz,comm,args | sed 1d | sort -rn | more
 767136 cimprovagt      /opt/wbem/lbin/cimprovagt 0 4 9 root SFMProviderModule
  61632 midaemon        /opt/perf/bin/midaemon
  58232 opcmona         /opt/OV/lbin/eaagt/opcmona
  50548 cmcld           /usr/lbin/cmcld -j
  44944 coda            /opt/OV/lbin/perf/coda
  43968 cimprovagt      /opt/wbem/lbin/cimprovagt 0 4 9 root HPUXFCIndicationProviderModule
  42028 java            /opt/hpservices/jre/opt/java1.4/jre/bin/IA64N/java -Xmx512m -DRunnerValuePairFile=/var/opt/runner/data/valuePairs -cp /opt/runn
  23424 cimprovagt      /opt/wbem/lbin/cimprovagt 0 4 9 root HPUXStorageIndicationProviderModule
  21768 nsrexecd        /opt/networker/bin/nsrexecd
  18700 vxpal           /opt/VRTSobc/pal33/bin/vxpal -a StorageAgent -x
  17632 vxsvc           /opt/VRTSob/bin/vxsvc -r /etc/vx/isis/Registry -e

----- output truncated -----





Listing of processes who are using high CPU:

# UNIX95= ps -e -o "vsz pcpu ruser pid stime time state args" | sort -rn |head -10
 767136  0.00 root      2025  Nov 15  1-04:47:22 R /opt/wbem/lbin/cimprovagt 0 4 9 root SFMProviderModule
  61632  0.07 root      2842  Nov 15  11:29:30 R /opt/perf/bin/midaemon
  58232  0.00 root      3616  Nov 15  05:52:05 R /opt/OV/lbin/eaagt/opcmona
  50548  0.14 root      6905  Nov 15  1-05:23:35 R /usr/lbin/cmcld -j
  44944  0.00 root      3485  Nov 15  04:46:17 R /opt/OV/lbin/perf/coda
  43968  0.00 root      2035  Nov 15  05:53:58 R /opt/wbem/lbin/cimprovagt 0 4 9 root HPUXFCIndicationProviderModule
  42028  0.09 root      2372  Nov 15  22:59:12 R /opt/hpservices/jre/opt/java1.4/jre/bin/IA64N/java -Xmx512m -DRunnerValuePairFile=/var/opt/runner/data/valuePairs -cp /opt/runn
  23424  0.00 root      2039  Nov 15  02:26:51 R /opt/wbem/lbin/cimprovagt 0 4 9 root HPUXStorageIndicationProviderModule
  21768  0.00 root      3414  Nov 15  01:22:45 R /opt/networker/bin/nsrexecd
  18700  0.00 root      2617  Nov 15  01:04:36 R /opt/VRTSobc/pal33/bin/vxpal -a StorageAgent -x