Monthly Archives: November 2016

Host to guest disk mapping in HP iVM

Learn how to identify virtual machine disk on the physical host machine in HP iVM. Disk mapping makes it easy to carry out disk related activities.

In HP integrity virtual machines, disk names on the host machines and virtual machines are always different for the same disk. Whenever we are presenting disk (storage LUN or local disk) from host to guest, it will be discovered as a different name on guests than a host. So it becomes necessary to know both names of the same disk for any disk-related activities.

Let see how we can map these two names. There are two methods to do this.

  1. Using xD command
  2. With hpvmdevinfo command

Using xD command

xD command used to read raw data on disk. Since the physical disk is the same on both servers only identification at kernel level differs, we will get the same raw data from both servers. We will use xD command to get PVID of disks from the host and guest. Whenever there is a match of PVID in both outputs, consider the disk is the same.

See below example where xD command is used with host and guest disks.

------ On guest -----
vm:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk76
70608a28 4ec7a7ff 70608a28 4ec7a942
vm:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk72
70608a28 4ec7a7ef 70608a28 4ec7a942
vm:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk74
70608a28 4ec7a7f6 70608a28 4ec7a942

----- On host -----
host:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk532
70608a28 4ec7a7ff 70608a28 4ec7a942
host:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk538
70608a28 4ec7a7f6 70608a28 4ec7a942
host:root-/>xd -An -j8200 -N16 -tx /dev/disk/disk526
70608a28 4ec7a7ef 70608a28 4ec7a942

Now, if you observe outputs (2nd field), guest disk disk76 has the same value as host disk disk532. That means its the same disk! So on host diusk532 is the same as disk76 on the guest. Same with disk538-disk74 & disk 523-disk72.

This is a bit of a tedious job to observe outputs and find a match if you have a huge number of disks. Also, if you are interested in only one VM’s data then its time consuming since you have to match all disks of the host with that VM’s disks. In that case we have hpvmdevinfo command which directly prints out mapping table for you.

With hpvmdevinfo command

This command comes with an HP iVM setup and shows device mappings from host to guest in tabular format. Since this command can be run against a particular VM, it’s pretty fast to get disk mapping than the previous method.

# hpvmdevinfo -P virtual_svr_2
Virtual Machine Name Device Type Bus,Device,Target Backing Store Type Host Device Name Virtual Machine Device Name
==================== =========== ================= ================== ================ ===========================
virtual_svr_2           disk            [0,1,0]          disk         /dev/rdisk/disk336 /dev/rdisk/disk4
virtual_svr_2           disk            [0,1,1]          disk         /dev/rdisk/disk332 /dev/rdisk/disk5
virtual_svr_2           disk            [0,1,3]          disk         /dev/rdisk/disk675 /dev/rdisk/disk9

You need to run this command by supplying VM name with -P option and you will be presented with device list, its ctd and disk mapping between host-guest servers.

In the above example, see the last two columns where the first one shows disk name on the host machine and last one shows guest/virtual machine. Pretty straight forward and fast!

NFS configuration in Linux and HPUX

Learn how to configure the network file system (NFS) in Linux and HPUX servers. Howto export NFS, start/stop NFS services, and control access on it.

NFS configurations

The network file system is one of the essential things in today’s IT infrastructure. One server’s file system can be exported as NFS over network and control access over it. Other servers can mount these exported mount points locally as an NFS mount. This enables the same file system making available to many systems thus many users. Let’s see NFS configurations in Linux and HPUX.

NFS Configuration file in Linux

We assume the NFS daemon is installed on a server and running in the background. If not check package installation steps and how to start service on Linux. One can check if NFS is running on the server with service or ps -ef command. For NFS server i.e. server exporting directory should have portmap service running.

Make sure you have TCP and UDP port 2049, 111 on firewalls between client and server. It can be in OS firewall, iptables, network firewalls, or security groups in the cloud.

root@kerneltalks # ps -ef |grep -i nfs
root      1904     2  0  2015 ?        00:00:08 [nfsd4]
root      1905     2  0  2015 ?        00:00:00 [nfsd4_callbacks]
root      1906     2  0  2015 ?        00:01:33 [nfsd]
root      1907     2  0  2015 ?        00:01:32 [nfsd]
root      1908     2  0  2015 ?        00:01:33 [nfsd]
root      1909     2  0  2015 ?        00:01:37 [nfsd]
root      1910     2  0  2015 ?        00:01:24 [nfsd]

root@kerneltalks # service nfs status
rpc.svcgssd is stopped
rpc.mountd (pid 1897) is running...
nfsd (pid 1913 1912 1911 1910 1909 1908 1907 1906) is running...
rpc.rquotad (pid 1892) is running...

root@kerneltalks # rpcinfo -p localhost
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
----- output clipped -----

/etc/exports is the configuration file which has all exported volume details along with their respective permissions. /etc/exports follows the format as below :

<export> <host> (options)

where –

  • export is filesystem/directory to be exported
  • the host is hostname/IP to which export is accessible where wile cards are acceptable
  • options are permissions which are ro, rw, sync, async.

Refer below chart which can be used to decide your entry in this file.

NFS config file parameters

Parameter
Example
/my_share server3 (rw, sync) Export /my_share directory for server3 with read write access in sync mode
/my_share * (ro, sync) Export /my_share for any host with read only permission and sync mdoe
/my_share 10.10.2.3 (rw,async) Export /my_share for IP 10.10.2.3 with rw in async.
/my_share server2 (ro, sync) server3 (rw, sync) Exporting to two diff servers with diff permissions
root@kerneltalks # cat /etc/exports
/my_share       10.10.15.2(rw,sync)
/new_share       10.10.1.40(rw,sync)

/etc/exports file can be edited using vi editor or using /usr/sbin/exportfs command.

How to start-stop NFS service in Linux

Once you made the changes in the file you need to restart NFS daemon to take all these changes in effect. This can be done using the service NFS restart command. If your NFS is already running and you just need to take a new configuration in action you can reload config using service NFS reload. To stop NFS you can run service nfs stop command.

root@kerneltalks # service nfs status
rpc.svcgssd is stopped
rpc.mountd (pid 1897) is running...
nfsd (pid 1913 1912 1911 1910 1909 1908 1907 1906) is running...
rpc.rquotad (pid 1892) is running...

How to re-export NFS shares after editing the configuration file

In running the NFS environment, where multiple clients already mounted NFS shares from the NFS server and you need to edit NFS share configuration. You can edit the NFS configuration file and re-export NFS shares using exportfs command.

Make sure only additions happened to config file while reloading config otherwise it may affect already connected NFS shares.

root@kerneltalks # exportfs -ra

How to mount NFS share

At the destination, where export needs to be mounted should have NFS daemon running too. Mounting a share is a very easy two-step procedure.

  1. Create a directory to mount the share
  2. mount share using the mount command.

To make this permanent i.e. mounting share at boot time, make an entry to /etc/fstab like below so that manually mounting after the reboot of server can be avoided.

10.10.2.3:/my_share /tmp/nfs_share nfs defaults 0 0

NFS configuration in HPUX

This part is the same as Linux. In some versions, you need to edit /etc/dfs/dfstab file. This file takes share commands as a per line entry. It can be filled like below :

share -F nfs -o root=server2:server3 /my_share

Above line indicates exporting /my_share directory for server2 and server3 with root account access.

Also, we need to specify NFS_SERVER=1 parameter in /etc/rc.config.d/nfsconf on the NFS server. By default, it is set to 0 i.e. server acts as NFS client. Along with this NFS_CORE and START_MOUNTD needs to be marked to value 1 as well.

How to start-stop NFS service in HPUX

We have covered it here: NFS server start/stop on HPUX

For reloading config file in HPUX, you can run shareall command.

Mounting share

This part is the same as Linux

Errors seen in Linux

If you did not prepare client properly then you might see below error :

mount: wrong fs type, bad option, bad superblock on 10.10.2.3:/mys_hare,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

Install nfs-utils, nfs-common packages and you should be able to mount NFS filesystem without any issues.

Howto get CPU details in HPUX

Learn how to extract CPU, core, socket details in HPUX. Get familiar with hardware related commands like print_manifest, machinfo, cstm, ioscan.

There are many times when one has to check CPU details of the server which are like the number of cores, sockets, etc. This detail is useful in capacity planning, troubleshooting, performance analysis, etc. There are many ways to get these details as below –

  • ioscan command
  • cstm tool
  • print_manifest report
  • machinfo command
  • MP console
  • top command
  • SAR output

Before going into these methods make sure you check if logical CPUs are enabled on server or not. If yes then you need to take that into consideration while calculating the number of CPUs.

Let’s see the above methods one by one.

ioscan command

This is a well-known command to every HPUX server administrator. To scan or list hardware on the system, we use this command. Filtering processors out of this command give you details about CPU. This is helpful to get the only number of processors on the system.

# ioscan -kfnC processor
Class       I  H/W Path  Driver    S/W State H/W Type  Description
===================================================================
processor   0  128       processor CLAIMED   PROCESSOR Processor
processor   1  129       processor CLAIMED   PROCESSOR Processor
processor   2  136       processor CLAIMED   PROCESSOR Processor
processor   3  137       processor CLAIMED   PROCESSOR Processor
processor   4  144       processor CLAIMED   PROCESSOR Processor
processor   5  145       processor CLAIMED   PROCESSOR Processor
processor   6  152       processor CLAIMED   PROCESSOR Processor
processor   7  153       processor CLAIMED   PROCESSOR Processor

# ioscan -kfnC processor | grep processor|wc -l
8

cstm tool

CSTM is another famous tool native to HPUX used to deal with hardware. This will give you in-depth details about each processor on the system. Type /usr/sbin/cstm and you will be on CSTM shell. Here type is below command :

cstm>selclass qualifier cpu;infolog
-- Converting multiple raw log files to text. --
Preparing the Information Tool Log for each selected device...

.... server1  :  10.10.11.1 ....

-- Information Tool Log for CPU on path 128 --

Log creation time: Wed Jul 20 11:09:00 2016

Hardware path: 128


Product ID:                CPU          Module Type:              0
Hardware Model:            0x894        Software Model:           0x4
Hardware Revision:         0            Software Revision:        0
Hardware ID:               0            Software ID:              3848593997
Boot ID:                   0x2          Software Option:          0x91
Processor Number:          0            Path:                     128
Hard Physical Address:     0xfffffffffe780000     Soft Physical Address:    0

Slot Number:               0            Software Capability:      0x100000f0
PDC Firmware Revision:     46.34        IODC Revision:            0
Instruction Cache [Kbyte]: 768          Processor Speed:          N/A
Processor State:           CPU Present Configured
Monarch:                   Yes          Active:                   Yes
Data Cache        [Kbyte]: 768
Instruction TLB   [entry]: 240          Processor Chip Revisions: 3.2
Data TLB Size     [entry]: 240          2nd Level Cache Size:[KB] 65536
Serial Number:             44549e6cf43f0605


-----------------  Processor 0 HPMC Information - PDC Version: 46.34  ------

   * * * No valid timestamp * * *
       No HPMC chassis codes logged


General Registers 0 - 31
00-03  0000000000000000  0000000000000000  0000000000000000  0000000000000000
04-07  0000000000000000  0000000000000000  0000000000000000  0000000000000000
08-11  0000000000000000  0000000000000000  0000000000000000  0000000000000000
12-15  0000000000000000  0000000000000000  0000000000000000  0000000000000000
16-19  0000000000000000  0000000000000000  0000000000000000  0000000000000000
20-23  0000000000000000  0000000000000000  0000000000000000  0000000000000000
24-27  0000000000000000  0000000000000000  0000000000000000  0000000000000000
28-31  0000000000000000  0000000000000000  0000000000000000  0000000000000000



Control Registers 0 - 31
00-03  0000000000000000  0000000000000000  0000000000000000  0000000000000000
04-07  0000000000000000  0000000000000000  0000000000000000  0000000000000000
08-11  0000000000000000  0000000000000000  0000000000000000  0000000000000000
12-15  0000000000000000  0000000000000000  0000000000000000  0000000000000000
16-19  0000000000000000  0000000000000000  0000000000000000  0000000000000000
20-23  0000000000000000  0000000000000000  0000000000000000  0000000000000000
24-27  0000000000000000  0000000000000000  0000000000000000  0000000000000000
28-31  0000000000000000  0000000000000000  0000000000000000  0000000000000000


Space Registers 0 - 7
00-03  0000000000000000  0000000000000000  0000000000000000  0000000000000000
04-07  0000000000000000  0000000000000000  0000000000000000  0000000000000000


IIA Space (back entry)       = 0x0000000000000000
IIA Offset (back entry)      = 0x0000000000000000
Check Type                   = 0x00000000
Cpu State                    = 0x00000000
Cache Check                  = 0x00000000
TLB Check                    = 0x00000000
Bus Check                    = 0x00000000
Assists Check                = 0x00000000

Assist State                 = 0x00000000
Path Info                    = 0x00000000
System Responder Address     = 0x0000000000000000
System Requestor Address     = 0x0000000000000000



Floating Point Registers 0 - 31
00-03  0000000000000000  0000000000000000  0000000000000000  0000000000000000
04-07  0000000000000000  0000000000000000  0000000000000000  0000000000000000
08-11  0000000000000000  0000000000000000  0000000000000000  0000000000000000
12-15  0000000000000000  0000000000000000  0000000000000000  0000000000000000
16-19  0000000000000000  0000000000000000  0000000000000000  0000000000000000
20-23  0000000000000000  0000000000000000  0000000000000000  0000000000000000
24-27  0000000000000000  0000000000000000  0000000000000000  0000000000000000
28-31  0000000000000000  0000000000000000  0000000000000000  0000000000000000


PIM Revision                 = 0x0000000000000000
CPU ID                       = 0x0000000000000000
CPU Revision                 = 0x0000000000000000
Cpu Serial Number            = 0x0000000000000000
Check Summary                = 0x0000000000000000
SAL Timestamp                = 0x0000000000000000
System Firmware Rev.         = 0x0000000000000000
PDC Relocation Address       = 0x0000000000000000
Available Memory             = 0x0000000000000000
CPU Diagnose Register 2      = 0x0000000000000000
MIB_STAT                     = 0x0000000000000000
MIB_LOG1                     = 0x0000000000000000
MIB_LOG2                     = 0x0000000000000000
MIB_ECC_DATA                 = 0x0000000000000000
ICache Info                  = 0x0000000000000000
DCache Info                  = 0x0000000000000000
Sharedcache Info1            = 0x0000000000000000
Sharedcache Info2            = 0x0000000000000000
MIB_RSLOG1                   = 0x0000000000000000
MIB_RSLOG2                   = 0x0000000000000000
MIB_RQLOG                    = 0x0000000000000000
MIB_REQLOGa                  = 0x0000000000000000
MIB_REQLOGb                  = 0x0000000000000000

Reserved                     = 0x0000000000000000
Cache Repair Detail          = 0x0000000000000000

PIM Detail Text:



--------------  Memory Error Log Information  --------------

   No errors logged for this bus

------------  I/O Module Error Log Information  ------------

  No IO subsystem errors recorded

FRU INFORMATION

        Module              Revision
        ------              --------
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2
        PA 8900 CPU Module  3.2

Board Info!
  Format Version  : 0x1                   Language Code : 0x0
  Mfg Date        :                       Mfg Name      : JABIL
  Product Name    : augustus baseboard
  Serial Number   : 52JAPE4822000149
  Part Number     : A6961-60401
  Fru File Tp/Len : 0x1  Fru File :
  Revision        : A  Eng Date Code : 4728
  Artwork Rev     : A5  Fru Info :



=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=-+-=

----- output clipped -----

I showed only one processor details above.

print_manifest report

This command comes bundled with the Ignite Ux package. If you have ignite installed on your server you will be able to run this command. This command shows a number of the processor along with its speed.

# print_manifest

NOTE: Could not read the /etc/resolv.conf file.

System Information

    Your Hewlett-Packard computer has software installed and
    configured as follows.

    The system was created September 24, 2008, 02:30:54 EDT.
    It was created with Ignite-UX revision C.6.10.97.

-------------------------------------------------------------
NOTE: You should retain this information for future reference.
-------------------------------------------------------------


System Hardware

    Model:              9000/800/rp4440
    Main Memory:        24574 MB
    Processors:         8
    Processor(0) Speed: 999 MHz
    Processor(1) Speed: 999 MHz
    Processor(2) Speed: 999 MHz
    Processor(3) Speed: 999 MHz
    Processor(4) Speed: 999 MHz
    Processor(5) Speed: 999 MHz
    Processor(6) Speed: 999 MHz
    Processor(7) Speed: 999 MHz
    OS mode:            64 bit
    LAN hardware ID:    0x001A4B08AF2E
----- output clipped -----

machinfo command

This command is available from HPUX 11.21 and above on RX models. This command gives you processor numbers, speed, sockets, and core details.

# machinfo
CPU info:
  4 Intel(R) Itanium 2 9000 series processors (1.6 GHz, 12 MB)
          533 MT/s bus, CPU version C2
          6 logical processors

MP console

Login to MP console and enter the command menu by typing cm. Then ss is the command which shows processor status. This shows you processor sockets. So if you are seeing 8 CPU in top command and below output in MP then its 4 processor sockets housing 4 duel-core processors.

[server12] MP:CM> ss

SS

System Processor Status:

   Monarch Processor: 0

   Processor Module 0: Installed and Configured
   Processor Module 1: Installed and Configured
   Processor Module 2: Installed and Configured
   Processor Module 3: Installed and Configured

top command

This is the simplest way to check the number of CPU on HPUX as well as any Linux system. The top output shows your list of the processor at top of the page.

# top
System: server1                                      Fri Nov 25 14:29:06 2016
Load averages: 0.15, 0.11, 0.11
386 processes: 362 sleeping, 24 running
Cpu states:
CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS
 0    0.13  23.8%   0.0%   0.0%  76.2%   0.0%   0.0%   0.0%   0.0%
 1    0.18  25.7%   0.0%   7.9%  66.3%   0.0%   0.0%   0.0%   0.0%
 2    0.16  14.9%   0.0%   2.0%  83.2%   0.0%   0.0%   0.0%   0.0%
 3    0.13   3.0%   0.0%   5.0%  92.1%   0.0%   0.0%   0.0%   0.0%
 4    0.13  23.8%   0.0%   4.0%  72.3%   0.0%   0.0%   0.0%   0.0%
 5    0.15  17.8%   0.0%   4.0%  78.2%   0.0%   0.0%   0.0%   0.0%
 6    0.15  11.9%   0.0%   4.0%  84.2%   0.0%   0.0%   0.0%   0.0%
 7    0.16  20.8%   0.0%   5.0%  74.3%   0.0%   0.0%   0.0%   0.0%
---   ----  -----  -----  -----  -----  -----  -----  -----  -----
avg   0.15  17.8%   0.0%   4.0%  78.2%   0.0%   0.0%   0.0%   0.0%

You can see CPU is numbered from 0 to 8 i.e. total of 8 CPU active.

sar output

Even sar output can be used to determine the number of CPU in the system. Use just one iteration for output for one second. sar will show one row for each cpu value.

Read our SAR tutorials

Counting the number of rows can help us figure out CPU count.

# sar -Mu 1 1

HP-UX apcrss78 B.11.11 U 9000/800    11/25/16

14:41:14     cpu    %usr    %sys    %wio   %idle
14:41:15       0       0       0       0      99
               1       0       1       0      98
               2       0       0       0      99
               3       0       0       0      99
               4       0       0       0      99
               5       0       0       0      99
               6      24       1       0      75
               7       0       1       0      98
          system       3       1       0      96

# sar -Mu 1 1 | awk 'END {print NR-5}'
8

See first command actual output. We are stripping off extra 5 lines which are for total, headers to get exact count using awk. in second command. Even first output shows CPU numbering like top in first column!

How to rename volume group

Learn how to rename the volume group in Linux or Unix. Understand what happens in the background when you change the volume group name of existing VG.

A volume group can be renamed with easy vgrename command Linux. But first, we will see how it can be done without vgrename command so that step by step you will understand what actually happens in the background while VG name changes.

We have seen how to create VG in the past and how to export/import VG. We are going to use these commands to rename VG. Below steps needs to be followed –

  1. Stop all user/app access to all mount points within VG using fuser
  2. Un-mount all LV using umount
  3. Deactivate VG using vgchange
  4. Export VG using vgexport
  5. Create a new name folder and group file using mknod
  6. Import VG with a new name in command options using vgimport
  7. Activate VG using vgchange
  8. Mount all LV using mount
  9. Edit related entries in /etc/fstab with a new name

See below output for the above-mentioned steps (HPUX console).

# fuser -cku /data
/data:   223412c(user1)
# umount /data
# vgchange -a n /dev/vg01
Volume group "/dev/vg01" has been successfully changed.
# vgexport -v -m /tmp/vg01.map vg01
Beginning the export process on Volume Group "/dev/vg01". 
/dev/dsk/c0t1d0 vgexport:Volume Group “/dev/vg01” has been successfully removed.
# mkdir /dev/testvg
# mknod /dev/testvg/group c major 0xminor
# vgimport -v -m /tmp/vg01.map /dev/testvg list_of_disk
vgimport: Volume group “/dev/testvg” has been successfully created.
Warning: A backup of this volume group may not exist on this machine.
Please remember to take a backup using the vgcfgbackup command after activating the volume group
# vgchange -a y testvg
Volume group “/dev/testvg” has been successfully changed.
# mount /dev/testvg/lvol1 /data

In the above step by step process, you can see how VG changes its name. We are changing its VG related file and directory and then we import it using old configuration but the new name.

In Linux, we have one command which does all this stuff in the background for you. vgrename is a command which used to rename VG in Linux. You have to supply the old VG name and required a new name.

# vgrename /dev/vg01 /dev/testvg
Volume group "/dev/vg01" successfully renamed to "/dev/testvg"
OR
# vgrename vg01 testvg
Volume group "vg01" successfully renamed to "testvg"

Keep in mind, this command also requires de-activated VG to work. So this is not an online process. It supports the below options :

  • -f Forcefully rename
  • -v Verbose mode

LVM cheatsheet

List of all LVM command of HPUX tutorials we have seen before on KernelTalks. LVM commands related to physical volume, volume group, and logical volume.

What is LVM?

LVM is a Logical Volume Manager.

LVM is a volume manager in Unix-Linux systems. It used to manage your disks. LVM enables raw disks to be used as a data store, file system defined mount points. LVM helps to manage your disk volumes efficiently for performance and data integrity. VxVM i.e. Veritas Volume Manager is another volume manager that is as popular as LVM.

Previously we have seen a series of LVM command tutorials on KernelTalks. Here is a summary of it along in the form of LVM cheatsheet for your quick reference.

Physical Volume Commands

Command
Description
Example
pvcreate Create physical volume Tutorial link
pvdisplay Display physical volume details Tutorial link
pvchange Activate, de-activate physical volume Tutorial link
pvmove Move data from one PV to another Tutorial link

Volume Group Commands

Command
Description
Example
vgcreate Create volume group Tutorial Link
vgdisplay Display volume group details Tutorial Link
vgscan Rebuild /etc/lvmtab file Tutorial Link
vgextend Add new PV to VG Tutorial Link
vgreduce Remove PV from VG Tutorial Link
vgexport Export VG from system Tutorial Link
vgimport Import VG into system Tutorial Link
vgcfgbackup Backup VG configurations Tutorial Link
vgcfgrestore Restore VG configurations Tutorial Link
vgchange Change details of VG Tutorial Link
vgremove Remove VG from system Tutorial Link
vgsync Sync stale PE in VG Tutorial Link

Logical Volume Commands

Command
Description
Example
lvcreate Create logical volume Tutorial Link
lvdisplay Display logical volume details Tutorial Link
lvremove Remove logical volume Tutorial Link
lvextend Increase size of logical volume Tutorial Link
lvreduce Decrease size of logical volume Tutorial Link
lvchange Change details of logical volume Tutorial Link
lvsync Sync stale LE of logical volume Tutorial Link
lvlnboot Set LV as root, boot, swap or dump volume Tutorial Link

Linux user management (useradd, userdel, usermod)

Learn how to create, delete, and modify a user in Linux (useradd, userdel, usermod). Basic user management which is must know for every Linux/Unix administrator.

Anyone accessing system locally or remotely has to has a user session on the server hence can be termed as a user. In this post, we will be seeing user management which is almost similar for all Linux, Unix systems. There are three commands useradd, userdel and usermod which are used to manage users on Linux systems.

Interesting related articles –

Command: useradd

Command to add a new user to the system. This command can be as short as just one argument of userid. When running with just userid as an argument then it takes all default values for creating that user as defined in /etc/default/useradd file. Or else a number of options can be specified which defines parameters of this new user while creation.

# cat /etc/default/useradd
# useradd defaults file
GROUP=100
HOME=/home
INACTIVE=-1
EXPIRE=
SHELL=/bin/bash
SKEL=/etc/skel
CREATE_MAIL_SPOOL=yes

The command supports the below options :

  • -b <base_dir> If the home directory is not specified this one is mandatory.
  • -c <comment> Any text like a description of the account
  • -d <home_dir> Home directory
  • -e <expire_date> Account expiry date in YYYY-MM-DD
  • -f <inactive> No of days after which acc will be disabled after password expiry
  • -g <gid> group id
  • -u <uid> User id
  • -G <groups> Secondary groups
  • -k <skel_dir> Files within skel_dir will be copied to home_dir of the user after creation
  • -K <key=value> To override default parameters in /etc/login.defs
  • -m Create the home directory if it doesn’t exist.
  • -o Allow non-unique UID
  • -p Encrypted password (not normal text one). It can be obtained from the crypt command.
  • -r Create a system account. This won’t have password aging and UID from system UID range
  • -s shell
# useradd -c "Test user" -d /home/test -m -e 2016-12-05 -f 7 -g 100 -u 956 -o -s /bin/bash testuser1
# cat /etc/passwd |grep testuser1
testuser1:x:956:100:Test user:/home/test:/bin/bash
# useradd testuser2
# cat /etc/passwd |grep testuser2
testuser2:x:54326:54329::/home/testuser2:/bin/bash

See the above example with and without using options. Also, check the below list, it shows where you can verify the account-related particular parameter which you specified in useradd command.

  • home_dir Check using ls -lrt
  • uid, gid In /etc/passwd and /etc/group
  • comment, shell In /etc/passwd file
  • groups In /etc/group file
  • skel_dir files Check-in home_dir
  • expire_date, inactive Check-in chage -l username output.
  • Encrypted password In /etc/shadow file

Command: userdel

As the name suggests its a command to delete users. It has only two options –

  • -r Remove user’s home_dir & mail spool
  • -f Removes user even if he/she logged in. Removes home_dir, mail spool & group of the same name even these are being shared by another user. Dangerous!

If none of the options used and command just ran with userid argument. It will only remove the user from the system keeping its home_dir, mail spool and a group of the same name (if any) intact on the server.

#  ll /home |grep testuser
drwx------   4 testuser   testuser  4096 Nov 23 10:43 testuser
# userdel testuser
#  ll /home |grep testuser
drwx------   4      54326    54329  4096 Nov 23 10:43 testuser
# userdel -r testuser
#  ll /home |grep testuser
#

See above example which shows without using -r option keeps home directory intact.

Command: usermod

This command used to modify user parameters which we saw in useradd command. All parameter options with useradd command compatible with this command. Apart from those options, it supports below ones –

  • -l <new_login> Change login name to different. You have to manually rename home_dir
  • -L Lock account. Basically it puts ! in front of encrypted password in passwd or shadow file.
  • -U Unlock account. It removes!
  • -m <new_home> Moves home_dir to new_dir. -d is mandatory to use with it.
# useradd usr1# cat /etc/passwd |grep usr1
usr1:x:54326:54330::/home/usr1:/bin/bash
# usermod -l usr2 usr1
# cat /etc/passwd |grep usr2
usr2:x:54326:54330::/home/usr1:/bin/bash
# cat /etc/shadow |grep usr2
usr2:$6$nEjQiroT$Fjda8KiOIbnELAffHmluJFRC8jjIRWuxEWBePK1gun/ELZRi3glZdKVtPaaZ4tcQLIK2KPZTxdpB3tJvDj3/J1:17128:1:90:7:::
# usermod -L usr2
# cat /etc/shadow |grep usr2
usr2:!$6$nEjQiroT$Fjda8KiOIbnELAffHmluJFRC8jjIRWuxEWBePK1gun/ELZRi3glZdKVtPaaZ4tcQLIK2KPZTxdpB3tJvDj3/J1:17128:1:90:7:::
# usermod -U usr2
# cat /etc/shadow |grep usr2
usr2:$6$nEjQiroT$Fjda8KiOIbnELAffHmluJFRC8jjIRWuxEWBePK1gun/ELZRi3glZdKVtPaaZ4tcQLIK2KPZTxdpB3tJvDj3/J1:17128:1:90:7:::

See the above examples of usermod command showing locking, unlocking user and changing user names.

These three commands take almost most of the user management tasks in Linux Unix systems. Password management is another topic which does not fall in user management. We will see it on some other day.

Linux scheduler: Cron, At jobs

Learn everything about Linux/Unix schedulers i.e. cron and at. Know how to schedule cronjobs and at jobs, their configuration files, log files.

Unix or Linux comes with native in-build job scheduler i.e. cron and at. Out of which cron used to schedule tasks to repeat over some period while at used to execute the job at a specific time one time.

Cron

Cron enables administrators/users to execute a particular script or command at a given time of choice repetitively. It’s a daemon that runs in the background whenever system clock configured time it executes respective script or command. It can be checked if running with ps/service command.

# ps -ef |grep -i cron
root      2390     1  0 Mar17 ?        00:01:24 crond
root      8129  8072  0 09:50 pts/0    00:00:00 grep -i cron
# service crond status
crond (pid  2390) is running...

Configurations

Cron saves commands/scripts and related schedules in a file called crontab. Normally crontab can be found in path /var/spool/cron and file with a username (root user crontab file can be seen in the below example). These are plain text files that can be viewed using cat, more commands and can be edited using a text editor.

# pwd
/var/spool/cron
# ll
total 4
-rw------- 1 root root 99 Jul 31  2015 root
# cat root
00 8 * * 1 /scripts/log_collection.sh

But, it’s not advisable to edit crontab file with a text editor, you need to use crontab -e <username> command to edit it so that syntax can be verified before saving. This command opens a crontab file in a native text editor only.

Cron access can be given a user basis. The administrator can enable or disable cron access to a particular user. There are two files cron.allow, cron.deny; either one of which will exist on the server. These are files with usernames only. No special file format/syntax follows within. If both files are missing then the only superuser is allowed to use cron.

If cron.allow exists on the server then only users specified in this file are allowed to use cron, rest all are denied. And if it exists and empty then all are denied.

If cron.deny exists then only users specified in it are not allowed to use cron, rest all are allowed. And if it exists and empty then all are allowed.

Syntax

Let’s see the syntax for the crontab file and commands.

A crontab file has 6 fields separated by space to be filled in. Those are as below :

where,

  • Minute: Timestamp in 24 hrs format
  • Hours: Timestamp in 24 hrs format
  • Day of month: Date in dd format
  • Month: Month number in mm format or Jan, Feb format.
  • Day of week: Numeric/text day of the week. 0 or 7 being Sunday or Sun, Mon, etc.

These fields also support a series of values or multiple values example 1,2,3 or 1-4. When multiple time values defined then the event will happen whenever the clock hits one of the values.

Default cron definitions i.e. path or shell used to execute commands/scripts in crontabs etc are defined in /etc/crontab file. See example below :

# cat /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# For details see man 4 crontabs

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name command to be executed

Crontab commands

We have a crontab command with several options to play around configurations.

  • -u Specify user
  • -l to view specified user’s crontab
  • -e to edit specified user’s crontab
  • -r to remove specified user’s crontab
  • -i Interactive removal. Should be used with -r

If a new crontab is being set then the system will show using empty crontab for the user!

# crontab -u testuser -e
no crontab for testuser - using an empty one
crontab: installing new crontab
# crontab -u testuser-l
00 8 * * 1 echo test
# crontab -u testuser -i -r
crontab: really delete testuser's crontab? y
# crontab -u testuser -l
no crontab for testuser

Cron logs

All activities by cron daemon are logged in logfile /var/log/cron. It includes crontab alterations and cron daemon executions. Let’s look at the file

# tail /var/log/cron
Nov 21 10:25:36 oratest02 crontab[29364]: (root) BEGIN EDIT (testuser)
Nov 21 10:25:48 oratest02 crontab[29364]: (root) REPLACE (testuser)
Nov 21 10:25:48 oratest02 crontab[29364]: (root) END EDIT (testuser)
Nov 21 10:26:52 oratest02 crontab[30139]: (root) LIST (testuser)
Nov 21 10:27:46 oratest02 crontab[30695]: (root) DELETE (testuser)
Nov 21 10:27:53 oratest02 crontab[30697]: (root) LIST (testuser)
Nov 21 10:30:01 oratest02 CROND[31983]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Nov 22 10:40:01 oratest02 CROND[6166]: (root) CMD (/usr/lib64/sa/sa1 1 1)

In the above example, you can see, crontab alteration is being logged with what actions took place. Those logs are against the crontab field where first braces show the user who did alterations and last braces show which user’s crontab was altered. In the last two, you can see cron commands being executed by cron daemon according to schedule hence logged against CROND. This file is very helpful in troubleshooting issues related to cron executions.

at

At enables administrators/users to execute a particular script or command at a given time of choice only once. It can also be termed as one-time task scheduling.  Same as crond, a daemon for at is atd which runs in background. This can be checked using ps or service commands.

# ps -ef |grep -i atd
root      2403     1  0 Mar17 ?        00:00:00 /usr/sbin/atd
root     13568  8072  0 10:51 pts/0    00:00:00 grep -i atd
# service atd status
atd (pid  2403) is running...

Configurations

at stores submitted jobs in files located at /var/spool/at where file names are system generated and unlike crontabs these files can not be read.

# pwd
/var/spool/at
# ll
total 12
-rwx------  1 root   root 2994 Nov 22 10:57 a000010178544d
-rwx------  1 root   root 2989 Nov 22 11:00 a000020178548c
drwx------. 2 daemon daemon  4096 Jan 30  2012 spool

at access also can be given a user basis. It also has at.allow and at.deny files and those works same as cron.allow and cron.deny files we saw earlier in this post.

Syntax

at command should be supplied with the time you prefer to execute the command. Once given in proper format, it will present you with a prompt. This prompt takes command inputs that need to be executed at a given time. Once finished entering commands/ scripts one can simply press ctrl+d to exit out of at prompt and save the job. Observe a new file that is being generated at the above-mentioned path once you submit the job. at commands takes numerous types of time formats like noon, midnight, now + 2 hours, now + 20 minutes, tomorrow, next Monday, etc. If you enter the wrong format it will return the “garbled time” error message.

# at +2 hours
syntax error. Last token seen: +
Garbled time
# at now + 2 hour
at> echo hello
at> <EOT>
job 2 at 2016-11-22 13:00

To view currently queued jobs in at scheduler run atq or at -l command. It shows the output with numbering in the first column. The second field is about a time when the execution will happen and the last field is the username.

# atq
2       2016-11-22 13:00 a root
1       2016-11-22 11:57 a root

# at -l
2       2016-11-22 13:00 a root
1       2016-11-22 11:57 a root

To remove a particular job from queue atrm command is used. It should be supplied with serial number of the job. In the below example, we removed job number 2. You can see its vanished from the queue. The same can be achieved using at -r command instead of atrm.

# atrm 1

# at -l
2       2016-11-22 13:00 a root

at logs:

at daemon is very much regressive in terms of logging. Normally it does not log anything anywhere about its job queue alterations or job executions. Only fatal errors related to daemon are logged in Syslog only. Even if we turn debugging on, it logs information which is merely informative to look at.

LVM commands tutorial: Part 3: Logical Volume (lvsync, lvlnboot)

Series of the tutorial to learn LVM commands. In this part, learn how to sync LV and set it as a boot, root, swap device (lvsync, lvlnboot)

This is the last part of LVM command tutorials and last post for logical volume command too. Last all parts of this tutorial can be found on below links :

Let’s start with our first command here.

Command: lvsync

It synchronizes stale PE in given LV. It’s used in mirroring environment. Whenever there is any disk failure or disk path issue, PE goes bad and LV, in turn, has stale PE. Once the issue is corrected we need to sync stale PE with this command if they don’t sync automatically.

The command doesn’t have many options. It should be supplied with the LV path only.

# /usr/sbin/lvsync /dev/vg00/lvol6
Resynchronized logical volume "/dev/vg00/lvol6".

Command: lvlnboot

This command used to define logical volume as a root, dump, swap or boot volume. You have to submit an LV path along with the specific option of your choice to command. Options are as below :

  • -b Boot volume
  • -d Dump volume
  • -r Root volume
  • -s Swap volume
  • -R Recover any missing links
  • -v Verbose mode
# lvlnboot -r /dev/vg00/lvol3
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf
# lvlnboot -b /dev/vg00/lvol1
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf
# lvlnboot -s /dev/vg00/lvol2
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf
# lvlnboot -d /dev/vg00/lvol2
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

We have already seen this command in root disk mirroring.

This concludes our LVM command tutorials!! Feel free to drop in any queries you have.

LVM commands tutorial: Part 3: Logical Volume (lvextend, lvreduce, lvchange)

Series of the tutorial to learn LVM commands. In this part, learn how to extend, reduce and change the state of the logical volume (lvextend, lvreduce, lvchange)

In continuation of last part of a logical volume, we will be seeing more commands on lvol in this post. Previous posts of this LVM command tutorial can be found on below links :

Logical volumes like VG can be extended and shrank. We will be seeing lvextend, lvreduce, lvchangecommands in this post.

Command: lvextend

To extend logical volume, you should have enough free space within that VG. Command syntax is pretty much similar to lvcreate command for size. The only thing is you need to supply the final required size in command. For example, the current LV size is 1GB and you want to extend it with 2GB. Then you need to give the final 3GB size in the command argument.

# lvextend -L 3072 /dev/vg01/lvol1
Logical volume "/dev/vg01/lvol1" has been successfully extended.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

Another important option is of mirror copies. It plays a vital role in root disk mirroring. -m is the option with the number of mirror copies as an argument.

# lvextend -m 1 /dev/vg00/lvol1 /dev/disk/disk2_p2
The newly allocated mirrors are now being synchronized. This operation will
take some time. Please wait ....
Logical volume "/dev/vg00/lvol1" has been successfully extended.
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

Command: lvreduce

This command used for decreasing the number of mirror copies or decreasing the size of LV. This is the data destroying command. Hence make sure you have data of related file system backed up first. The size and mirror copy options are works the same for this command as well. -L for LE_reduce_size, -l number of LE to be reduced and -m is the number of copies to be reduced.

# lvreduce -L 500 /dev/vg01/lvol1
When a logical colume is reduced useful data might get lost;
do you really want the command to proceed (y/n) : y
Logical volume "/dev/vg01/lvol1" has been successfully reduced.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

While reducing mirror copies if one of the PV is failed or missing then command won’t run successfully. you need to supply -k option which will proceed to remove the mirror in case PV is missing.

Command: lvchange

This command is used for changing the characteristics of LV. There are numerous options that can be used.

  • -a y/n Activate or deactivate LV
  • -C y/n Change contiguous allocation policy
  • -D y/n Change distributed allocation policy
  • -p w/r Set permission
  • -t timeout Set timeout in seconds
  • -M y/n Change mirror write cache flag
  • -d p/s Change scheduling policy

This is the end of the second post on LV commands. In the next post, we will see lvsync and lvlnboot commands.

LVM commands tutorial: Part 3: Logical Volume (lvcreate, lvdisplay, lvremove)

Series of the tutorial to learn LVM commands. In this part, learn how to create, delete the logical volume and view details of it (lvcreate, lvdisplay, lvremove)

This is the last part of the LVM commands tutorial. Previously we have seen physical volume, volume group commands which can be seen on below links :

Logical volumes are small slices carved out of physical volumes storage space which is collectively available in the volume group. For more details check LVM legends.

Command: lvcreate

This command used to create a new logical volume. Logical volumes are mounted on directories as a mount point. So logical volume size is the size you want for the mount point. Use a command like below :

# lvcreate -L 1024 /dev/vg01
Logical volume "/dev/vg01/lvol1" has been successfully created with character device "/dev/vg01/rlvol1"
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

In the above command, you need to supply size in MB (1 GB in the above example) to -L argument and volume group name in which you need to create that LV. If no name suggested in command then by default command creates LV with name /dev/vg01/lvolX (X is next available number).

This command supports below options –

  • -l Number of LEs
  • -n LV Name

Created LV details can be seen using command lvdisplay.

Command: lvdisplay

We have seen above how to create LV, now we will see how to view details of it. This command is the same as pvdisplay for PV and vgdisplay for VG. It shows you details like name, volume group it belongs to, size, permission, status, allocation policy, etc.

# lvdisplay /dev/vg01/lvol1
--- Logical volumes ---
LV Name                     /dev/vg01/lvol1
VG Name                     /dev/vg01
LV Permission               read/write
LV Status                   available/syncd
Mirror copies               0
Consistency Recovery        MWC
Schedule                    parallel
LV Size (Mbytes)            1024
Current LE                  32
Allocated PE                32
Stripes                     0
Stripe Size (Kbytes)        0
Bad block                   on
Allocation                  strict
IO Timeout (Seconds)        default

More detailed output can be obtained with -v option. In this detailed output, you can get the LE details where they reside and LV distribution across disks.

# lvdisplay -v /dev/vg01/lvol1
--- Logical volumes ---
LV Name                     /dev/vg01/lvol1
VG Name                     /dev/vg01

----- Output clipped ----

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV
   /dev/disk/disk22        32        32

   --- Logical extents ---
   LE    PV1                     PE1   Status 1
   00000 /dev/disk/disk22        00000 current
   00001 /dev/disk/disk22        00001 current
   00002 /dev/disk/disk22        00002 current
   00003 /dev/disk/disk22        00003 current
   00004 /dev/disk/disk22        00004 current
   00005 /dev/disk/disk22        00005 current
   00006 /dev/disk/disk22        00006 current
   00007 /dev/disk/disk22        00007 current
   00008 /dev/disk/disk22        00008 current
   00009 /dev/disk/disk22        00009 current
   00010 /dev/disk/disk22        00010 current
   00011 /dev/disk/disk22        00011 current
   00012 /dev/disk/disk22        00012 current
   00013 /dev/disk/disk22        00013 current
   00014 /dev/disk/disk22        00014 current

----- output truncated -----

Command: lvremove

Removing a logical volume is data destroying task. Make sure you take the backup of data within the mount point then empty it and stop all user/app access to it. If LV is not empty then the command will prompt you for confirmation to proceed. 

# lvremove /dev/vg01/lvol1
The logical volume "/dev/vg01/lvol1" is not empty;
do you really want to delete the logical volume (y/n) : y
Logical volume "/dev/vg01/lvol1" has been successfully removed.
Volume Group configuration for /dev/vg03 has been saved in /etc/lvmconf/vg01.conf

Once lvol is deleted its number is again available for next new lvol which is being created in the same VG. All PE assigned to this LV will be released as free PE and hence free space in VG will increase.

We will be seeing how to extend and reduce LV also how to activate or deactivate LV in the next post.