LVM commands tutorial: Part 3: Logical Volume (lvextend, lvreduce, lvchange)

Leave a reply

Series of the tutorial to learn LVM commands. In this part, learn how to extend, reduce and change the state of the logical volume (lvextend, lvreduce, lvchange)

In continuation of last part of a logical volume, we will be seeing more commands on lvol in this post. Previous posts of this LVM command tutorial can be found on below links :

Logical volumes like VG can be extended and shrank. We will be seeing lvextend, lvreduce, lvchangecommands in this post.

Command: lvextend

To extend logical volume, you should have enough free space within that VG. Command syntax is pretty much similar to lvcreate command for size. The only thing is you need to supply the final required size in command. For example, the current LV size is 1GB and you want to extend it with 2GB. Then you need to give the final 3GB size in the command argument.

# lvextend -L 3072 /dev/vg01/lvol1
Logical volume "/dev/vg01/lvol1" has been successfully extended.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

Another important option is of mirror copies. It plays a vital role in root disk mirroring. -m is the option with the number of mirror copies as an argument.

# lvextend -m 1 /dev/vg00/lvol1 /dev/disk/disk2_p2
The newly allocated mirrors are now being synchronized. This operation will
take some time. Please wait ....
Logical volume "/dev/vg00/lvol1" has been successfully extended.
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

Command: lvreduce

This command used for decreasing the number of mirror copies or decreasing the size of LV. This is the data destroying command. Hence make sure you have data of related file system backed up first. The size and mirror copy options are works the same for this command as well. -L for LE_reduce_size, -l number of LE to be reduced and -m is the number of copies to be reduced.

# lvreduce -L 500 /dev/vg01/lvol1
When a logical colume is reduced useful data might get lost;
do you really want the command to proceed (y/n) : y
Logical volume "/dev/vg01/lvol1" has been successfully reduced.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

While reducing mirror copies if one of the PV is failed or missing then command won’t run successfully. you need to supply -k option which will proceed to remove the mirror in case PV is missing.

Command: lvchange

This command is used for changing the characteristics of LV. There are numerous options that can be used.

  • -a y/n Activate or deactivate LV
  • -C y/n Change contiguous allocation policy
  • -D y/n Change distributed allocation policy
  • -p w/r Set permission
  • -t timeout Set timeout in seconds
  • -M y/n Change mirror write cache flag
  • -d p/s Change scheduling policy

This is the end of the second post on LV commands. In the next post, we will see lvsync and lvlnboot commands.

LVM commands tutorial: Part 3: Logical Volume (lvcreate, lvdisplay, lvremove)

Leave a reply

Series of the tutorial to learn LVM commands. In this part, learn how to create, delete the logical volume and view details of it (lvcreate, lvdisplay, lvremove)

This is the last part of the LVM commands tutorial. Previously we have seen physical volume, volume group commands which can be seen on below links :

Logical volumes are small slices carved out of physical volumes storage space which is collectively available in the volume group. For more details check LVM legends.

Command: lvcreate

This command used to create a new logical volume. Logical volumes are mounted on directories as a mount point. So logical volume size is the size you want for the mount point. Use a command like below :

# lvcreate -L 1024 /dev/vg01
Logical volume "/dev/vg01/lvol1" has been successfully created with character device "/dev/vg01/rlvol1"
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

In the above command, you need to supply size in MB (1 GB in the above example) to -L argument and volume group name in which you need to create that LV. If no name suggested in command then by default command creates LV with name /dev/vg01/lvolX (X is next available number).

This command supports below options –

  • -l Number of LEs
  • -n LV Name

Created LV details can be seen using command lvdisplay.

Command: lvdisplay

We have seen above how to create LV, now we will see how to view details of it. This command is the same as pvdisplay for PV and vgdisplay for VG. It shows you details like name, volume group it belongs to, size, permission, status, allocation policy, etc.

# lvdisplay /dev/vg01/lvol1
--- Logical volumes ---
LV Name                     /dev/vg01/lvol1
VG Name                     /dev/vg01
LV Permission               read/write
LV Status                   available/syncd
Mirror copies               0
Consistency Recovery        MWC
Schedule                    parallel
LV Size (Mbytes)            1024
Current LE                  32
Allocated PE                32
Stripes                     0
Stripe Size (Kbytes)        0
Bad block                   on
Allocation                  strict
IO Timeout (Seconds)        default

More detailed output can be obtained with -v option. In this detailed output, you can get the LE details where they reside and LV distribution across disks.

# lvdisplay -v /dev/vg01/lvol1
--- Logical volumes ---
LV Name                     /dev/vg01/lvol1
VG Name                     /dev/vg01

----- Output clipped ----

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV
   /dev/disk/disk22        32        32

   --- Logical extents ---
   LE    PV1                     PE1   Status 1
   00000 /dev/disk/disk22        00000 current
   00001 /dev/disk/disk22        00001 current
   00002 /dev/disk/disk22        00002 current
   00003 /dev/disk/disk22        00003 current
   00004 /dev/disk/disk22        00004 current
   00005 /dev/disk/disk22        00005 current
   00006 /dev/disk/disk22        00006 current
   00007 /dev/disk/disk22        00007 current
   00008 /dev/disk/disk22        00008 current
   00009 /dev/disk/disk22        00009 current
   00010 /dev/disk/disk22        00010 current
   00011 /dev/disk/disk22        00011 current
   00012 /dev/disk/disk22        00012 current
   00013 /dev/disk/disk22        00013 current
   00014 /dev/disk/disk22        00014 current

----- output truncated -----

Command: lvremove

Removing a logical volume is data destroying task. Make sure you take the backup of data within the mount point then empty it and stop all user/app access to it. If LV is not empty then the command will prompt you for confirmation to proceed. 

# lvremove /dev/vg01/lvol1
The logical volume "/dev/vg01/lvol1" is not empty;
do you really want to delete the logical volume (y/n) : y
Logical volume "/dev/vg01/lvol1" has been successfully removed.
Volume Group configuration for /dev/vg03 has been saved in /etc/lvmconf/vg01.conf

Once lvol is deleted its number is again available for next new lvol which is being created in the same VG. All PE assigned to this LV will be released as free PE and hence free space in VG will increase.

We will be seeing how to extend and reduce LV also how to activate or deactivate LV in the next post.

LVM commands tutorial: Part 2: Volume group (vgremove, vgsync)

Leave a reply

Series of the tutorial to learn LVM commands. In this part, learn how to remove volume group from the system and how to sync stale PE within VG (vgremove, vgsync)

This is the fourth and the last post of part two which is related to volume group in our LVM commands tutorial. Rest of the parts and posts can be found in below links :

We have seen 10 commands of volume group activities until now under part two. Now we will cover the remaining two commands in this post.

Command: vgremove

vgremove commands used to remove the volume group from the system. But this is destroying command since it requires removal of all LV, PV in VG. It is always recommended to use vgexport instead of vgremove. Since vgexport also removes VG information from the system but keeps it untouched on PV so that the same PV can be imported to new VG on the new/same system using vgimport.

Safe removal of VG can be done with the below steps :

  1. Backup all user data in that VG
  2. Get information about all LV and PV in that VG using vgdisplay -v command
  3. Make sure no LV is in use using fuser -cu /mount_point command 
    # fuser -cu /data /data:   223412c(user1)
  4. Unmount mount points of related LV 
    # umount /data
  5. Remove all LVs with lvremove /dev/vg01/lvol-name command 
    # lvremove /dev/vg01/lvol1 The logical volume "/dev/vg01/lvol1" is not empty; do you really want to delete the logical volume (y/n) : y Logical volume "/dev/vg01/lvol1" has been successfully removed. Volume Group configuration for /dev/vg03 has been saved in /etc/lvmconf/vg01.conf
  6. Remove all PVs in VG except anyone with vgreduce /dev/vg01 /dev/disk/diskX command 
    #vgreduce /dev/vg01 /dev/disk/disk4 Volume group "/dev/vg01" has been successfully reduced. Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf
  7. Finally, use vgremove command to remove VG from system 
    # vgremove /dev/vg01 Volume group "/dev/vg01" has been successfully removed
  8. Remove related group files from system using rm /dev/vg01 command

Command: vgsync

This command used to sync stale LE of LV mirrors in current VG. This used in mirroring only. One can observe the output of vgdisplay -v and confirm if there are any stale LE in current VG. If you found stale LE then you can synchronize them using this command.

# vgsync /dev/vg01
Resynchronized logical volume "/dev/vg01/lvol01".
Resynchronized logical volume "/dev/vg01/lvol02".
Resynchronized volume group "/dev/vg01".

There are no special options required for this command.

This concludes part two (Volume group) of our LVM tutorials.

LVM commands tutorial : Part 2 : Volume group (vgcfgbackup, vgcfgrestore, vgchange)

Leave a reply

Series of the tutorial to learn LVM commands. In this part, how to backup/restore VG configuration and to change VG state (vgcfgbackup, vgcfgrestore, vgchange)

This is the third post of part 2 in our LVM commands tutorial. In this post, we are going to see how to backup and restore volume group configurations and how to change volume group states i.e. how to activate/de-activate volume group. Other parts of this LVM tutorial can be found on below links :

Volume group configurations play a vital role in moving PV, VG, data across systems. Let’s start with the configuration backup command.

Command: vgcfgbackup

As the command reads, it takes volume group configuration backup into a disk file residing under /etc/lvmconf directory in the file /etc/lvmconf/vg_name.conf. It reads the LVM header details from the system area of the disk and copies it to file. This file helps you to restore configuration on newly added disk in place of the old disk which may have got corrupted or failed.

It is recommended to have this backup taken after every LVM level change. By default all LVM commands altering LVM details are designed to take this backup automatically hence manually running command is not necessary.

To take manual configuration backup use command as below :

# vgcfgbackup /dev/vg01
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

This needs to run for every volume group available on the system. This command has a couple of options :

  • -f file Save backup in the file specified rather than the default location
  • -u Only updates extra PV which are added to VG after the last backup. Only new PVs required to be online on the system. If -u not used all PV under VG should be online while running the command.

Command: vgcfgrestore

This command restores the VG configuration taken using the above command. Command mainly needs 2 argument volume group name of which configuration needs to be restored and a PV name on which configuration needs to be restored. There are two things which need to keep in mind.

If PV on which we are restoring the backup is part of the mirror copy, then it should be deactivated first. Then restore back up and reactivate it. Command sequence will be :

# pvchange -a n /dev/disk/disk5
Physical volume "/dev/disk/disk5" has been successfully changed.

# vgcfgrestore -n /dev/vg01 /dev/rdisk/disk5
Volume Group configuration has been restored to /dev/rdisk/disk5

# pvchange -a y /dev/disk/disk5
Physical volume "/dev/disk/disk5" has been successfully changed.

If PV is not a part of the mirror then you should deactivate the volume group first then restore the backup and then activate the volume group again. Command sequence will be :

# vgchange -a n /dev/vg01
Volume group "/dev/vg01" has been successfully changed.

# vgcfgrestore -n /dev/vg01 /dev/rdisk/disk5
Volume Group configuration has been restored to /dev/rdisk/disk5

# vgchange -a y /dev/vg01
Volume group "/dev/vg01" has been successfully changed.

This command has several options which can be used with it :

  • -l List configuration only
  • -F Forcefully restore
  • -f file  Restore the configuration from a specified file, not from default backup location
  • -v Verbose mode
  • -R Forcefully restore even if there is a mismatch between PV in the kernel and in config.
  • -o old_path Restore config saved for old PV (path supplied) to new PV mentioned in the command.

Command: vgchange

This command used to make the volume groups active or inactive. The activated volume group simply means it’s available for use. There are different modes of volume groups in which they can be activated. They can be listed as below :

  1. Normal availability mode
  2. Cluster aware mode
  3. Shareable mode
  4. Quoram requirement mode

vgchange command can be used with specified options and their values to activate or deactivate the volume group. For example to normally activate it should be supplied with the argument -a and its value y. See below output to activate and then deactivate volume group

# vgchange -a y vg01
Volume group “/dev/vg014” has been successfully changed.

# vgchange -a n vg01
Volume group “/dev/vg014” has been successfully changed.

Above stated modes can be activated/deactivated using the below options :

  • -a y/n Normal availability mode
  • -c y/n Cluster aware mode
  • -S y/n Shareable mode
  • -q y/n Quoram requirement mode
  • -p Only activate if all related PVs are online
  • -s Disable stale PE sync
  • -x Cross activate the shareable volume group

This concludes the third post of part two related to the volume group. In the next and last post of part two, we are covering how to remove the volume group from the system and how to sync stale PE within the volume group.

LVM commands tutorial : Part 2 : Volume group (vgextend, vgreduce, vgexport, vgimport)

Leave a reply

Series of the tutorial to learn LVM commands. In this part, how to extend, reduce, export, and import volume group  (vgextend, vgreduce, vgexport, vgimport).

In this post, we are going to see how to extend the volume group, how to reduce the volume group, how to export volume groups, and how to import it. Previous posts of this LVM tutorial series can be found on below links :

This is the second post in part two of the volume group. In previous posts, we have seen commands like vgcreate, vgdisplay and vgscan. Let’s walk through new VG related commands.

Command: vgextend

Whenever there is a need for extra space on mount points and new disks are added to the system, related volume groups need to accommodate these new disks. Let’s say vg01 is existing VG and new disk5 is added to the server then you need to use vgextend command to add these new disks into vg01. This will, in turn, add new free PE in existing VG vg01 and hence free space will hike in VG.

To add disks into existing VG, you need to initialize it first with pvcreate command (see part one tutorial for pvcreate). Once done disk can be used in vgextend like below :

# vgextend /dev/vg01 /dev/disk/disk5
Volume group "/dev/vg01" has been successfully extended.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

After successful completion, you can verify that new PV is added to VG by looking at vgdisplay -v vg01 output.

This command can be used with below options :

  • -f Forcefully extend
  • -x Set allocation permissions
  • -z sparepv Should be used to mirroring to define spare PV
  • -g pvg_name Extend with the physical volume group specified.

Command: vgreduce

Its just reverse of the above command. It reduces VG by removing specified PV in command. It’s a bit risky to use hence extra precaution needed. You should make sure that no PE from targeted PV is being used in any LV of VG. This can be verified by lvdisplay and vgdisplay command. If no then you can use pvmove, lvremove commands to move/delete data. Once you are sure that no PE from targeted PV is being used i.e. all PEs are free to form this PV then run below command to remove PV from VG.

# vgreduce /dev/vg01 /dev/disk/disk5
Volume group "/dev/vg01" has been successfully reduced.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

If you are using multiple PV links then you should remove all PV paths from VG using the above command.

This command allows two options :

  • -f Forcefully remove PV
  • -l pv_path Remove only specified particular PV path from VG

Command: vgexport

Using vgexport one can export all VG related information from disks into map file leaving data on disks intact. With the help of this map file, the same VG can be imported on another system provided all related disks are presented to this another system.

Please note that running this command removes all VG related files from /dev directory and all its related entries in /etc/lvmtab file. This means after running vgexport, that particular VG is turned into non-existent VG for the system.

Before running vgexport, one has to make sure that no one is using data on LVOLs from this VG. This can be verified using fuser -cu /mount_point command. Also, VG shouldn’t be active at the time of export hence you need to de-activate VG using vgchange.

# vgchange -a n /dev/vg01
Volume group "/dev/vg01" has been successfully changed.

# vgexport -v -m /tmp/vg01.map vg01
Beginning the export process on Volume Group "/dev/vg01". 
/dev/dsk/c0t1d0 vgexport:Volume Group “/dev/vg01” has been successfully removed.

This command allows below options :

  • -p Preview only. It won’t actually export Vg but creates a map file. Useful for live VG
  • -v Verbose mode
  • -f file Writes all PV paths to file. This file can be used as input to vgimport.

Command: vgimport

This command imports VG which been exported using the above command. Map file extracted from vgimport command should be supplied as an argument to vgimport. In case if a map file is not available, it will still try to import VG by reading LVM information lies of physical disks. In this case, a list of all PV should be supplied.

After vgimport successfully imports VG, note that it’s in an inactive state. You need to activate VG to use it on the system. Also, its recommended to take a configuration backup once you import new VG on system

# vgimport -v -m /tmp/vg01.map /dev/vg01 list_of_disk
vgimport: Volume group “/dev/vg01” has been successfully created.
Warning: A backup of this volume group may not exist on this machine.
Please remember to take a backup using the vgcfgbackup command after activating the volume group.

# vgchange -a y vg01
Volume group “/dev/vg01” has been successfully changed.

# vgcfgbackup /dev/vg01
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

This command can be supplied with below options :

  • -N Import with persistent device file names
  • -s Scan each disk
  • -p Preview only
  • -v Verbose mode
  • -f file Import PV paths from the file

This concludes the second post of part two. In the next post, we will be seeing how to backup and restore volume group configurations and how to change volume group states.

LVM commands tutorial : Part 2 : Volume group (vgcreate, vgdisplay, vgscan)

Leave a reply

Series of the tutorial to learn LVM commands. In this part of the tutorial, learn how to create volume groups, and see details of it  (vgcreate, vgdisplay, vgscan).

Today we are starting with part two of the LVM command tutorial series. Other series posts can be found here :

This is the first post of part two which will be taking care of volume group activities. In the following writeup, we are going to see commands on how to create a volume group, how to view details of the volume group and how to scan volume groups to build new /etc/lvmtab file.

Command: vgcreate

As we all know volume group i.e. VG is a collection if PV. Refer to LVM legends for better understanding. VG creation is the next necessary step after PV creation to use disk space in mount points. Since in Unix everything is a file, to manage VG, the kernel creates /dev/vgname/group file. This file represents VG in the kernel. Let us see how to create VG from a bunch of available PV. Post Mar 2008 releases of HPUX creates group file automatically with vgcreate command. But we will see here how to make one.

First, create a directory with the desired name of your VG for example. let’s say testvg is our VG name. After that using mknod command creates a special group file. In command, you need to supply major and minor numbers.

# mkdir /dev/testvg
# mknod /dev/testvg/group c major 0xminor

major number: 64 for version 1.0 VG and 128 for version 2.0 VG
minor number: Its hexadecimal number. 0xnn0000 for v1.0 and 0x0000 for v2.0 where nn/nnn is unique number.

Now that special file is generated go-ahead to create VG

# vgcreate /dev/testvg /dev/disk/disk3 /dev/disk/disk4
Volume group "/dev/testvg" has been successfully created.
Volume Group configuration for /dev/testvg has been saved in /etc/lvmconf/testvg.conf

vgcreate can be run with a single PV argument too. This command has several options as below :

  • -V 1.0            To decide version.
  • -s PE_size    Size of PE to be used in MB. Default is 4MB
  • -e max_PE   Max PE per PV. Default is 1016
  • -l max_lv      Max number of LV per VG. Default is 255
  • -p max_pv   Max number of PV per VG. Default is 255
  • -S vg_size    Max future size of VG. Only in v2.0.

The above options can be supplied to command with proper values but it’s not mandatory. If not supplied then their respective values will be taken into consideration while creating VG. Changes in parameters with these options can be seen in vgdisplay output which you can see incoming section.

Command: vgdisplay

Same as pvdisplay command, vgdisplay is used to view VG details. vgdisplay command can be run with -v option to get more detailed output. If run without -v option then it will show output like below but PV details portion won’t be there.

# vgdisplay -v vg_new
--- Volume groups ---
VG Name                     /dev/testvg
VG Write Access             read/write     
VG Status                   available                 
Max LV                      255    
Cur LV                      0      
Open LV                     0      
Max PV                      16     
Cur PV                      2      
Act PV                      2      
Max PE per PV               6000         
VGDA                        2   
PE Size (Mbytes)            32              
Total PE                    26    
Alloc PE                    0       
Free PE                     26    
Total PVG                   0        
Total Spare PVs             0              
Total Spare PVs in use      0 

   --- Physical volumes ---
   PV Name                     /dev/disk/disk3
   PV Status                   available                
   Total PE                    10       
   Free PE                     10       
   Autoswitch                  On        

   PV Name                     /dev/disk/disk4
   PV Status                   available                
   Total PE                    10       
   Free PE                     10       
   Autoswitch                  On

In the above output, you can see PE Size (Mbytes), Max PE per PV, Max LV, Max PV fields which can be altered with options -s, -e, -l, -p arguments in vgcreate command we saw above.  The above output is pretty self-explanatory. It also shows PV details which are part of this VG.

Command: vgscan

This command used for scanning all available PVs to get information on VG and related LV’s and rebuild /etc/lvmtab file. This command used the below options :

  • -a Scan all available PV paths.
  • -B Write all persistent and legacy paths to /etc/lvmtab file
  • -f vg_name Force to update entries of existing VG in /etc/lvmtab with updated info, if any
  • -k Avoid probing disks. Build file from kernel known structure.
  • -N Populate the file with persistent DSF names
  • -p Preview mode only. It won’t update lvmtab file
  • -v verbose mode. Prints messages in the console.
# vgscan -v
/dev/vg00
/dev/disk/disk1_p2
/dev/disk/disk2_p2

Scan of Physical Volumes Complete.

Normally we use vgscan -v only. After this command, you can verify the timestamp of /etc/lvmtab file to verify it’s been updated.

This concludes the first 3 commands of VG. In the next post, we will see how to extend the volume group, how to reduce the volume group and how to export/import volume groups.

LVM commands tutorial : Part 1 : Physical Volume (pvchange, pvmove)

Leave a reply

Series of the tutorial to learn LVM commands. In this part of the tutorial, learn different tasks on physical volume  (pvchange, pvmove).

This is the second post on the LVM tutorial. Rest posts from this series can be found on the below links.

This is the last post for part 1 which is dedicated to physical volume. In the last post, we saw pvcreate and pvdisplay. This post we will be checking out pvchange and pvmove commands.

Command: pvchange

pvchange used for changing attributes of physical volume. There are 6 options switch which can be used with this command. Let’s see them one by one.

-a

This switch used for the availability attribute of PV. This should be accompanied with y or n argument. y making PV available and n making it unavailable. You can see the example below with n argument which turns PV unavailable which can be verified in pvdisplay output.

# pvchange -a n /dev/disk/disk3
Physical volume "/dev/disk/disk3" has been successfully changed.

# pvdisplay /dev/disk/disk3
--- Physical volumes ---
PV Name                     /dev/disk/disk3
VG Name                     /dev/vg01
PV Status                   unavailable
Allocatable                 yes
VGDA                        2
Cur LV                      2
PE Size (Mbytes)            8
----- output truncated -----
-S

This switch enables or disables auto-switching. It also has a y or n argument. If activated it makes the current access path to switch on the better available path. If deactivated, path switching happens only if the current access path goes to an unavailable state (due to hardware, cable failure). This applies to disks with multi-paths only.

-t

It’s a timeout value. This switch should be supplied with a number of seconds. This value determines if IO timeout happens and the problem can be declared for IO on particular PV. Useful in clustering.

# pvchange -t 90 /dev/disk/disk3
Physical volume "/dev/disk/disk3" has been successfully changed.
-x

It enables or disables the extensibility of PV. This switch has y and n arguments that enable admin to add/restrict physical extents to PV. Refer to LVM legends to have a better understanding.

-s

Immediately begin accessing the associated physical volume.

-z

Defines PV is spare or regular. This option has significance in mirroring only.

Command: pvmove

As name suggests, it used to move data (in LVM term PE) from one PV to another PV. Command essential moves PEs and hence data within from source to destination PV. If destination PV is not specified then all available PV’s in the current volume group are considered for move operation. Command decides himself best-suited PV for move operation so that allocation policies can be met correctly.

Also if the command is supplied with a list of PV names then the first PV is always considered as source and rest all PVs are considered as a destination.

# pvmove /dev/disk/disk1 /dev/disk/disk2 /dev/disk/disk3

In above example PEs will be moved from disk 1 to disk2 & disk3.

You can also move data for particular lvol to new PV.

# pvmove -n /dev/vg01/lvol2 /dev/disk/disk1 /dev/disk/disk2
Transferring logical extents of logical volume "/dev/vg01/lvol2"...
Physical volume "/dev/disk/disk1" has been successfully moved.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf

# /usr/sbin/lvdisplay -v /dev/vg01/lvol2

----- output clipped -----

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV
   /dev/disk/disk2         1557      1557

   --- Logical extents ---
   LE    PV1                     PE1   Status 1
   00000 /dev/disk/disk2         00000 current
   00001 /dev/disk/disk2         00001 current
   00002 /dev/disk/disk2         00002 current
   00003 /dev/disk/disk2         00003 current

In above example PE belonging to lvol2 will be moved from disk1 to disk2. PE distribution can be confirmed via lvdisplay output.

This marks end of part 1 : Physical volume. If you have any questions, please drop in the comments below.

LVM commands tutorial : Part 1 : Physical Volume (pvcreate, pvdisplay)

Leave a reply

Series of the tutorial to learn LVM commands. In this part of the tutorial, learn how to create physical volume and how to view its details (pvcreate, pvdisplay)

We are starting this tutorial series for LVM (Logical Volume Manager) which is a widely used volume manager in HPUX and some Linux flavors too. This tutorial will be split into 4 parts :

Let’s dive into the first part i.e. physical volume.

A physical volume is a raw disk presented to the operating system. It can be a local disk, LUN from remote storage, disk from local disk array, etc. All storage disks from these types are formatted as physical volumes under LVM so that those can be used in definite file systems.

Command: pvcreate

Lets start with the first command to create physical volume. Before using this command you need to confirm your new raw disk on OS. You can use ioscan -fnCdisk command or confirm by checking into /dev/rdsk (or /dev/rdisk) directory.

Note : /dev/rdsk/cXdXtX is legacy naming conventions (HPUX 1111,11.21,10.x) whereas /dec/rdisk/diskX is persistent naming convention (11.21, 11.31).

Now that you identified new disk presented to server lets say /dev/rdisk/disk3 for example, you can run pvcreate command to create physical volume out of it.

# /usr/sbin/pvcreate /dev/rdisk/disk3
Physical volume "/dev/rdisk/disk3" has been successfully created.

The command outputs success message confirming PV has been created. Note here that you should use character device file i.e. with /dev/rdisk/diskX.

If by mistake you put in wrong disk name or disk is already being used in LVM on the same server then the command will fail. If you presented disk which was used earlier on another server and now you want to use it here with data loss then -f option can be used to forcefully create PV by destroying any data present on the disk. Note here that -f option do not ask any confirmation before deleting data.

There are other options can be used with this command which are:

  • -B to make it bootable disk. Used during root disk mirroring
  • -b, -d related to bad blocks
  • -s to specify effective size. Normally everyone uses full disk so this options doesn’t matter much
  • -t driver related

Normally these options are not used when we are aiming at PV for mount point usage only. Hence we are not going drill down these options.

Command: pvdisplay

Now, PV has been created one can see its details with pvdisplay command.

# /usr/sbin/pvdisplay /dev/disk/disk3
--- Physical volumes ---
PV Name                     /dev/disk/disk3
VG Name                     
PV Status                   available
Allocatable                 yes
VGDA                        2
Cur LV                      0
PE Size (Mbytes)            32
Total PE                    1557
Free PE                     1557
Allocated PE                0
Stale PE                    0
IO Timeout (Seconds)        default
Autoswitch                  On
Proactive Polling           On

Here many fields are self-explanatory. Refer LVM legends to have a better understanding.  From this output, you can also calculate disk size. Total PE x PE size = Disk size (available to use i.e. after formatting space loss)

If you want to drill down PE details i.e. which PE is serving which LV then -v option can be used. This is helpful when the disk has bad sectors. Below is the output in which PV is part of the volume group already.

# /usr/sbin/pvdisplay -v /dev/disk/disk3
--- Physical volumes ---
PV Name                     /dev/disk/disk3
VG Name                     /dev/vg01
PV Status                   available
Allocatable                 yes
VGDA                        2
Cur LV                      1
PE Size (Mbytes)            32
Total PE                    1557
Free PE                     0
Allocated PE                1557
Stale PE                    0
IO Timeout (Seconds)        default
Autoswitch                  On
Proactive Polling           On

   --- Distribution of physical volume ---
   LV Name                 LE of LV  PE for LV
   /dev/vg01/lvol1         1557      1557

   --- Physical extents ---
   PE    Status   LV                      LE
   00000 current  /dev/vg01/lvol1         00000
   00001 current  /dev/vg01/lvol1         00001
   00002 current  /dev/vg01/lvol1         00002
   00003 current  /dev/vg01/lvol1         00003
   00004 current  /dev/vg01/lvol1         00004
   00005 current  /dev/vg01/lvol1         00005
   00006 current  /dev/vg01/lvol1         00006
   00007 current  /dev/vg01/lvol1         00007
   00008 current  /dev/vg01/lvol1         00008
   00009 current  /dev/vg01/lvol1         00009
   00010 current  /dev/vg01/lvol1         00010
   00011 current  /dev/vg01/lvol1         00011
   00012 current  /dev/vg01/lvol1         00012
----- output truncated -----

Here you can see how PV is distributed among different LV. and furthermore, it gives you table mapping of PE to LE! There are numerous options can be used with this command but normally -v is used commonly.

There are two more commands used for PV operations: pvchange, pvmove. We covered theme commands in the next post.

Root disk mirroring in itanium HPUX

1 Reply

Step by step root disk mirroring guide for HPUX running on Itanium hardware. Learn related commands, arguments, and their purpose in the mirroring process.

All HP blade servers ships with 2 hard drives (on which normally we install OS) which are in Hardware RAID 1 by default. Which means they are in a mirror serving high redundancy to the server. If one disk fails, the server continues to work from the second disk. But in case of vPars, IVM, and blades where hardware raid is broke manually, we need to do software mirrors for OS disks. This is called root mirroring i.e. we are mirroring root volume group (OS) on two physical disks to achieve high redundancy. Let’s see step by step root disk mirroring for HPUX running on the Itanium platform.

Step 1

Create a partition file that will be used to write on disk to define partitions on the physical disk.

# echo "3
EFI 400MB
HPUX 100%
HPSP 500MB">/tmp/partitionfile

Here we are defining 3 partitions (1st line). EFI partition with 400MB size which will house EFI shell. HPSP i.e. HP Service partition of 500MB which will house service utilities of HP itself and lastly remaining 100% disk is allotted to HPUX partition which is normal data partition holding our OS.

Step 2

Identify disk on the server which is to be mirrored with current root disk. This can be done using ioscan -fnCdisk & insf -e -C disk command. Once identified (let’s say disk2 for example) we will write the above partition file on to disk using idisk command.

# echo yes | idisk -wf /tmp/partitionfile /dev/rdisk/disk2
idisk version: 1.43
********************** WARNING ***********************
If you continue you may destroy all data on this disk.
Do you wish to continue(yes/no)? yes

EFI Primary Header:
        Signature                 = EFI PART
        Revision                  = 0x10000
        HeaderSize                = 0x5c
        HeaderCRC32               = 0x5ca983b2
        MyLbaLo                   = 0x1
        AlternateLbaLo            = 0x3fffff
        FirstUsableLbaLo          = 0x40
        LastUsableLbaLo           = 0x3fffaf
        Disk GUID                 = f7ar7b9c-8f6t-11dc-8000-d6245b60e588
        PartitionEntryLbaLo       = 0x2
        NumberOfPartitionEntries  = 0xc
        SizeOfPartitionEntry      = 0x80
        PartitionEntryArrayCRC32  = 0x4ec7aafc

  Primary Partition Table (in 512 byte blocks):
    Partition 1 (EFI):
        Partition Type GUID       = c12a7328-f81f-11d2-ba4b-00a0c93ec93b
        Unique Partition GUID     = f7cb52cc-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0x40
        Ending Lba                = 0xf9fff
    Partition 2 (HP-UX):
        Partition Type GUID       = 75894c1e-3aeb-11d3-b7c1-7b03a0000000
        Unique Partition GUID     = f7cb52f4-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0xfa000
        Ending Lba                = 0x337fff
    Partition 3 (HPSP):
        Partition Type GUID       = e2a4t728-32r53-11d6-a623-7b03a0000000
        Unique Partition GUID     = f7cb5312-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0x338000
        Ending Lba                = 0x3fffbf

EFI Alternate Header:
        Signature                 = EFI PART
        Revision                  = 0x10000
        HeaderSize                = 0x5c
        HeaderCRC32               = 0xfcc1ebde
        MyLbaLo                   = 0x3fffff
        AlternateLbaLo            = 0x1
        FirstUsableLbaLo          = 0x40
        LastUsableLbaLo           = 0x3fffbf
        Disk GUID                 = f7cb4b9c-8f2d-11dc-8000-d6217b60e588
        PartitionEntryLbaLo       = 0x3fffdf
        NumberOfPartitionEntries  = 0xc
        SizeOfPartitionEntry      = 0x80
        PartitionEntryArrayCRC32  = 0x4ec7aafc

  Alternate Partition Table (in 512 byte blocks):
    Partition 1 (EFI):
        Partition Type GUID       = c12a7328-f81f-11d2-ba4b-00a0c93ec93b
        Unique Partition GUID     = f7cb52cc-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0x40
        Ending Lba                = 0xf9fff
    Partition 2 (HP-UX):
        Partition Type GUID       = 75894c1e-3aeb-11d3-b7c1-7b03a0000000
        Unique Partition GUID     = f7cb52f4-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0xfa000
        Ending Lba                = 0x337fff
    Partition 3 (HPSP):
        Partition Type GUID       = e2a4t728-32r53-11d6-a623-7b03a0000000
        Unique Partition GUID     = f7cb5312-8f2d-11dc-8000-d6217b60e588
        Starting Lba              = 0x338000
        Ending Lba                = 0x3fffbf

Legacy MBR (MBR Signatures in little endian):
   MBR Signature = 0xd44acbf7

Protective MBR

Step 3

Make the disk bootable with mkboot and write boot instructions on it. Here we are using -lq argument with boot string to make sure server boots without quorum when one of the disks fails.

# mkboot -e -l /dev/rdisk/disk2
# mkboot -a "boot vmunix -lq" /dev/rdisk/disk2

Verify if boot string is properly written on first partition i.e. EFI partition of the disk

# efi_cp -d /dev/rdisk/disk2_p1 -u /EFI/HPUX/AUTO /tmp/x
# cat /tmp/x
boot hpux -lq

Step 4

Now we are taking this disk into OS LVM and mirror. Create PV on this disk. Make sure you use -B argument to understand LVM that its bootable PV.

# pvcreate -B -f /dev/rdisk/disk2_p2
Physical volume "/dev/rdisk/disk2_p2" has been successfully created.

Step 5

Extend current root VG vg00 to accommodate this new disk.

# vgextend vg00 /dev/disk/disk2_p2
Volume group "vg00" has been successfully extended.
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

Step 6

Mirror all logical volumes on current boot disk on the new disk using mirror copies -m as 1 in an argument in lvextend command.

# lvextend -m 1 /dev/vg00/lvol1 /dev/disk/disk2_p2
The newly allocated mirrors are now being synchronized. This operation will
take some time. Please wait ....
Logical volume "/dev/vg00/lvol1" has been successfully extended.
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

This task can be easied with for a loop. Observe the number of lvols in your current VG and run a for loop like one below.

# for i in 1 2 3 4 5 6 7 8 9 10
do
lvextend -m 1 /dev/vg00/lvol$i /dev/disk/disk2_p2
done

Step 7

The second last step is to set boot, swap, dump, and root volumes on the new disk.

# lvlnboot -r /dev/vg00/lvol3
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

# lvlnboot -b /dev/vg00/lvol1
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

# lvlnboot -s /dev/vg00/lvol2
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

# lvlnboot -d /dev/vg00/lvol2
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf

Verify it

# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/disk/disk1_p2 -- Boot Disk
        /dev/disk/disk2_p2 -- Boot Disk
Boot: lvol1     on:     /dev/disk/disk1_p2
                        /dev/disk/disk2_p2
Root: lvol3     on:     /dev/disk/disk1_p2
                        /dev/disk/disk2_p2
Swap: lvol2     on:     /dev/disk/disk1_p2
                        /dev/disk/disk2_p2
Dump: lvol2     on:     /dev/disk/disk1_p2, 0

Step 8

Add a new disk name in /stand/bootconf file. Here l means the disk is managed by lvm.

# echo "l /dev/disk/disk2_p2" >> /stand/bootconf

Lastly, set a new disk’s hardware path as an alternative boot path, and you are done. Hardware path of the new disk can be obtained from ioscan output.

# setboot -a 2/0/0/2/0.6.0

Verify it

# /usr/sbin/setboot
Primary bootpath : 2/0/0/3/0.0x6.0x0 (/dev/rdisk/disk1)
HA Alternate bootpath :
Alternate bootpath : 2/0/0/2/0.0x6.0x0 (/dev/rdisk/disk2)

Autoboot is ON (enabled)
Hyperthreading : OFF
               : OFF (next boot)

Your root mirror on new disk completed!!

You can reboot the system and try to boot from alternate disk from EFI to test.

Run command on multiple linux servers from windows

Leave a reply

Learn how to run the same/repetitive command on multiple Linux servers from the Windows machine. This trick uses command line putty ‘plink’ utility.

One of the major concerns for sysadmin is to run the same/repetitive command on multiple Linux servers in infra when there is no centralized tool available. In this post, we are going to see how to run a command on multiple Linux servers in one go. There is no need of saving your account password anywhere and even no need to have expect function in your source machine!

Pre-requisite

  • Windows machine with plink downloaded on it (download plink here)
  • Linux servers should be reachable from a windows machine

How to do it

plink is a putty command-line utility. Using plink we will be able to connect to the server by supplying IP, username, password on the command line. plink can be invoked from a command prompt on windows.

C:\Users\noname\Desktop>plink -ssh user1@10.10.1.11 -pw password@123 (hostname; date)
testserver
Tue Nov  1 12:54:33 IST 2016

C:\Users\noname\Desktop>

Goto Windows command prompt by typing cmd in a run window (windows key + r). Navigate to folder where plink executable is kept and then type in plink command as above.

  • ssh: protocol to connect
  • id@server IP
  • pw: account password
  • Commands to execute on Linux server in braces.

This is how plink works. Now to execute the same command on multiple servers we see below example. We will connect to 3 servers and execute hostname & date command.

First, put all 3 server’s IP addresses in single file ip_list.txt. Then execute simple for loop on that file as below in command prompt. This is a very basic batch script.

C:\Users\noname\Desktop>FOR /F "tokens=1,2* delims=," %G IN (C:\Users\noname\Desktop\ip_list.txt) DO plink -ssh user1@%G -pw password@123 (hostname; date)

testserver
Tue Nov  1 12:54:33 IST 2016

testserver1
Tue Nov  1 12:54:36 IST 2016

testserver2
Tue Nov  1 12:54:39 IST 2016

C:\Users\noname\Desktop>

Voila! all server’s command output is there! Commands executed on all servers in one go.

This is very useful when you need to check some single line outputs from all servers or if you want to run account refresh commands on all servers in one go.