HP/UX Logical Volume Manager notes

Configure a brand new volume group and logical volumes from scratch (without using SAM) o Find out which devices you have ioscan -fn o Find out the size of the disk you want to allocate diskinfo -v /dev/rdsk/cXtXdX o Create a directory for the new volume group (with a character special file called group). mkdir /dev/vgXX mknod /dev/vgXX/group c 64 0xNN0000 Note: The minor number for the group file should be unique among all the volume groups on the system. It has the format 0xNN0000, where NN runs from 00 to 09. The maximum value of NN is controlled by the kernel tunable parameter maxvgs. Convention: NN is the same as the volume group number XX above. o Initialize the disk using pvcreate pvcreate /dev/rdsk/cXtXdX o Create the physical volume group consisting of 1 or many disks vgcreate /dev/vgXX /dev/dsk/cXtXdX [disk2] [disk3] ... Note 1: Use the block device notation here, not the character device. Note 2: You can use the -s option to specify the physical extent size (PE SIZE) Note 3: A volume group can consist of multiple disks. o Create a (zero size) logical volume lvcreate -n lvol1 /dev/vgXX Note: by not allocating the size here, the door is open to create mirrored logical volume using lvextend where the mirror is on a different disk drive. o Extend the logical volume to the desired size lvextend -L [Megs] /dev/vgXX/lvol1 /dev/dsk/cXtXd0 o Create a mirror on a different disk drive lvextend -m 1 /dev/vgXX/lvol1 /dev/dsk/cXtXd1 There is alternate way to create and extend a mirrored logical volume all in one step. o lvcreate -n lvol2 -L 1024 -m 1 -D y -s g /dev/vg01

Logical volume name = lvol2 Size = 1 gig (1024 meg) -m 1 = One mirror copy -D y = Turn on distributed allocation (distribute evenly between physical volumes) -s g = Set a strict allocation policy. Mirrors of a logical extent cannot share the same physical volume.

Prepare logical volumes for use with Informix o Note: make sure you use the character device here (rlvol1) o chmod 660 /dev/vgXX/rlvol1 o chown informix:informix /dev/vgXX/rlvol1 o ln -s /dev/vgXX/rlvol1 /RawSpaces/chunk1 Run the hp_disk_table1.sh to interrogate /etc/lvmtab to map hardware addresses to device names on HP/UX o Output looks like: sdisk card instance 2 SCSI target 6 SCSI LUN 0 section 0 at address 0/0/2/1.6.0 /dev/dsk/c2t6d0 sdisk card instance 1 SCSI target 6 SCSI LUN 0 section 0 at address 0/0/2/0.6.0 /dev/dsk/c1t6d0

Here's an example of the whole thing in a script. This will need to be edited before use. Note: When using 2 Gb logical volumes with a size of 2,048 Mb, create Informix chunks that are 2,097,150 Kb.
mkdir /dev/vg04 mknod /dev/vg04/group c 64 0x040000 ls -ls /dev/vg04 pvcreate /dev/rdsk/c5t1d3 vgcreate /dev/vg04 /dev/dsk/c5t1d3 lvcreate lvcreate lvcreate lvcreate lvcreate lvcreate lvcreate lvcreate lvcreate lvcreate -n -n -n -n -n -n -n -n -n -n root_dbs physical_dbs logical_dbs chunk1 chunk2 chunk3 chunk4 chunk5 chunk6 chunk7 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04 /dev/vg04

lvextend lvextend lvextend lvextend lvextend lvextend lvextend lvextend lvextend lvextend chmod chmod chmod chmod chmod chmod chmod chmod chmod chmod chown chown chown chown chown chown chown chown chown chown ln ln ln ln ln ln ln ln ln ln -s -s -s -s -s -s -s -s -s -s 660 660 660 660 660 660 660 660 660 660

-L -L -L -L -L -L -L -L -L -L

2048 2048 2048 2048 2048 2048 2048 2048 2048 2048

/dev/vg04/root_dbs /dev/vg04/physical_dbs /dev/vg04/logical_dbs /dev/vg04/chunk1 /dev/vg04/chunk2 /dev/vg04/chunk3 /dev/vg04/chunk4 /dev/vg04/chunk5 /dev/vg04/chunk6 /dev/vg04/chunk7

/dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3 /dev/dsk/c5t1d3

/dev/vg04/rroot_dbs /dev/vg04/rphysical_dbs /dev/vg04/rlogical_dbs /dev/vg04/rchunk1 /dev/vg04/rchunk2 /dev/vg04/rchunk3 /dev/vg04/rchunk4 /dev/vg04/rchunk5 /dev/vg04/rchunk6 /dev/vg04/rchunk7 /dev/vg04/rroot_dbs /dev/vg04/rphysical_dbs /dev/vg04/rlogical_dbs /dev/vg04/rchunk1 /dev/vg04/rchunk2 /dev/vg04/rchunk3 /dev/vg04/rchunk4 /dev/vg04/rchunk5 /dev/vg04/rchunk6 /dev/vg04/rchunk7 /RawSpaces/root_dbs /RawSpaces/physical_dbs /RawSpaces/logical_dbs /RawSpaces/chunk1 /RawSpaces/chunk2 /RawSpaces/chunk3 /RawSpaces/chunk4 /RawSpaces/chunk5 /RawSpaces/chunk6 /RawSpaces/chunk7

informix:informix informix:informix informix:informix informix:informix informix:informix informix:informix informix:informix informix:informix informix:informix informix:informix

/dev/vg04/rroot_dbs /dev/vg04/rphysical_dbs /dev/vg04/rlogical_dbs /dev/vg04/rchunk1 /dev/vg04/rchunk2 /dev/vg04/rchunk3 /dev/vg04/rchunk4 /dev/vg04/rchunk5 /dev/vg04/rchunk6 /dev/vg04/rchunk7

mirror_stripe.ksh Script to do mirroring and striping at the same time on HP-UX. mirror_stripe_config.ksh Config file used with above script.

Change the size of /usr, /var, /tmp filesystems

If you want to change the size of a Unix filesystem that is always mounted and used by the operating system, it is a problem because you can only change the size of a filesystem that is un-mounted. This is how you get around that problem. The following example is how I increased the size of /usr (on /dev/vg00/lvol7) from 400 MG to 700 MG.

Make sure you have free PE's in that volume group. o vgdisplay -v Backup the system. At least make a tar tape of /usr. Shutdown to single user mode. When prompted, press any key to interrupt the boot and go into single user mode. o shutdown -y now Boot from the primary device and invoke ISL. o bo pri isl Interact with ISL ? - respond: Y Boot into single user mode. o hpux -is Extend the logical volume that holds the filesystem. o /sbin/lvextend -L 700 /dev/vg00/lvol7 o Note: 700 is the desired end result size, NOT just the amount to increase. o Note: If you want to run 'bdf', it is located in /usr/bin. Make sure /usr is not mounted. If it is mounted, mount and unmount /usr to get it to be unmounted. I know this should be unneccessary - but I had to do it! o mount (to see if /usr is mounted) o mount /dev/vg00/lvol7 /usr (to mount /usr) o umount /dev/vg00/lvol7 (to un-mount /usr) o mount (to see if /usr is still mounted) Extend the filesystem. Don't forget to use the character device below. o /sbin/extendfs -F hfs /dev/vg00/rlvol7 Reboot back to multi-user mode. o shutdown -r now Check the size of the new /usr filesystem o bdf

Logical Volume Manager (LVM) using HP-UX Glossary

Term Journaled File System (JFS) Logical Extents Logical Interchange Format (LIF) Logical Volumes Mirroring Physical Extent (PE) Physical Partitions Physical Volume Striping Volume Volume Group

Unix filesystem that uses a journaled log to record chan

Logical extents are used to create the logical volume - th

HP Storage format used for interchange of files between

Allocates the space inside a volume group, the logical v

Mapping a logical extent to more than one physical exte

Unit used for allocating disk space to logical volumes, d Physical disk that is being managed by LVM

The physical volume is organized into blocks called phy

A logical volume that is configured to write across more

A device used for storage - either an entire disk or a par

Contain one or more physical volumes, a physical volum

Commands
Command lvcreate lvdisplay -v /dev/<VG>/<LV> pvchange pvcreate /dev/rdsk/cxtxdx pvcreate -f /dev/rdsk/cxtxdx pvdisplay /dev/dsk/cxtxdx Command used to create logical volumes Changes the attributes of existing physical volumes.

Command used to display information about logical vol

Creates a physical volume by creating the LVM data str creation of the PV - potentially destroying old PV inform

Command used to display information about physical vo

pvdisplay -v /dev/dsk/cxtxdx vgchange vgcreate vgdisplay [-v] [VG_NAME} vgexport vgimport Command used to create the volume groups

Command used to change information about the volume

Command used to display information about volume gro

Files
File /dev/dsk/cxtxdx /dev/rdsk/cxtxdx /dev/<volumegroup>/group /etc/lvmrc /etc/lvmtab /etc/vpath.cfg Block device for disk Character device for disk

Character device file (created with the mknod command

Script that starts each volume group based upon the con

This file has the device file associated with each disk in

If IBM SDD is being used this file contains the mapping command.

Procedures Filesystem Procedures:

Create a new filesystem: (A new filesystem will require a new logical volume)

1. Create the Logical Volume (Procedure Below) 2. Create the File System o Using newfs: 1. newfs -F <filesystem_type> <path_to_lvol_devicefile> 2. newfs -F vxfs /dev/vg02/lvol1
o

Using mkfs: 1. mkfs -F <FILESYSTEM TYPE> -o bsize=<OPTIONS>,<OPTIONS> <PATH TO LV>

1. -F is filesystem type, such as vxfs for the Veritas File System or JFS 2. -o are options, such as bsize (Block Size) and largefiles. 3. Path to LV is of the form /dev/VGNAME/LVNAME
2.

For example: "mkfs -F vxfs -o bsize=8192,largefiles /dev/bgbill/lvbill"

1. Create the Mount Point 0. mkdir -p <full path to mountpoint> 1. -p will make any subdirectories in the path that do not already exist 2. Set the proper ownership and permissions on the mount point 0. chown owner:group /<mount point> 1. chmod XXX /<mount point> 3. Mount the File System 0. Edit /etc/fstab using vi and add the proper entry 2. <LV PATH> <MOUNTPOINT PATH> <FILESYSTEM> <OPTIONS> <BACKUP FREQUENCY> <PASS NUMBER> 3. For example: "/dev/vgbill/lvbill /maindir/subdir/share vxfs rw,suid,largefiles,delaylog,datainlog 0 2" "mount /<mountpoint>" to mount the file system Do a "bdf <mount point>". This command will show if the file system is mounted, and it's size. Do an "ls -ld <mount point>" to verify that the proper permissions are there. Have the user test the filesystem, and report back with any problems

1.

2. Test
0. 1. 2.

Extending a VxFS Filesystem (Online JFS)

1. Verify that there is enough space in the volume group o Check the volume group for enough space vgdisplay <VG Name> o Check the logical volume to determine if mirroring or striping is in effect lvdisplay /dev/vg_name/lv_name o If there is not enough space then the volume group must be extended see procedure below. 2. First lvextend the logical volume - 2 Options for lvextend:
1. 2.

lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME

1. fsadm -F vxfs -b <new size of filesystem in K> <MOUNT POINT>

o o o o o

NOTE: extendfs will NOT work with Online JFS, you must use fsadm NOTE: new size of the filesystem = <# of LEs> * <LE Size in MB> * 1024 NOTE: If lvextend -L was used simply multipy size in MB by 1024 NOTE: If this FAILS with errno 28, then the filesystem is 100% full and must be reduced to less than 100%. NOTE: If this FAILS with "write failure at block XXXXXXXX : No such device or address" then the new size may be too big, use a smaller number

Extending a VxFS Filesystem (No Online JFS)

1. Verify that there is enough space in the volume group o vgdisplay <VG Name> o If there is not enough space then the volume group must be extended see procedure below. 2. First lvextend the logical volume - 2 Options for lvextend: lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME 2. lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME 2. Unmount the filesystem 3. extendfs -F vxfs /dev/<VG_Name>/r<LV_Name>
1. o

Note: extendfs uses the raw logical volume name (ie rlvol1)

How to determine if you have Online JFS

swlist -l fileset | grep -i advanced

How to determine the filesystem type

grep <mount point> /etc/fstab Results should show the filesystem type (vxfs or hfs) in the return string

Logical Volume Procedures:

Creating a Logical Volume:

1. Determine if an existing volume group has enough spare capacity for a new logical volume. 1. Is mirroring or striping required (if not known, then see if other logical volumes in that volume group use it)? 1. vgdisplay <This will list existing volume groups 1. Spare capacity is indicated by the number of free PEs 2. Multiply the "Free PE" by "PE Size (MBytes)" to get spare capacity in MB of the volume group Note: If mirroring or striping is required then there must be enough space available on multiple disks 3. Is there a volume group with enough spare capacity? If yes, the continue to "Create the Logical Volume" 1 1 If no, then the Volume Group must be extended or a new Volume Group must be created - see procedures below. 2. Create the logical volume:
1 1

lvcreate -1 <size in LEs> -n <LV Name> -r N <VG Directory path> OR 1. -l is size in Logical Extents (same as PE size) 2. -n is the name of the Logical Volume 3. -r N disable bad block allocation

o lvcreate -L <size in MBs> -n <LV Name> -r N <VG Directory path> 1. lvcreate -L 100 /dev/vg01 Logical volume will be 100MB in size Name of the logical volume default name will be lvolx 2. lvcreate -L 100 -n oracle /dev/vg01 This creates a logical volume called oracle of 100 MB

2.

Verify the lv was created properly "vgdisplay -v <volume group>"

Removing a Logical Volume

Unmount the filesystem o umount /<Path to Mount Point> Remove the volume group o lvremove /dev/<VG_NAME>/<LV_NAME>

Clean up o Delete the mount point o Remove the mount from /etc/fstab

Troubleshooting (just an idea, not sure if this really works??) ONCTRA01# lvchange -a n /dev/vg_wte_test/lv_wte_test Logical volume "/dev/vg_wte_test/lv_wte_test" has been successfully changed. Volume Group configuration for /dev/vg_wte_test has been saved in /etc/lvmconf/vg_wte_test.conf Unmirroring a Logical Volume

lvreduce -m 0 /dev/<VG_NAME>/<LV_NAME>

Volume Group Procedures:

Create a Volume Group

1. Determine the list of disk devices that will make up the volume group (procedure below) 2. Create PV disks for each of the disks (procedure below) 3. Create the volume group directory 1. mkdir /dev/vg01 4. Create the group file 1. Determine the minor number to use 1. ls -l /dev/*/group | sort +5 2. Choose the next unused number 1. mknod /dev/vol_group_directory/group c Major_number Minor_number 1. mknod /dev/vg01/group c 64 0x010000 2. The group file is used to allow communications between the lvm commands and the lvm kernel 2. Create the volume group 1. vgcreate -l 255 -p 64 -s 16 /dev/<vg id>/dev/dsk/<disk id> <-- create the volume group with initial disk devices 1. -l sets the maximum number of logical volumes for the volume group 2. -p sets the maximum number of physical volumes for the volume group

3. -s sets the physical extent size 4. /dev/<vg id> is the volume group directory 5. /dev/dsk/diskid are the physical volumes to be added to the volume group 6. vgcreate /dev/vg01 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0 2. vgextend /dev/vgxx /dev/dsk/diskid <-- adds additional disk devices to the volume group
Extending a Volume Group

A volume group is extended by adding additional PVs to it with vgextend o vgextend /dev/vgxx /dev/dsk/diskid Note: If there are no available PVs, then disk space (LUNS) must be added to the system and PVs must be created - see procedures below.

Activating a Volume Group

1. vgchange -a y /dev/<vg name> 2. mount -a ==> This may indicate that a filesystem check needs to be run (fsck) 3. fsck -m /dev/<vg name>/<lv name> ==> this is a sanity check, it is very quick, but may indicate that a complete fsck needs to be run 4. fsck /dev/<vg name>/<lv name> ==> this is the full file system check
Deactivating a Volume Group

1. umount all of the logical volumes in the volume group o Determine all of the mounted logical volumes by "bdf | grep vg_name" 2. vgchange -a n /dev/<vg name>
Adding mirroring to the root vg - vg00

1. pvcreate -B /dev/rdisk/c2t2d0 o Create the PV on the second disk so that LVM can manage it o -B makes it a bootable volume 2. mkboot /dev/rdsk/c2t2d0 o Installs the boot files on the second disk 3. mkboot -a "hpux-lq (;0)/stand/vmunix" /dev/rdsk/c2t2d0 o Creates an autoboot file on the disk, with information on how to boot 4. vgextend /dev/vg00 /dev/dsk/c2t2d0 o extends the volume group to include this second disk.

5. vgdisplay -v vg00 o To verify that the second disk is now part of the volume group 6. lvlnboot -v o To verify that the system thinks this disk can boot 7. lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c2t2d0 o This extends the logical volume to the mirrored disk, effectively creating the mirrored copy. o This command needs to be run for each logical volume in the volume group
Removing a Volume Group To remove a volume group using vgreduce and vgremove:

Remove all Logical Volumes (see procedure above) remove all disks except on using vgreduce <VG_NAME> </dev/dsk/DISK> o This will need to be done for each disk in the volume group, except for the last disk remove the final disk in the volume group with vgremove <VG_NAME>

To remove a volume group using vgexport:

Deactive the volume group vgexport -m <mapfile> -v -f <devicefile> <VG_NAME>

PV Procedures:
How to recognize a newly added LUN:

1. Check for the new hardware o ioscan -fnC disk | more <-- the new devices will have a hardware path, but no device file associated with it. 2. Create the device file for the hardware path o insf 3. If using vpaths then create the vpath association o /opt/IBMdpo/bin/cfgvpath o /opt/IBMdpo may not be the path of the sdd software, "whereis cfgvpath" may need to be run to find it if it is not in path. 4. Verify the new devices/vpaths are there: o ioscan -fnC disk o strings /etc/vpath.cfg

/opt/IBMdpo/bin/showvpath 5. Create the PV o For each disk device (not vpath) issue a "pvcreate /dev/rdsk/cxtxdx" o Note for vpaths this information can be found in /etc/vpath.cfg
o

How to create Physical Volumes:

3 Methods of Using pvcreate: o pvcreate /dev/rdsk/cxtxdx Note this command uses the "r" version of the device o pvcreate -f /dev/rdsk/cxtxdx Note: the -f option will overwrite existing pv data on the disk device Note this command uses the "r" version of the device o pvcreate -B /dev/rdsk/cxtxdx Makes the PV boot capable Note this command uses the "r" version of the device

How to determine available disks to be used in a Volume Group:

"ioscan -funC disk" will list all of the disk devices 1. Some of these devices will be allocated, some will not 2. "vgdisplay -v" will list all of the PVs and their devices for all of the existing volume groups 1. This is a list of the devices that are in use 3. Any devices that are on the ioscan, but are NOT on the vgdisplay are available for use 1. Possible strategy to automate this process using sed, awk and grep 1. create a file that has all of the disks that can be used (in this example HITACHI)
1.

1. cat wte_disk_ioscan | sed 1,2d | grep -v -e TOSHIBA -e c2t0d0 | xargs -n 10 | grep HITACHI | grep -vi subsystem > wte_hitachi_disk 1. cat will output the file that contains the disk ioscan (ioscan -fnC disk) 2. sed is used to delete the first 2 header lines of the file. 3. Next grep is used to print any lines that DO NOT include TOSHIBA or c2t0d0 4. xargs is used to group the output into groups of 10

5. Next grep finds all of the lines with HITACHI in them 6. All of the lines that have HITACHI in them are saved to a file 1. Refine the ioscan of HITACHI disks to include just the disk devices, sorted - this is a list of all HITACHI disks on the system 1. awk '{ print $9 }' wte_hitachi_disk | sort -u > wte_hitachi_sorted_u 1. awk prints just the 9th field of the file 2. sort - u sorts the file and surpresses any duplicates 3. This is saved to a sorted file 2. Print a list of all the disks that are currently being used (a list of PVs) 1. vgdisplay -v | grep "PV Name" > wte_pvdisk_used 1. vgdisplay -v prints a verbose listing of all volume groups 2. grep only prints lines that contain PV Name 3. The list of PVs is saved to a file 3. Refine the list of disks that are being used 1. awk '{ print $3 }' wte_pvdisk_used | sort -u > wte_pvdisk_sorted_u 1. awk prints on the 3rd field (the disk device) 2. sort will sort the list, surpressing any duplicate entries 3. the results are saved to a file 4. Compare the 2 files - the list of all Hitachi disks on the system with the list of all disks being used 1. diff wte_hitachi_sorted_u wte_pvdisk_sorted_u 1. diff compares the 2 files and prints out any differences. The difference will be a disk that the system sees, but that is not being used by LVM
How to remove PVs

Identify the hardware path of the disk to remove o ioscan -fnC disk Remove the special device file o rmsf -H <HW Path from ioscan>

How to perform an non-destructive test of a disk: dd if=/dev/rdsk/cxxxxxx of=/dev/null bs=1024k