1. Purpose
This document describes disks and filesystems.
2. scope
The Operation Checklist addresses the Filesystem creation
/extention with disk process and defines usage instructions for creating and
publishing checklists, preliminary configuration information and prerequisite
requirements, procedural steps, and verification activities
3. Filesystem and disk
3.1 Add disks (LUNs) to volume group / logical volume
To add new LUNs to the system you can use the following
procedure: The ticket or SAN team will inform the following information
(probably under comments): cu:ldev of disk presented to Unix server via
requested HBA.
The lun number points to the XP-ARRAY. This number is of no
importance. The hwu number is the unique internal address to the storage. Use
the following command to find out the corresponding device number:
Xpinfo will
#-> /usr/local/CPR/bin/xpinfo -il >xpinfo.output
&
#-> grep -iE '40:90|40:91'
/home/ssingh45/xpinfo.output |grep -i
<array number>
/dev/rdisk/disk32
dc --- 00 CL5B
40:90 OPEN-V 00090451
/dev/rdisk/disk33
d1 --- 01 CL6B
40:91 OPEN-V 00090451
/dev/rdisk/disk34
d1 --- 01 CL6B
40:92 OPEN-V 00090451
/dev/rdisk/disk35
d1 --- 01 CL6B
40:93 OPEN-V 00090451
Depednig on the version of OS, cul:dev will give the device
file output as legacy or persisteant device file (11.31)
3.2 Verificatoin of disk
# insf
(Note: if this does not work try 'insf -e' to force
re-scanning of all devices). Search for devices (u = usable; quicker):
# ioscan –funCdisk
Create a physical volume for every physical device:
Check the disk provided by SAN is not been used and it is
new on server.
#-> strings /etc/lvmtab |grep -i -e
disk32 –e disk33 -e disk34 –e disk35
Above command should not return any output , it will show no
disk are part of any VG/LV on the system.
#->
cat <<'EOF'
>> ~ssingh45/disk.output
/dev/disk/disk32
/dev/disk/disk33
/dev/disk/disk34
/dev/disk/disk35
EOF
#-> cat
~ssingh45/disk.output |while read disk
> do
> pvdisplay $disk
> done
Expected output
pvdisplay: Warning: couldn't query physical volume
"/dev/disk/disk24":
The specified path does not correspond to physical volume
attached to
this volume group
3.3 Initialization of disk for filesystem creation
# pvcreate /dev/rdsk/c.t.d.
Physical volume "/dev/rdsk/c.t.d."
has been successfully created.
If the system complains (because it finds the device is
already in use) and you are sure the volume can be created, you can use the
'-f' option to force the command:
# pvcreate -f /dev/rdsk/c.t.
Create the device directory for the volume group:
3.4 volume group creation
# mkdir /dev/VGNAME
Search for the next available major device number. First
display the existing:
# ll /dev/*/group
> crw-r----- 1 root
sys 64 0x000000 Nov
19 1999 group
> crw-r--r-- 1 root
sys 64 0x020000 Dec
11 2002 group
> crw------- 1 root
sys 64 0x010000 May
15 2001 group
Watch Out: The (hexadecimal) number after 64 is the
node-number. Create a new one by choosing the next available, in this case
0x030000:
# mknod /dev/VGNAME/group c 64 0x030000
Now add the physical volumes to the volume group. When you
don't use 'alternate pathing' (= two separate paths to same device) then you
can leave out the last dev:
# vgcreate /dev/VGNAME /dev/dsk/c.t.d.
/dev/dsk/c.t.d.
Volume group "/dev/VGNAME" has been
successfully created.
Volume Group configuration for /dev/VGNAME
has been saved in /etc/lvmconf/VGNAME.conf
Add the other physical volumes to the vg (if necessary):
LVM Parameter
|
Default Value
|
Maximum Value
|
Can be set by
|
Comments
|
||||||||||||||||
Max
# of Volume Groups
|
10
|
255
|
Kernel
parameter maxvgs
|
HP-UX
11.11 – Default value in the OS image is 10.
HP-UX
11.23 - Default value in the OS image is 256.
HP-UX
11.31 – Kernel parameter is obsolete. The default value is 256 when using LVM
1.0.
|
||||||||||||||||
#
of physical volumes(PVs) per volume group(VG)
|
16
|
255
|
vgcreate
–p <max_pv>
|
The
default is too low and must be set when the volume group is created.
|
||||||||||||||||
#
of logical volumes(LVs) per Volume group(VG)
|
255
|
255
|
vgcreate
–l <max_lv>
|
This
setting can be left at the default setting or lowered.
|
||||||||||||||||
Physical
Extent size
|
4MB
|
256MB
|
vgcreate
–s <pe_size>
|
Setting the pe_size has a
direct impact on the max size of a single logical volume. The following table shows the correlation
to this effect.
|
||||||||||||||||
Max
# of Physical Extents(PE) per Physical Volume(PV)
|
1016
|
65535
|
vgcreate
–e <max_pe>
|
Formula
: pe_size * max_pe = Max size of usable disk.
|
# vgextend /dev/VGNAME /dev/dsk/c.t.d.
/dev/dsk/c.t.d.
Display the characteristics of the new volume group:
#vgdisplay -v VGNAME
--- Volume groups ---
VG Name /dev/VGNAME
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 0
Open LV 0
Max PV 16
Cur PV 1
Act PV 1
Max PE per PV 1016
VGDA 2
PE Size (Mbytes) 4
Total PE 538
Alloc PE 0
Free PE 538
Total PVG
0
Total Spare PVs 0
Total Spare PVs in use 0
---
Physical volumes ---
PV
Name
/dev/dsk/c36t14d2
PV
Name
/dev/dsk/c37t14d2 Alternate
Link
PV
Status available
Total PE 538
Free
PE 538
Autoswitch On
VG creation for distributed layout file system.
Physical volume groups can be created by using the "-g" option
on the vgcreate and vgextend commands, or by manually editing the /etc/lvmpvg
file. Refer to lvmpvg(4) man page for syntax.
Physical volume groups are used in conjunction with PVG-strict allocation
policy. PVG-strict allocation ensures that primary and mirror extents do not
reside within the same PVG. Typically each PVG consists of disk devices
from the same controller card or set of controllers. Another example would be
the case where LVs are mirrored between two separate disk arrays - the PVs in
each of the arrays being assigned to separate PVGs.
Example /etc/lvmpvg file:
VG /dev/vgDist
PVG PVG1
/dev/dsk/c0t0d0
/dev/dsk/c0t1d0
/dev/dsk/c0t6d0
/dev/dsk/c0t7d0
PVG PVG2
/dev/dsk/c1t2d0
/dev/dsk/c1t3d0
/dev/dsk/c0t4d0
/dev/dsk/c0t5d0
vgextend –g PVG<NAME> /dev/VGNAME
/dev/dsk/c.t.d. /dev/dsk/c.t.d. /dev/dsk/c.t.d. /dev/dsk/c.t.d. #four way distribution layout.
3.5 logical volume creation.
You can define a size in megabytes by using the '-L' option
or in number of Physical Extents (PE's) by using the '-l' option.
If you want a size equal to the total available size you can use '-l' and the amount of 'Free PE's'
As default lvol1 is taken:
If you want a size equal to the total available size you can use '-l' and the amount of 'Free PE's'
As default lvol1 is taken:
3.5.1 Strict logical creatoin
# lvcreate -l SIZE /dev/VGNAME
3.5.2 PVG-strict /distributed logical creation
# lvcreate -D y -s g -m 1 -L 1000 -n LVOL VGNAME
-D y specifies distributed allocation policy
-s g specifies PVG-strict allocation policy
-m 1 specifies 1 mirror copy
-L 1000 specifies size in
MBytes = 1000
-n LVOL names logical volume VGNAME
3.5.3 Strip/distributed logical creatoin
# lvcreate
-i 3 -I 32 -l 240 -n lvol1 /dev/VGNAME
Please use the parameter as per standard what exist on system or
consult standard hpux guidline for Filesystem.
Now the devices should exist in the lvmtab:
# strings /etc/lvmtab|grep c.t.
# ll /dev/VGNAME
total 0
crw-r--r--
1 root sys 64 0x0a0000 Dec 15 14:18 group
brw-r-----
1 root sys 64 0x0a0001 Dec 15 14:30 lvol1
crw-r-----
1 root sys 64 0x0a0001 Dec 15 14:30 rlvol1
Check the new volume group:
# vgdisplay -v /dev/VGNAME
create a filesystem on the nem logical volume:
# newfs -o largefiles -F vxfs
/dev/VGNAME/rlvol.
version 3 layout
2203648 sectors, 2203648 blocks of size 1024, log size 1024 blocks
unlimited
inodes, 2203648 data blocks, 2202000 free data blocks
68
allocation units of 32768 blocks, 32768 data blocks
last
allocation unit has 8192 data blocks
first allocation unit starts at block 0
overhead per allocation unit is 0 blocks
Note that following command must be used to create new
sapdata-filesystems:
# newfs -F vxfs -b 8192 /dev/VGNAME/rlvol
(blocksize of 8kB is needed by SAP/Oracle).
(blocksize of 1kB is needed by Redo / Mirr logs).
Create a mount-point:
# mkdir /.....
#-> cp -p /etc/fstab /etc/fstab.ssingh45-`date
+%d-%m-%Y`
Add the necessary entries to /etc/fstab:
# vi /etc/fstab
Try to mount the new filesystems:
# mount -a
#mount -v #
Mounting option as per file system.
Type of File
System
|
Required
Blocksize (use –o bsize with mkfs)
|
Required Mount
options
|
Comments
|
Oracle redolog file systems
|
1K
|
largefiles,rw,mincache=direct,convosync=direct,delaylog,nodatainlog
|
Striping is not required. These file systems should not be
on the same disks as the Oracle Indexes and Datafiles.
|
Oracle archive log file systems
|
1K
|
largefiles,rw,mincache=direct,convosync=direct,delaylog,nodatainlog
|
Striping is not required. Can be on the same disks as the
Oracle redologs.
|
Oracle Indexes and
Datafiles
|
8K
|
largefiles,rw,delaylog,nodatainlog
|
Striping is required. Create file systems using
Distributing extent based Striping
|
General Purpose File systems
|
8K
|
largefiles,rw,delaylog,nodatainlog
|
Striping can be used but is not required. If striping is
used, create the file system(s) using Distributed striping.
|
Set the permissions for the new top dir :
# chown ...:...
/.....
# chmod ... /.....
Check the new situation:
# bdf
# vgdisplay -v /dev/VGNAME
3.6 Validaton of Filesystem creation
Check the layout of the Filesystem as per standard and
requested.
#-> fstyp -v /dev/VGNAME/LVOL
vxfs
version: 6 <version of the VxFS>
f_bsize: 8192 ß This value represents the maximum
block size that can be set. This value cannot be changed.
f_frsize: 2048 ß This value represents the
fragmentation block size which was set using –o bsize option
f_blocks: 4612096 To understand more about f_bsize and f_frsize you can review the man
page for statvfs
f_bfree: 4602773
f_bavail: 4458937
f_files: 1150720
f_ffree: 1150688
f_favail: 1150688
f_fsid: 1074987009
f_basetype: vxfs
f_namemax: 254
f_magic: a501fcf5
f_featurebits: 0
f_flag: 16
f_fsindex: 9
f_size: 4612096
#->lvdisplay -v /dev/VGNAME/LVOL
This will show the value and parameter which are used during
the LV creation.
3.7 Adding the disk into secure path
Secure path is used for loadbalacing of I/O on the server.
it is very important to set the policy as per guidline.
For 11.11 and 11.23
autopath discover
autopath display | grep -e /dev/dsk | awk '{print $1}' | while
read device
do
autopath set_lbpolicy SST $device
done
For 11.31 :
For 11iv3 systems make sure the
load balance policy is also set on the newly added disk:
Verify LB policy: scsimgr -p
get_attr all_lun -a device_file -a load_bal_policy
Set LB policy on all disks: scsimgr
save_attr -N "/escsi/esdisk" -a load_bal_policy=<LB policy>
Set LB policy on one disk: scsimgr save_attr -D /dev/rdisk/disk530 -a
load_bal_policy=<LB policy>
No comments:
Post a Comment