This blog is collection of daily issue reported in infrastructure support. And this Blog is dedicated to all ADMIN (ROCKSTARS) for their contribution. Daily fight and struggle with (monster) server to deliver it flawlessly to (King)Client
Sunday, September 22, 2013
HPUX : NETTL LOG ARE NOT GETTING COLLECTED
HPUX : EXT_BUS LIMITATION ISSUE ON 11.23
Issue : as extn_bus has cross the limit of 255 limit, extn_bus number greater then 255 has been marked by kernal as no_hw, but IO request can be observe on device file in " sar -d 1 10 |grep -i <c?t?d? which is in no_hw status>
Recommendation:
Need to look for possibility to release unsed extn_bus and reconfigure the ioconfig tree.
/var/adm/crash/ioconfig/ioconfig_dump.exe
## ioconfig_dump -w -o /tmp/ioconfig.new
# ioconfig_dump -r -o /tmp/io.ascii.old
# grep ext_bus /tmp/io.ascii.old|wc -l
#Backup /stand/ioconfig and /etc/ioconfig and copy ioconfig.new into both locations.
# cp /stand/ioconfig /stand/ioconfig.orig
# cp /etc/ioconfig /etc/ioconfig.orig
# cp /tmp/ioconfig.new /stand/ioconfig
# cp /tmp/ioconfig.new /etc/ioconfig
Saturday, September 21, 2013
HPUX : HOW TO GENERATE A TEST EVENT ON CLIENT FOR HPSIM
HPUX : SIMPLE SCRIPT TO GET IMPORTANT INFO FROM SYSTEM TO COMPARE
HPUX : HOW TO USE CPROP IN BLADE SYSTEM TO GATHER INFORMATION
Tuesday, September 17, 2013
HPUX :VPAR IS SHOWING IN CRASH STATUS IN VPARSTATUS FOR LONG TIME
[Virtual Partition]
Boot
Virtual Partition Name State Attributes Kernel Path Opts
============================== ===== ============ ======================= =====
Vjupitor Crash Dyn,Auto,Nsr /stand/vmunix -lq
Vpluto Up Dyn,Auto,Nsr /stand/vmunix -lq
CPU Num Num Memory Granularity
Virtual Partition Name Min/Max CPUs IO ILM CLM
============================== ======= ==== ==== ========== ==========
Vjupitor 2/ 6 6 6 1024 1024Vpluto 2/ 4 2 6 1024 1024
ILM CLM
# User # User
Virtual Partition Name Ranges/MB Total MB Ranges/MB Total MB
============================== ====================== ======================
Vjupitor 0/ 0 64831 0/ 0 0Vpluto 0/ 0 24576 0/ 0 0
(A FULL dump is not possible.)
*** Enter your selection now.
*** A system crash has occurred. (See the above messages for details.)
*** The system is now preparing to dump physical memory to disk, for use
*** in debugging the crash.
*** To change this dump type, press any key within 10 seconds.
*** The dump will be a SELECTIVE dump: 11036 of 64796 megabytes.
*** To change this dump type, press any key within 10 seconds.
S) The dump will be a SELECTIVE dump: 11036 of 64796 megabytes.
P) The dump will be a PARTIAL dump: 24166 of 64796 megabytes.
(A FULL dump is not possible.)
*** Enter your selection now.
HPUX : TTYTYPE: COULDN'T OPEN /DEV/TTY FOR READING
Saturday, September 14, 2013
HPUX : NOT ABLE TO DELETE THE QUEUE.
NETBACKUP : NETBACKUP FS NOT RELEASING THE SPACE
Issue:
After Deleting the large log file from them system utilization of FS did not happen.
#/opt/atdd/bin/lsof /usr/openv/logs # show large number of open file on the Filesystem it was happen due to removal of live file which do not got release by netbackup process.
Recommendation :
Bouncing of Netbackup application. Backup team restarted the NBU.
Friday, September 13, 2013
HPUX : Enabling Legacy Model.
-L Enable the legacy naming model. This command
reinstalls the legacy I/O nodes and legacy device
special files which might have been removed with
rmsf -L (see rmsf(1M)). When used with the -v
option, insf reports on whether legacy mode is
enabled or disabled.
Enable
#insf -v -L
Disable
#rmsf -L
HPUX : Configuring LAN Information for the MP
This section describes how to set and verify the server management processor (MP) LAN port
information. LAN information includes the MP network name, the MP IP address, the subnet mask,
and gateway address. This information is provided by the customer.
To set the MP LAN IP address:
1. At the MP Main Menu prompt (MP>), enter cm.
2. From the MP Command Menu prompt (MP>CM:), enter lc (for LAN configuration).
The screen displays the default values and asks if you want to modify them. It is a good idea
to write down the information or log it to a file, because it may be required for future
troubleshooting.
Enter lc and press the Return key
3. NOTE: The value in the IP address field has been set at the factory. Obtain the LAN IP
address from the customer.
4. At the prompt, Do you want to modify the configuration for the customer
LAN?, enter Y.
The current IP address is shown; then the following prompt is displayed: Do you want to
modify it? (Y/[N])
5. Enter Y.
6. Enter the new IP address.
The customer provides this address for network interface 0.
7. Confirm the new address.
8. Enter the MP host name.
This is the host name for the customer LAN. The name can be up to 64 characters long and
can include alpha numerics, -dash (-), under score (_), period (.), or a space ( ). HP recommends
that the name be a derivative of the complex name, for example, Acme.com_MP.
9. Enter the LAN parameters for Subnet mask and Gateway address.
This information comes from the customer.
When this step is completed, the system will indicate that the parameters have been updated
and return to the MP Command Menu prompt (MP:CM>)
10. To check the LAN parameters and status, enter the ls command at the MP Command Menu
prompt (MP:CM>).
Monday, September 9, 2013
HPUX : Deactivate AUTO_VG_ACTIVATE in cluster aware Env
Friday, September 6, 2013
HPUX : SAVE CRASH MANUALLY
If the dump was not saved completely due to lack of space in the crash directory you have the possibility to save the dump again. The -r option (resave) need to be included when this is not the first time that savecrash runs.
# savecrash -v [-r] <crash directory>
There is also the possibility to save the dump directly to a DDS tape:
# savecrash -v [-r] -t /dev/rmt/0m
The crashconf(1M) command was enhanced to be able to configure dump compression:
# crashconf -c on
# crashconf -v
CLASS PAGES INCLUDED IN DUMP DESCRIPTION
-------- ---------- ---------------- ------------------------------
UNUSED 3645411 no, by default unused pages
USERPG 7113 no, by default user process pages
BCACHE 210990 no, by default buffer cache pages
KCODE 2670 no, by default kernel code pages
USTACK 264 yes, by default user process stacks
FSDATA 116 yes, by default file system metadata
KDDATA 68736 yes, by default kernel dynamic data
KSDATA 259004 yes, by default kernel static data
Total pages on system: 4194304
Total pages included in dump: 328120
Dump compressed: ON
DEVICE OFFSET(kB) SIZE (kB) LOGICAL VOL. NAME
------------ ---------- ---------- ------------ -----------------
31:0x03a000 310112 4194304 64:0x000002 /dev/vg00/lvol2
----------
4194304
Tuesday, September 3, 2013
Monday, September 2, 2013
HPUX :Business volume copy script sync and split is not working, permission issue.
Issue: Business volume copy script sync and split is not working, is logs below message. indicating
problem with authenticatoin.
[root@jupiterA:/var/adm/ExSID_PJ6/DB/exsid.09-02-2013_09-55-25]#
#-> more exsid.log
Warning: Permanently added 'dbcitx6,xxx.xxx.xxx.xxx' (ECDSA) to the list of known h
########################################################################
# #
# WARNING NOTICE: #
# "You are about to enter a Private Network that is intended for the #
# authorized use of a Private Company and its affiliate companies #
# (the "Company") for business purposes only. The actual or attempted #
# unauthorized access,use or modification of this network is strictly #
# prohibited by the Company. Unauthorized users and/or unauthorized #
# use are subject to Company disciplinary proceedings and/or criminal #
# and civil penalties in accordance with applicable domestic and #
# foreign laws. The use of this system may be monitored and recorded #
# for administrative and security reasons. If such monitoring and/or #
# recording reveals possible evidence of criminal activity, the #
# Company may provide the monitored evidence of such activity to law #
# enforcement officials." #
# #
########################################################################
WARNING: Your password has expired.
Password change required but no TTY available.
Warning: Permanently added 'dbcitx6,xxx.xxx.xxx.xxx' (ECDSA) to the list of known hosts.
########################################################################
# #
# WARNING NOTICE: #
# "You are about to enter a Private Network that is intended for the #
# authorized use of a Private Company and its affiliate companies #
# (the "Company") for business purposes only. The actual or attempted #
# unauthorized access,use or modification of this network is strictly #
# prohibited by the Company. Unauthorized users and/or unauthorized #
# use are subject to Company disciplinary proceedings and/or criminal #
# and civil penalties in accordance with applicable domestic and #
# foreign laws. The use of this system may be monitored and recorded #
# for administrative and security reasons. If such monitoring and/or #
# recording reveals possible evidence of criminal activity, the #
# Company may provide the monitored evidence of such activity to law #
# enforcement officials." #
# #
########################################################################
WARNING: Your password has expired.
Password change required but no TTY available.
Warning: Permanently added 'dbcitx6,xxx.xxx.xxx.xxx' (ECDSA) to the list of known h
########################################################################
Recommended action:
Check the account status of the account involve in the script.
Here Root password got expired and causing the script to failed to create the directory on the primary node.
HPUX : UX:vxfs mount: ERROR: V-3-24996: Unable to get disk layout version
After system reboot vg failed to activate , manual activation show system is failed to query the disk. ioscan show the disk status are claimed and available on the system.
Error :
mountall: cannot mount /dev/vg_edwpsw/orasw01_11g
mountall: diagnostics from mount
UX:vxfs mount: ERROR: V-3-20003: Cannot open /dev/vg_edwpsw/orasw01_11g: No such device or address
UX:vxfs mount: ERROR: V-3-24996: Unable to get disk layout version
mountall: cannot mount /dev/vg_edwpsw/orasw01
mountall: diagnostics from mount
UX:vxfs mount: ERROR: V-3-20003: Cannot open /dev/vg_edwpsw/orasw01: No such device or address
UX:vxfs mount: ERROR: V-3-24996: Unable to get disk layout version
mountall: cannot mount /dev/vg_edwpsw/lv_apps
mountall: diagnostics from mount
UX:vxfs mount: ERROR: V-3-20003: Cannot open /dev/vg_edwpsw/lv_apps: No such device or address
UX:vxfs mount: ERROR: V-3-24996: Unable to get disk layout version
[root@jupiterA:/etc/lvmconf]#
#-> grep -i "Wrong disk" /var/adm/syslog/syslog.log
Sep 1 16:42:42 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x00009f Attach failed. Wrong disk!
Sep 1 17:14:45 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x00009f Attach failed. Wrong disk!
Sep 1 17:14:45 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x0000a6 Attach failed. Wrong disk!
Sep 1 17:14:45 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x00011e Attach failed. Wrong disk!
Sep 1 17:14:45 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x00011f Attach failed. Wrong disk!
Sep 1 17:15:13 jupiterA vmunix: LVM: VG 64 0x080000: PVLink 1 0x0000a6 Attach failed. Wrong disk!
[root@jupiterA:/etc/lvmconf]#
Recommendation Action :
Restore the vg cofinguration on the Disk , after verifying the disk and VG association.
#-> vgcfgrestore -f vg_edwpsw.conf.old -l
Volume Group Configuration information in "vg_edwpsw.conf.old"
VG Name /dev/vg_edwpsw
---- Physical volumes : 4 ----
/dev/rdisk/disk149 (Non-bootable)
/dev/rdisk/disk156 (Non-bootable)
/dev/rdisk/disk276 (Non-bootable)
/dev/rdisk/disk277 (Non-bootable)
vgcfgrestore -f /etc/lvmconf/vg_edwpsw.conf.old /dev/rdisk/disk149
vgcfgrestore -f /etc/lvmconf/vg_edwpsw.conf.old /dev/rdisk/disk156
vgcfgrestore -f /etc/lvmconf/vg_edwpsw.conf.old /dev/rdisk/disk276
vgcfgrestore -f /etc/lvmconf/vg_edwpsw.conf.old /dev/rdisk/disk277
[root@jupiterA:/home/ssingh45]#
#-> vgimport -sN -m vg_edwpsw.mapfile.ssingh45 vg_edwpsw
vgimport: Quorum not present, or some physical volume(s) are missing.
vgimport: Volume group "/dev/vg_edwpsw" has been successfully created.
Warning: A backup of this volume group may not exist on this machine.
Please remember to take a backup using the vgcfgbackup command after activating the volume group.
Size of disk in conf and in diskscan are different which causing the conflict. it seems something messy how it was happen. no clue , might someone imported with wrong mapfile.
#-> vgcfgrestore -vl -n vg_edwpsw
Volume Group Configuration information in "/etc/lvmconf/vg_edwpsw.conf"
VG Name /dev/vg_edwpsw
---- Physical volumes : 4 ----
PV Type Size (kb) Start (kb) PVkey
disk149 Non-Boot 104857920 21184 2
disk156 Non-Boot 104857920 21184 3
disk276 Non-Boot 39846720 21184 4
disk277 Non-Boot 39846720 21184 5
max_pv 255 max_pe 10000 max_lv 255 vg_version 1.0
SCSI describe of /dev/rdisk/disk149:
vendor: HP
product id: OPEN-V
type: direct access
size: 39846720 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdisk/disk156:
vendor: HP
product id: OPEN-V
type: direct access
size: 39846720 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdisk/disk276:
vendor: HP
product id: OPEN-V
type: direct access
size: 104857920 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdisk/disk277:
vendor: HP
product id: OPEN-V
type: direct access
size: 104857920 Kbytes
bytes per sector: 512
HPUX : SWINSTALL :PERMISSION ISSUE WHILE INSTALLING THE PRODUCT.
while installation of the product below error enter in swinstall, related the permssion issue on the depot.
--------------
08/15/13 03:04:08 EDT BEGIN swinstall SESSION
(non-interactive) (jobid=jupiterA-0158)
* Session started for user "root@jupiterA".
* Beginning Selection
* Target connection succeeded for "jupiterA:/".
ERROR: "jupiterA:/home/averma10/idmsnags.hpux.depot": You do not
have permission for this operation. The depot owner, system
administrator, or alternate root owner may need to use the
"swreg" or "swacl" command to give you permission. Or, to
manage applications designed and packaged for nonprivileged
mode, see the "run_as_superuser" option in the "sd" man page.
* Source connection failed for
"jupiterA:/home/averma10/idmsnags.hpux.depot".
WARNING: More information may be found in the daemon logfile on this
target (default location is
jupiterA:/var/adm/sw/swagentd.log).
* Selection had errors.
======= 08/15/13 03:04:10 EDT END swinstall SESSION (non-interactive)
(jobid=jupiterA-0158)
Reccommeded action :
Above error indicate depot is not register.
[root@jupiterA:/var/adm/install-logs]#
#-> swreg -l depot /home/averma10/idmsnags.hpux.depot
======= 09/02/13 08:38:09 EDT BEGIN swreg SESSION (non-interactive)
* Session started for user "root@jupiterA".
* Beginning Selection
* Targets: jupiterA
* Objects: /home/averma10/idmsnags.hpux.depot
* Selection succeeded.
======= 09/02/13 08:38:09 EDT END swreg SESSION (non-interactive)
[root@jupiterA:/var/adm/install-logs]#
#-> swlist -l depot
# Initializing...
# Target "jupiterA" has the following depot(s):
/var/opt/mx/depot11
/tmp/ident/itsgimgia11-31-042011
/tmp/idmsnags.hpux.depot
/home/averma10/idmsnags.hpux.depot
Regristratoin of the depot resolve the prermission issue on the depot and this time installation of the
product went fine.