RHEL4 Kernel – PowerPath – OCFS2 upgrade procedure

We have been working on a project for a customer to upgrade the kernel, PowerPath, OCFS2 and other operating system patches.  The project was interesting as no single source of information existed on the appropriate process.

The procedure issue comes in because OCFS2 relies on seeing the disks which PowerPath presents.  PowerPath and OCFS2 are reliant on a particular kernel version to work properly.  With all of the inter-dependencies,  which should be done and in what order was the question. To top it off, this was also running a RHEL cluster and another instance running Oracle RAC.

Comment: Why not just use device-mapper-multipath? It saves you a chunk of cash, makes applying patches/upgrades easier and does the same thing.  I have implemented this solution (dm-multipath)  in numerous critical SAP/Oracle environments on RHEL4/5 and SLES9/10.

Kernel Upgrade Procedure

– Current versions
RHEL: 4.3
Kernel: 2.6.9-34.0.1.ELsmp
PowerPath: EMCpower.LINUX-4.5.1-022


– New versions
RHEL: 4.7
Kernel: 2.6.9-78.0.5
PowerPath: 5.1 SP2
Ocfs2: 1.2.9-1

Upgrade path:   Stop services and devices -> OS Patches -> PowerPath Patches -> OCFS2 Patches-> Reboot -> Post changes

Upgrade process

# powermt save
# mkdir /var/backup
# tar cvf /var/backup/powerpath.`date +%Y%m%d`.backup.tgz /etc/powermt.custom /etc/emcp_registration

– Stop SAP.
– Unmount filesystems.

# /etc/init.d/rgmanager stop
# chkconfig rgmanager off

– Umount all OCFS2 volumes.
# umount -at ocfs2

– Stop OCFS2
# /etc/init.d/o2cb offline
# /etc/init.d/o2cb unload

– stop ocfs2 on reboot (just to be safe, and to keep it from being activated while the other node is active with an older version).
# chkconfig o2cb off
# chkconfig ocfs2 off

– Deactivate any volume groups using PowerPath devices

# powermt display dev=all > /var/backup/powermt.display.out

– Stop the navisphere agent if running
# /etc/init.d/naviagent stop
# chkconfig naviagent off

– Stop all cluster related software in preparation for the patches
# /etc/init.d/fenced stop
# service cman stop

# chkconfig fenced off
# chkconfig cman off
# chkconfig ccsd off

– Ensure /etc/yum.conf has the kernel exclude removed.
– Ensure the /etc/yum.repos.d/internal.repo is up-to-date.

– Stop powerpath
# /etc/init.d/PowerPath stop

– Apply system updates.
# yum update

– Update powerpath (these are in /opt/patches)
# rpm -Uhv EMCpower.LINUX-

– Update OCFS2 modules (these are in /opt/patches)
# rpm -Uhv ocfs2console-1.2.7-1.el4.x86_64.rpm ocfs2-tools-1.2.7-1.el4.x86_64.rpm ocfs2-tools-debuginfo-1.2.7-1.el4.x86_64.rpm ocfs2-2.6.9-78.0.5.ELsmp-1.2.9-1.el4.x86_64.rpm


Delete the symlinks in /lib/module/`uname –r`/weak-update/
# rm -f /lib/modules/2.6.9-78.0.5.ELsmp/weak-updates/emcplib.ko

# /etc/init.d/PowerPath start

– re-enable ocfs2 service
# /etc/init.d/o2cb start
# /etc/init.d/ocfs2 start

# chkconfig ccsd on
# chkconfig cman on
# chkconfig fenced on
# chkconfig naviagent  on
# chkconfig ocfs2 on
# chkconfig o2cb on
# chkconfig rgmanager  on

Further detail on the OCFS2 upgrade process http://oss.oracle.com/projects/ocfs2/dist/documentation/v1.2/ocfs2_faq.html

Stop worrying about your server issues

Click here for additional detail or request a proposal so you can start focusing on growing your business, rather than supporting your servers.