From:
http://abcdba.com/abcdbarachowtoapplyrollingopatch
OVERVIEW
An Oracle OPatch for a RAC configuration can take take one of two forms, either as a Minimum Downtime OPatch or as a Rolling Opatch. The instructions for applying the former can be foundhere. Instructions for performing the latter can be found below.
In addition to being informed by Oracle Support, the type of OPatch can be determined by checking the <Patch No>/etc/config/inventory file. If the the variable online_rac_installable is set totrue then the patch is a rolling patch.
PATCH INSTALLATION INSTRUCTIONS
- The worked example below will use the following settings:
- CRS_HOME = /u01/crs/oracle/product/10/crs
- RDBMS_HOME = /u01/app/oracle/product/db10.2.0
- The patch being applied is p123456
- The example output from the various commands has been abridged for readability.
1. Make sure all instances running under the ORACLE_HOME being patched are cleanly shutdown before installing this patch. Also ensure that the tool used to terminate the instance(s) has exited cleanly.
2. Ensure that the directory containing the opatch script appears in your $PATH. Execute which opatch to confirm.
oracle:> which opatch
no opatch in /u01/app/oracle/product/db10.2.0/bin /bin /usr/sbin /usr/bin /usr/local/bin /usr/ccs/bin
/usr/bin /usr/local/bin /u01/crs/oracle/product/10/crs/bin /u01/crs/oracle/product/10/crs/bin
oracle:> export PATH=$PATH:$ORACLE_HOME/OPatch
oracle:> which opatch
/u01/crs/oracle/product/10/crs/OPatch/opatch
3. Each node of the cluster has its own CRS Home, the patch should be applied as a rolling upgrade. All of the following
steps should be followed for each node.
steps should be followed for each node.
- Do not patch two nodes at once.
4. As the Oracle Clusterware (CRS) software owner check CRS_HOME.
oracle:> opatch lsinventory -detail -oh /u01/crs/oracle/product/10/crs
Invoking OPatch 10.2.0.3.0
...
Remote node = <node x>
--------------------------------------------------------------------------------
OPatch succeeded.
5. As the RDBMS server owner check ORACLE_HOME.
oracle:> opatch lsinventory -detail -oh /u01/app/oracle/product/db10.2.0/
Invoking OPatch 10.2.0.3.0
...
Remote node = <node x>
--------------------------------------------------------------------------------
OPatch succeeded.
- The above should list the components and the list of nodes. If the Oracle inventory is not setup correctly the OPatch utility will fail.
6. Unzip the patch set container file, this will create one or more sub-directories.
% unzip p123456.zip
Archive: p123456.zip
creating: 123456/
...
7. Shut down the RDBMS and ASM instances, listeners and nodeapps followed by CRS daemons on the local node.
- To shutdown RDBMS instance on the local node run the following command:
% $ORACLE_HOME/bin/srvctl stop instance -d dbname -i instance_name
- To shutdown ASM instances run the following command on each node:
% $ORACLE_HOME/bin/srvctl stop asm -n <node_name>
- To shutdown nodeapps run the following comand on each node:
% $ORACLE_HOME/bin/srvctl stop nodeapps -n <node_name>
8. Now shutdown CRS daemons on each node by running as root:
root # $CRS_HOME/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
9. Prior to applying this part of the fix, invoke the unlock script as root to unlock protected files.
su -
root # cd <patch directory>/123456
root # custom/scripts/prerootpatch.sh -crshome /u01/crs/oracle/product/10/crs -crsuser oracle
root # exit
10. Now invoke an additional script as the crs software installer/owner. This script will save important configuration settings.
oracle:> cd <patch directory>/123456
custom/scripts/prepatch.sh -crshome /u01/crs/oracle/product/10/crs
custom/scripts/prepatch.sh completed successfully.
- Note: Make sure the RDBMS portion is only applied to an RDBMS home that meets all the pre-requiste versions.
oracle:> cd <patch directory>/123456
oracle:> custom/server/123456/custom/scripts/prepatch.sh -dbhome /u01/app/oracle/product/db10.2.0
custom/server/123456/custom/scripts/prepatch.sh completed successfully.
11. After unlocking any protected files and saving configuration settings run opatch as the Oracle Clusterware (CRS) software owner.
cd <patch directory>/123456
oracle:> opatch apply -local -oh /u01/crs/oracle/product/10/crs
Invoking OPatch 10.2.0.3.0
Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..
Oracle Home : /u01/crs/oracle/product/10/crs
Central Inventory : /u01/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.3.0
OUI version : 10.2.0.3.0
OUI location : /u01/crs/oracle/product/10/crs/oui
Log file location : /u01/crs/oracle/product/10/crs/cfgtoollogs/opatch/opatch2008-02-28_11-51-38AM.log
ApplySession applying interim patch '123456' to OH '/u01/crs/oracle/product/10/crs'
ApplySession: Optional component(s) [ oracle.rdbms, 10.2.0.3.0 ]
not present in the Oracle Home or a higher version is found.
Invoking fuser to check for active processes.
You selected -local option, hence OPatch will patch the local system only.
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/crs/oracle/product/10/crs')
Is the local system ready for patching?
Do you want to proceed? [y|n]
y
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '123456' for restore. This might take a while...
Backing up files affected by the patch '123456' for rollback. This might take a while...
Patching component oracle.rdbms.rsf, 10.2.0.3.0...
Updating archive file "/u01/crs/oracle/product/10/crs/lib/libgeneric10.a" with "lib/libgeneric10.a/skgfr.o"
Updating archive file "/u01/crs/oracle/product/10/crs/lib32/libgeneric10.a" with "lib32/libgeneric10.a/skgfr.o"
ApplySession adding interim patch '123456' to inventory
Verifying the update...
Inventory check OK: Patch ID 123456 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 123456 are present in Oracle Home.
Running make for target client_sharedlib
The local system has been patched and can be restarted.
OPatch succeeded.
12. Use opatch to patch the RDBMS software.
oracle:> opatch apply -local -oh /u01/app/oracle/product/db10.2.0
Invoking OPatch 10.2.0.3.0
Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..
Oracle Home : /u01/app/oracle/product/db10.2.0
Central Inventory : /u01/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.3.0
OUI version : 10.2.0.3.0
OUI location : /u01/app/oracle/product/db10.2.0/oui
Log file location : /u01/app/oracle/product/db10.2.0/cfgtoollogs/opatch/opatch2008-02-28_11-53-12AM.log
ApplySession applying interim patch '123456' to OH '/u01/app/oracle/product/db10.2.0'
Invoking fuser to check for active processes.
Invoking fuser on "/u01/app/oracle/product/db10.2.0/bin/oracle"
You selected -local option, hence OPatch will patch the local system only.
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/db10.2.0')
Is the local system ready for patching?
Do you want to proceed? [y|n]
y
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '123456' for restore. This might take a while...
Backing up files affected by the patch '123456' for rollback. This might take a while...
Patching component oracle.rdbms.rsf, 10.2.0.3.0...
Updating archive file "/u01/app/oracle/product/db10.2.0/lib/libgeneric10.a" with "lib/libgeneric10.a/skgfr.o"
Updating archive file "/u01/app/oracle/product/db10.2.0/lib32/libgeneric10.a" with "lib32/libgeneric10.a/skgfr.o"
Patching component oracle.rdbms, 10.2.0.3.0...
ApplySession adding interim patch '123456' to inventory
Verifying the update...
Inventory check OK: Patch ID 123456 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 123456 are present in Oracle Home.
Running make for target client_sharedlib
Running make for target ioracle
The local system has been patched and can be restarted.
OPatch succeeded.
- After opatch completes, some configuration settings need to be applied to the patched files.
13. As the RDBMS software owner execute the following:
% <patch directory>/123456/custom/server/123456/custom/scripts/postpatch.sh -dbhome <RDBMS_HOME>
oracle:> <patch directory>/123456/custom/server/123456/custom/scripts/postpatch.sh \
> -dbhome /u01/app/oracle/product/db10.2.0
Reading ...
...
Reapplying file permissions on /u01/app/oracle/product/db10.2.0/lib/libsrvmhas10.so
14. Restore the security settings and restart CRS by running the following as root:
su -
Sourcing /root/.profile-EIS.....
root # cd <patch directory>/123456
root # custom/scripts/postrootpatch.sh -crshome /u01/crs/oracle/product/10/crs
Checking to see if Oracle CRS stack is already up...
Checking to see if Oracle CRS stack is already starting
WARNING: directory '/u01/crs/oracle/product/10' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
Startup will be queued to init within 30 seconds.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
PATCH DEINSTALLATION INSTRUCTIONS
To roll back the patch, invoke the following opatch commands to roll back the patch in all homes.
% opatch rollback -id 123456 -local -oh <CRS_HOME>
% opatch rollback -id 123456 -local -oh <RDBMS_HOME>
- Note: Any problems installing a patch should have a call logged with Oracle support.
OVERVIEW
An Oracle OPatch for a RAC configuration can take take one of two forms, either as a Rolling Opatch or as a Minimum Downtime OPatch. The instructions for applying the former can be foundhere. Instructions for performing the latter can be found below.
In addition to being informed by Oracle Support, the type of OPatch can be determined by checking the <Patch No>/etc/config/inventory file. If the the variable online_rac_installable is set totrue then the patch is a rolling patch.
A WORKED EXAMPLE
1. Ensure all of the instances that make up the RAC database can be shutdown.
2. On each node, shutdown the instance, using the following commands:
. oraenv
<SID name>
sqlplus '/ as sysdba'
shutdown transactional
exit
3. On the primary node in the cluster follow the Opatch instructions as prescribed by theREADME.txt file.
- Here is a real example:
cd $HOME/DBA/patches
unzip p6239052_10203_SOLARIS64.zip
cd 6239052
export OBJECT_MODE=32_64
export PATH=$ORACLE_HOME/OPatch:$PATH
opatch apply
Invoking OPatch 10.2.0.3.0
Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..
Oracle Home : /u01/app/oracle/product/db10.2.0
Central Inventory : /u01/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.3.0
OUI version : 10.2.0.3.0
OUI location : /u01/app/oracle/product/db10.2.0/oui
Log file location : /u01/app/oracle/product/db10.2.0/cfgtoollogs/opatch/opatch2007-11-20_02-17-48AM.log
ApplySession applying interim patch '6239052' to OH '/u01/app/oracle/product/db10.2.0'
Invoking fuser to check for active processes.
Invoking fuser on "/u01/app/oracle/product/db10.2.0/bin/oracle"
OPatch detected the node list and the local node from the inventory.
OPatch will patch the local system then propagate the patch to the remote nodes.
This node is part of an Oracle Real Application Cluster.
Remote nodes: 'node2' 'node3'
Local node: 'node1'
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/db10.2.0')
Is the local system ready for patching?
Do you want to proceed? [y|n]
y
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '6239052' for restore. This might take a while...
Backing up files affected by the patch '6239052' for rollback. This might take a while...
Patching component oracle.rdbms, 10.2.0.3.0...
Updating archive file "/u01/app/oracle/product/db10.2.0/lib/libserver10.a"
with "lib/libserver10.a/kjbm.o"
ApplySession adding interim patch '6239052' to inventory
Verifying the update...
Inventory check OK: Patch ID 6239052 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 6239052 are present in Oracle Home.
Running make for target ioracle
The local system has been patched. You can restart Oracle instances on it.
Patching in rolling mode.
Remaining nodes to be patched:
'node2' 'node3'
What is the next node to be patched?
node2
You have selected 'node2' from 'node2' 'node3'
The node 'node2' will be patched next.
Please shutdown Oracle instances running out of this ORACLE_HOME on 'node2'.
(Oracle Home = '/u01/app/oracle/product/db10.2.0')
Is the node ready for patching?
Do you want to proceed? [y|n]
y
User Responded with: Y
Updating nodes 'node2'
Apply-related files are:
FP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt"
DP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_dirs.txt"
MP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/make_cmds.txt"
Instantiating the
file "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt.instantiated"
by replacing $ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt"
with actual path.
Propagating files to remote nodes...
Instantiating the file
"/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_dirs.txt.instantiated" by replacing
$ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_dirs.txt" with actual path.
Propagating directories to remote nodes...
Instantiating the file
"/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/make_cmds.txt.instantiated" by replacing
$ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/make_cmds.txt" with actual path.
Running command on remote node 'node2': cd /u01/app/oracle/product/db10.2.0/rdbms/lib;
/usr/ccs/bin/make -f ins_rdbms.mk
ioracle ORACLE_HOME=/u01/app/oracle/product/db10.2.0 || echo REMOTE_MAKE_FAILED::>&2
The node 'node2' has been patched. You can restart Oracle instances on it.
The node 'node3' will be patched next.
Please shutdown Oracle instances running out of this ORACLE_HOME on 'node3'.
(Oracle Home = '/u01/app/oracle/product/db10.2.0')
Is the node ready for patching?
Do you want to proceed? [y|n]
y
User Responded with: Y
Updating nodes 'node3'
Apply-related files are:
FP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt"
DP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_dirs.txt"
MP = "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/make_cmds.txt"
Instantiating the file
"/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt.instantiated" by replacing
$ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_files.txt" with actual path.
Propagating files to remote nodes...
Instantiating the file
"/u01/app/oracle/product/db10.2.0/.patch_storage/.../rac/copy_dirs.txt.instantiated" by replacing
$ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/....txt" with actual path.
Propagating directories to remote nodes...
Instantiating the file
"/u01/app/oracle/product/db10.2.0/..../rac/make_cmds.txt.instantiated" by replacing
$ORACLE_HOME in "/u01/app/oracle/product/db10.2.0/.patch_storage/....txt" with actual path.
Running command on remote node 'node3': cd /u01/app/oracle/product/db10.2.0/rdbms/lib;
/usr/ccs/bin/make -f ins_rdbms.mk
ioracle ORACLE_HOME=/u01/app/oracle/product/db10.2.0 || echo REMOTE_MAKE_FAILED::>&2
The node 'node3' has been patched. You can restart Oracle instances on it.
There were relinks on remote nodes.
Remember to check the binary size and timestamp on the nodes 'node2' 'node3' .
The following make commands were invoked on remote nodes:
'cd /u01/app/oracle/product/db10.2.0/rdbms/lib; /usr/ccs/bin/make -f ins_rdbms.mk
ioracle $ORACLE_HOME=/u01/app/oracle/product/db10.2.0'
OPatch succeeded.
4. Ensure the new patch appears in the inventory by running the following command:
opatch lsinventory
Invoking OPatch 10.2.0.3.0
Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..
Oracle Home : /u01/app/oracle/product/db10.2.0
Central Inventory : /u01/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.3.0
OUI version : 10.2.0.3.0
OUI location : /u01/app/oracle/product/db10.2.0/oui
Log file location : /u01/app/oracle/product/db10.2.0/cfgtoollogs/opatch/....log
Lsinventory Output file location : /u01/app/oracle/product/db10.2.0/....txt
--------------------------------------------------------------------------------
Installed Top-level Products (2):
Oracle Database 10g 10.2.0.1.0
Oracle Database 10g Release 2 Patch Set 2 10.2.0.3.0
There are 2 products installed in this Oracle Home.
Interim patches (5) :
Patch 6239052 : applied on Tue Nov 20 02:18:58 GMT 2007
Created on 31 Oct 2007, 08:01:04 hrs PST8PDT
Bugs fixed:
6239052
Patch 4655998 : applied on Fri May 04 10:23:04 BST 2007
Created on 13 Apr 2007, 00:31:22 hrs PST8PDT
Bugs fixed:
4655998
Patch 5749953 : applied on Fri Apr 20 16:13:34 BST 2007
Created on 15 Feb 2007, 02:45:16 hrs US/Pacific
Bugs fixed:
5749953
Patch 5556081 : applied on Mon Apr 16 10:18:22 BST 2007
Created on 9 Nov 2006, 22:20:50 hrs PST8PDT
Bugs fixed:
5556081
Patch 5557962 : applied on Mon Apr 16 10:17:33 BST 2007
Created on 9 Nov 2006, 23:23:06 hrs PST8PDT
Bugs fixed:
4269423, 5557962, 5528974
Rac system comprising of multiple nodes
Local node = node1
Remote node = node2
Remote node = node3
--------------------------------------------------------------------------------
OPatch succeeded
5. On each node, start the instance, using the following commands:
. oraenv
<SID name>
sqlplus '/ as sysdba'
startup
exit
6. On the primary node, ensure the RAC cluster has returned to normal with the following commands:
- Note: It takes a few minutes to achieve a completely ONLINE state.
cd $HOME
. ./10gcrs.env
crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....L01.srv application ONLINE ONLINE node1
ora....L02.srv application ONLINE ONLINE node2
ora....L03.srv application ONLINE ONLINE node3
...
ora....b03.gsd application ONLINE ONLINE node3
ora....b03.ons application ONLINE ONLINE node3
ora....b03.vip application ONLINE ONLINE node3
TROUBLESHOOTING
- If the OPatch is NOT applied to the primary node in the cluster then the following error output will be produced:
- ACTION: Restart the patch process on the primary node.
Invoking OPatch 10.2.0.3.0
Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..
Oracle Home : /u01/app/oracle/product/db10.2.0
Central Inventory : /u01/app/oracle/oraInventory
from : /var/opt/oracle/oraInst.loc
OPatch version : 10.2.0.3.0
OUI version : 10.2.0.3.0
OUI location : /u01/app/oracle/product/db10.2.0/oui
Log file location : /u01/app/oracle/product/db10.2.0/cfgtoollogs/opatch/opatch2007-11-20_02-16-36AM.log
ApplySession applying interim patch '6239052' to OH '/u01/app/oracle/product/db10.2.0'
OPatchSession cannot load inventory for the given Oracle Home /u01/app/oracle/product/db10.2.0. Possible #
causes are:
No read or write permission to ORACLE_HOME/.patch_storage
Central Inventory is locked by another OUI instance
No read permission to Central Inventory
The lock file exists in ORACLE_HOME/.patch_storage
The Oracle Home does not exist in Central Inventory
ApplySession failed: ApplySession failed to prepare the system. No detail available.
System intact, OPatch will not attempt to restore the system
OPatch failed with error code 73
- If the crs_stat -t output doesn't show all items in an ONLINE state after a suitable amount of time, e.g. 15 mins, the cluster may require restarting completely.
- ACTION: On each node, ONE NODE AT A TIME, use the following commands:
su -
<root password>
sync;sync;reboot
No comments:
Post a Comment