OPatch Methods in RAC

OPatch supported patch methods in RAC environment

Before starting this article let's have an idea about oracle Interim Patch (formerly known as a "one-off" patch). An oracle Interim Patch or "one-off" patch is a bug fix (or set of fixes) made available to customers to solve a particular bug. This is necessary because for business reasons many ones cannot wait till the next Patch Set or new product release to get a fix. Oracle sometimes recommend interim patches specially security patches to be applied to your systems. Note that , interim patches can only be applied to a particular product version (base release or patch set). That is an interim patch created for 10.2.0.2 should NOT be installed on 10.2.0.3 or 10.2.0.4.

The OPatch (also called the Interim Patch Installer) is the oracle supplied utility which is used to apply interim patches to Oracle's database software. OPatch/Interim Patch Installer supports -
- Applying an interim patch
- Rolling back the application of an interim patch
- Conflict resolution when applying an interim patch after previous interim patches have been applied
- Reporting on installed products and Interim (One-Off) patch

You can download OPatch from metalink as patch 6880880. Patches for Oracle 9i Release 2 Patchset 1 (and beyond) do not include OPatch. OPatch is included from Oracle9i Release 2.

OPatch supports 3 different patch methods on a RAC environment.
1)Patching RAC as a single instance (All-Node Patch)
2)Patching RAC using a minimum down-time strategy (Min. Downtime Patch)
3) Patching RAC using a rolling strategy - No down time (Rolling Patch)

1) Patching RAC as a single instance (All-Node Patch) 
In this mode, OPatch applies the patch to the local node first, then propagates the patch to all other nodes, and finally updates the inventory. All instances will be down during the whole patching process.

The flow diagram of All-node patch is,

. Shutdown all Oracle instances on all nodes
. Apply the patch to all nodes
. Bring all nodes up

2) Patching RAC using a minimum down-time strategy (Min. Downtime Patch)
In this mode, OPatch patches the local node, asks users for a sub-set of nodes, which will be the first nodes to be patched. After the initial subset of nodes are patched, Opatch propagates the patch to the other nodes and finally updates the inventory. The downtime would happen between the shutdown of the the second subset of nodes and the startp of the initial subset of nodes patched.

The flow diagram of Minimum downtime is,

. Shutdown the Oracle instance on node 1
. Apply the patch to the Oracle instance on node 1
. Shutdown the Oracle instance on node 2
. Apply the patch to the Oracle instance on node 2
. Shutdown the Oracle instance on node 3
. At this point, instances on nodes 1 and 2 can be brought up
. Apply the patch to the Oracle instance on node 3
. Startup the Oracle instance on node 3

3) Patching RAC using a rolling strategy - No down time (Rolling Patch)
With this method, there is no downtime. Each node would be patched and brought up while all the other nodes are up and running, resulting in no disruption of the system.

The flow diagram of Rolling patch (no downtime) is,

. Shutdown the Oracle instance on node 1
. Apply the patch to the Oracle instance on node 1
. Start the Oracle instance on node 1
. Shutdown the Oracle instance on node 2
. Apply the patch to the Oracle instance on node 2
. Start the Oracle instance on node 2
. Shutdown the Oracle instance on node 3
. Apply the patch to the Oracle instance on node 3
. Start the Oracle instance on node 3

How does OPatch select which method to use?
All oracle one off patches are not rolling patch. Oracle developers determine whether a patch will be rolling patch or not. OPatch also follows an algorithm to determine which methods it will use in RAC environment. The algorithm is,

If (users specify minimize_downtime)
patching mechanism = Min. Downtime
else if (patch is a rolling patch)
patching mechanism = Rolling
else
patching mechanism = All-Node

How to determine if a patch is a "rolling patch" or not?
Based on the oracle database version we can determine whether a patch is a rolling patch or not.

- For oracle version 9i or 10gR1 issue,
$ opatch query -is_rolling

Opatch will ask the patch location and then will inform if the patch is or not a "rolling patch"

- In oracle 10gR2 issue,
$ opatch query -all <patch_location> | grep rolling

For Windows, the following command can be used as grep is not suitable:

> opatch query -all C:\stage\10.2.0.3_Mini_Patches\5731537 | findstr rolling

gR2 RAC: Patch Set Update 3 Install

September 24th, 2011 | Posted in 11gR2, Blog, patch, RAC | 2 Comments

Pdf version of the post available for download here
Introduction:

In this post, I am going to explain you how to successfully apply a Patch Set Update 3 (PSU 3) on the top of the Patch Set Update 2 (11.2.0.2.2 version).
In my previous article “RAC 11.2.0.2 Patch 12311357 – 11.2.0.2.2 GI Patch Set Update and mutex patch – 4 Nodes on Redhat Linux”, I demonstrated how to apply the patch set update 2 on the top of patch set version 2 (11.2.0.2.0). You may ask this question; what is difference between Patch Set and Patch Set Update?
I will try to answer to that question in the following paragraph.

Definition:

Yesterday, (14 September 2011) Oracle released Patch set number 3 (11.2.0.3) on Linux x86 and Linux x86-64 (refer to My Oracle Support 10404530 for download. So what is Patch Set?
Patch Set is the The fourth digit, component specific release number of 11.2.0.3. Oracle Support adds new features to the Patch Set.
Patch Set Update is the fifth number 11.2.0.2.3; Every 3 months or so, Oracle Support release a Patch Set Update (PSU) to fix Patch Set bugs and does not add any new features to the PSU.

Now, you may ask what about the other 3 digits; the first, the second and the third?

The first digit is the major version, in our case 11g. The second digit is the release number, in our case it is release 2 also called (11g release 2 or shortly 11gR2) with major upgrades and major new features. The third digit is Oracle application specific number, always equal to 0 for database software.

RAC rolling upgrade:

Only the node that is currently being patched has its cluster resources down (database instance, services, db console, …) where the other nodes of the cluster remain available. In this post we will use roll upgrade method to patch the cluster.

Patch Set Update 3:

Patch number 12419353 includes the grid infrastructure and database patch set update number 3.
Since, we are going to patch a RAC (Real Application Cluster) environment, we should apply this patch number. In the case, you need to patch only the database, you can use the patch number: 12419331.
Actually, there are different issues with the opatch auto method to patch the grid infrastructure and database homes. In this post, I am going to use the manual method.

Prerequisites for manual patching:

These are the prerequisites to patch the RAC environment:

1- OPatch utility version 11.2.0.1.5 or higher (Recommended the latest version).
2- Validate the Oracle Inventory
3- Cluster resources must be down for the node that is currently being patched.

Patching Steps:

The following steps must be performed on all nodes of the cluster.

1- Download the latest opatch version: In this post we have downloaded the opatch version 11.2.0.1.6. Please refer to the document ID: 6880880 for opatch download.
2- Donwload the PSU number 3: From my Oracle Support web site download the patch number 12419353.
3- Check the download files:

All the Best:):)