Cluster db alerts :

Cluster db alerts in Real Time:

Data Guard
Data Guard Fast-Start Failover
Data Guard Performance 
Data Guard Status
Database Cardinality 
Database Job Status
Database Wait Bottlenecks
Deferred Transactions 
Failed Logins
Flash Recovery
Invalid Objects
Invalid Objects by Schema
Recovery
Recovery Area
Response
Segment Advisor Recommendations
Session Suspended
Snapshot Too Old
Streams Processes Count 
Suspended Session
Tablespace Allocation
Tablespaces Full 
Tablespaces Full (dictionary managed) 
Tablespaces With Problem Segments 
User Block 

Automatic Storage Managemnt Alerts:

Automatic Storage Managemnt Alerts:
============================
 
Alert Log
Alert Log Content 
Alert Log Error Status 
Cluster Disk Group Performance 
Cluster Disk Performance
Database Disk Group Usage
Disk Group Usage
Disk Path 
Disk Status
Instance Disk Group Performance 
Instance Disk Performance 
Offline Disk Count
Response
Single Instance Disk Group Performance

Database Upgradation from 10.2.0.4 to 11.2.0.3 Silent Mode using DBUA

Database Upgradation from 10.2.0.4 to 11.2.0.3 Silent Mode using DBUA:
===============================================

pre upgrade :
===============
connect to the database.
SQL> @/local/oracle/product/rdbms/11g/rdbms/admin/utlu112i.sql
minimum required SYSTEM size: 1652 M
minimum required SYSAUX size: 1047 M
minimum required UNDOTBS size: 400 M
minimum required TEMP size: 60 M
minimum required SGA size : 1076M
PURGE DBA_RECYCLEBIN; // form db level
EXECUTE dbms_stats.gather_dictionary_stats;
select count(*) from dba_objects where status='INVALID';
@?/rdbms/admin/utlrp.sql

"How to Pre-Process SYS.AUD$ Records Pre-Upgrade From 10.1 or Later to 11.2". My Oracle Support Note 1329590.1
Standard Auditing
               SELECT count(*) FROM sys.aud$ WHERE dbid is null;
Standard Auditing, only when Oracle Label Security (OLS) and/or Database Vault (DV) is installed
               SELECT count(*) FROM system.aud$ WHERE dbid is null;
Fine Grained Auditing
               SELECT count(*) FROM sys.fga_log$ WHERE dbid is null;

copy the password file to all the nodes.
verify that the db is registered in oratab or update it manually
<SID_NAME>:<old oracle home>:N
nrtd:/local/oracle/product/rdbms/10g:N

known issue :
===============
ora-00205 error in identifying control,check alert log for more info
/local/oracle/product/crs/bin >ls -ltr oracle
/local/oracle/product/crs/bin >chmod 6751 oracle
/local/oracle/product/crs/bin >ls -ltr oracle
-rwsr-s--x 1 oracle oinstall 203974257 Feb  1 12:01 oracle

no end user should not connect the database.
ps -ef | grep nrtd |grep -i oracle | grep LOCAL=NO

take the cold backup
stop the database using srvctl
start the database using srvctl

-------------------------------------------------------------------------------------------

Actual upgrade :
==================
export the new home environmental file
export ORACLE_BASE=/local/oracle/prodcut
export ORACLE_SID=nrtdtest1
export ORACLE_HOME=/local/oracle/product/rdbms/11g
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME:$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$PATH

check which dbua it should refer 11g home
/local/oracle/product/rdbms/11g/bin/dbua


dbua -silent dbaname nrtd -oracleHome /local/oracle/product/rdbms/10g

logfile location :
--------------------
/local/oracle/product/cfgtoollogs/dbua/logs

known issue :
===============
ora-00205 error in identifying control,check alert log for more info
/local/oracle/product/crs/bin >ls -ltr oracle
/local/oracle/product/crs/bin >chmod 6751 oracle
/local/oracle/product/crs/bin >ls -ltr oracle
-rwsr-s--x 1 oracle oinstall 203974257 Feb  1 12:01 oracle

post upgrade steps :
========================
ps -ef |grep pmon
change home location in oratab.dat
connect to the database
select COMP_NAME,STATUS,VERSION,MODIFIED from dba_registry
select * from v$version;
select instance_name,host_name,version,status,startup_time from gv$instance;
select count(*) from dba_objects where status='INVALID';
@?/rdbms/admin/utlu112s.sql // status 





Log Files in RAC Environment:

Log Files in RAC Environment:
=========================
Log Files in RAC Environment:
The Cluster Ready Services Daemon (crsd) Log Files
Log files for the CRSD process (crsd) can be found in the following directories:
ORA_CRS_HOME/log/hostname/crsd
the crsd.log file is archived every 10MB as crsd.101, crsd.102 ...
ORACLE_CRS_HOME/log//alert.log
Oracle Cluster Registry (OCR) Log Files
The Oracle Cluster Registry (OCR) records log information in the following location:
ORA_CRS_HOME/log/hostname/client
Cluster Synchronization Services (CSS) Log Files
You can find CSS information that the OCSSD generates in log files in the following locations:
ORA_CRS_HOME/log/hostname/cssd
OCSSD responsible for inter node health monitoring and instance endpoint recovery.
It runs as oracle user.  
The cssd.log file is archived every 20MB as cssd.101, cssd.102....

Oracle ASM Cluster File Systems (ACFS) in Oracle Database 11g R2:

Oracle ASM Cluster File Systems (ACFS) in Oracle Database 11g R2:
=================================================
Oracle ASM Cluster File System (ACFS) is a general purpose cluster file system implemented as part of ASM. It can be used to store almost anything, including the database executables. The only things that should not be stored in ACFS are the Grid Infrastructure home and any Oracle files that can be directly stored in Oracle ASM.
This article describes three ways to create an ASM Cluster File System in an Oracle 11G Release 2 RAC database. It is assumed the RAC database is already present. For these examples I used the VMware RAC installation described here.
ASM Configuration Assistant (ASMCA)
As the "oracle" user, switch to the ASM environment on node 1 of the RAC, then we start the ASM Configuration Assistant (asmca).
[oracle@rac1 ~]$ . oraenv
ORACLE_SID = [RAC1] ? +ASM1
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle
[oracle@rac1 ~]$ dbhome
/u01/app/11.2.0/grid
[oracle@rac1 ~]$ asmca


When the ASM configuration assistant starts you are presented with the "ASM Instances" tab.
Click on the "ASM Cluster File Systems" tab, then click the "Create" button.
Select "Create Volume" from the Volume list.
Enter the Volume Name and Size and click the "OK" button. Wait for the volume to be created, then click the "OK" button on the subsequent message dialog.
The newly created volume will now be selected in the Volume list. Select the "General Purpose File System" option, enter a previously created mount point directory (or leave the suggested mount point), select the "Yes" option for Register MountPoint and click the "OK" button.
Click the "OK" button on the resulting message.
The newly created cluster file system is now listed under the "ASM Cluster File Systems" tab.
Either perform another action, or click the "Exit" button.
At the command line on the first RAC node, navigate to the ACFS mount point and create a test file.
[oracle@rac1 data_acfsvol1]$ cd /u01/app/oracle/acfsmounts/data_acfsvol1
[oracle@rac1 data_acfsvol1]$ echo "This is a test" > test.txt
[oracle@rac1 data_acfsvol1]$ ls -al
total 80
drwxrwx--- 4 root   dba       4096 Dec 29 12:39 .
drwxr-xr-x 3 root   root      4096 Dec 29 12:24 ..
drwxr-xr-x 5 root   root      4096 Dec 29 12:24 .ACFS
drwx------ 2 root   root     65536 Dec 29 12:24 lost+found
-rw-r--r-- 1 oracle oinstall    15 Dec 29 12:39 test.txt
[oracle@rac1 data_acfsvol1]$
Check the file is present on the second RAC node.

What is oraInventory in Oracle 10g/11g:

What is oraInventory in Oracle 10g/11g:
=============================

You all might be familiar with oraInventory & its importance so lets take a look at few common things about it like binary/xml inventory, What to do if Inventory is corrupted , What is global / Local Inventory and where to find documentation related to oraInventory .

What is oraInventory ?
oraInventory is repository (directory) which store/records
oracle software products & their oracle_homes location on a machine. This Inventory now a days in XML format and called as XML Inventory where as in past it used to be in binary format & called as binary Inventory.
There are basically two kind of Inventory Global Inventory (also called as Central Inventory) and Local Inventory also called as Oracle Home Inventory.

Global Inventory ?
Global Inventory holds information
about Oracle Products on a Machine. These products can be various oracle components like database, oracle application server, collaboration suite, soa suite, forms & reports or discoverer server . This global Inventory location will be determined by file oraInst.loc in /etc (on Linux) or /var/opt/oracle (solaris). If you want to see list of oracle products on machine check for file inventory.xml under ContentsXML in oraInventory (Please note if you have multiple global Inventory on machine check all oraInventory directories
)

You will see entry like
HOME NAME="ORA10g_HOME" LOC="/u01/oracle/10.2.0/db" TYPE="O" IDX="1"/
...
...

Local Inventory
Inventory inside each Oracle Home is called as local Inventory or oracle_home Inventory. This Inventory holds information to that oracle_home only.

Can I have multiple Global Inventory on a machine ?
Quite common questions is that can you have multiple global Inventory and answer is YES you can have multiple global Inventory but if your upgrading or applying patch then change Inventory Pointer oraInst.loc to respective location. If you are following single global Inventory and if you wish to uninstall any software then remove it from Global Inventory as well.

What to do if my Global Inventory is corrupted ?
No need to worry if your global Inventory is corrupted, you can recreate global Inventory on machine using Universal Installer and attach already Installed oracle home by option
-attachHome

./runInstaller -silent -attachHome -invPtrLoc $location_to_oraInst.loc
ORACLE_HOME="Oracle_Home_Location" ORACLE_HOME_NAME="Oracle_Home_Name"
CLUSTER_NODES="{}"

Do I need to worry about oraInventory during oracle Apps 11i cloning ?
No, Rapid Clone will update both Global & Local Inventory with required information , you don't have to worry about Inventory during Oracle Apps 11i cloning.

What is oraInventory
The Orainventory is the location for the OUI's Book keeping
The inventory stores information about.
  • All the Oracle software products installed on all ORACLE_HOMES on a machine
  • Other non-oracle products such as Java Runtime env's (JRE)
Binary OraInventory
Before OUI 2.X (or 11.5.7 or earlier)the inventory was binary, the binary orainvenory maintains in inventory in binary format
XML Inventory
Starting from OUI 2.X and 11.5.8 information in the inventory is stored in the Extensible Markup Language (XML) format.
The XML format allows easier diagnostic of the problem and faster loading of data.
XML inventory is divided into 2 components.
  1. Global Inventory
Global Inventory holds information about Oracle Products on a Machine, The inventory contains the high level list of all oracle products installed on a machine such as ORACLE_HOMES or JRE.
It doesn't have any information about the details of patches applied on each ORACLE_HOMES.
There should be only one per machine. Its locations is defined in the oraInst.loc in /etc (on Linux) or /var/opt/oracle (solaris).
  1. Local Inventory
There is one Local inventory per ORACLE_HOME.
Inventory inside each Oracle Home is called as local Inventory or ORACLE_HOME Inventory. This Inventory holds information to that ORACLE_HOME only.


Can I have multiple Global Inventories on a machine?
Can you have multiple global Inventory and answer is YES you can have multiple global Inventory but if your upgrading or applying patch then change Inventory Pointer oraInst.loc to respective location.
If you are following single global Inventory and if you wish to uninstall any software then remove it from Global Inventory as well.

What to do if my Global Inventory is corrupted?
If your global Inventory is corrupted, you can recreate global Inventory on machine using Universal Installer and attach already Installed oracle home by option
-attachHome
./runInstaller -silent -attachHome -invPtrLoc $location_to_oraInst.loc ORACLE_HOME=Oracle_Home_Location ORACLE_HOME_NAME=Oracle_Home_Name CLUSTER_NODES={}

Do I need to worry about oraInventory during oracle Apps 11i cloning ?
No, Rapid Clone will update both Global & Local Inventory with required information, you don't have to worry about Inventory during Oracle Apps 11i cloning.

How to Move oraInventory from one location to other?
Find the current location of the central inventory (Normally $ORACLE_BASE/oraInventory):
Open the oraInst.loc file in /etc and check the value of inventory_loc
cat /etc/oraInst.loc
inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall
Remark: The oraInst.loc file is simply a pointer to the location of the central inventory (oraInventory)
Copy the oraInventory directory to the destination directory
cp -Rp /u01/app/oracle/oraInventory /u02/app/oracle/oraInventory
Edit the oraInst.loc file to point to the new location
vi /etc/oraInst.loc
inventory_loc=/u02/app/oracle/oraInventory
inst_group=dba

OraInventory and its use -:
In the oraInventory directory, the Installer keeps track of what is installed, and information on how to deinstall products. Many of the libraries needed to run the Installer are also in this directory. The Oracle Installer uses the "oraInst.loc" file to determine the location of the oraInventory directory. If this file is blank or contains an invalid entry the Installer will give an error about unable to find oraInventory or may provide an incomplete listing of installed products.


The oraInventory directory must be owned by the user root.

Oracle Universal Installer (OUI) uses oraInventory directory as a repository of the Oracle products that are installed on your machine.

OraInventory Directory

The installer creates the oraInventory directory the first time it is run on your computer. The oraInventory directory keeps an inventory of products that the installer installs on your computer as well as other installation information. If you have previously installed Oracle products, then you may already have an oraInventory directory.
  • When a UNIX group name is specified, it grants that group the permission to write to the oraInventory directory. If another group attempts to run the installer, then they must have permission to write to the oraInventory directory. If they do not have permission the installation will fail.
  • Be sure the user running the installer has permission to write to the oraInventory directory and all its files. This is required to run the installer.
  • The location of oraInventory is defined in /etc/oratab/oraInst.loc for HP 9000 Series HP-UX and AIX-Based Systems.
  • The location of oraInventory is defined in /var/opt/oraInst.loc for Sun Solaris.
  • The latest log fileDescription: G:\OCTOBER FROM INTERNET DELHI\OCT 3RD\orainventory\Oracle Dba  OraInventory and its use - _files\arrow-10x10.png is oraInventory_location/logs/installActions.log. Log fileDescription: G:\OCTOBER FROM INTERNET DELHI\OCT 3RD\orainventory\Oracle Dba  OraInventory and its use - _files\arrow-10x10.png names of previous installation sessions take the form installActionsdatetime.log.
  • Do not delete or manually alter the oraInventory directory or its contents. Doing so can prevent the installer from locating products that you have installed on your system.




Data Guard Physical Standby Database Best Practices:

Data Guard Physical Standby Database Best Practices
Introduction
Oracle Data Guard is one of the software solutions provided by Oracle Corporation to maximize high availability of Oracle databases. Oracle Data Guard maintains one or many secondary databases as alternatives to the primary production database.
Data Guard Architecture
Oracle Data Guard supports both physical standby and logical standby sites.
Physical Standby: When the primary database transactions generate redo entries, a redo apply process keeps up the secondary databases with the exact block copies of the primary database.
Logical Standby: SQL apply processes read the redo and convert it to SQL transactions. These are then applied to the secondary database.
Data Guard Modes
Oracle Data Guard can operate in 3 different modes:
o    Maximum Protection: Transactions are not allowed to commit until all redo data are written to the online redo logs and propagated to at least one synchronized secondary database. If for any reason, the primary database cannot propagate its redo stream to one secondary database, the primary will shutdown to ensure maximum protection mode.
o    Maximum Performance: Transactions are not allowed to commit as soon as the redo are written to the online redo logs. The redo stream is asynchronously propagated to the secondary databases to ensure maximum performance mode.
o    Maximum Availability: Transactions are not allowed to commit until all redo data are written to the online redo logs and propagated to at least one synchronized secondary database. If for any reason, the primary database cannot propagate its redo stream to one secondary database, the primary will NOT shutdown and operates as it it were in maximum performance mode until issues are fixed.