11i/R12 - Daily Checks (for Apps. DBA)


 EBusiness 11i/R12 - Daily Checks (for Apps. DBA) - Admin Node:

As Oracle Apps DBA / Sysadmin you Apply patches & restart servives or make some configuration changes in applications 11i before releasing Instance to Application Users you wish to check if all components in E-Business Suite 11i are working fine. Here is list of components you should check apart from your custom services.
  1. Check Web Server is Up : Check in browser http://hostname.domainname:port where hostname.domain name is Machine Name on which Web Server(OHS) is running. If you are using load balancer or Proxy Server in front of Web Server it will be loadbalancer / Proxy Server Name. Port is port on which Web Server is Listening. It is defined in $IAS_ORACLE_HOME /Apache /Apache /conf /httpd.conf , To find more about WebServer & its parameters
Oracle Web Server also called as OHS (Oracle Http Server) or Oracle-Apache is built on Apache Webserver. Oracle has added few custom component (like mod_plsql, mod_osso...) in Apache webserver & shipped it as Oracle Http Server.
Discussion in this page is currently w.r.t. Oracle Http Server Version 1.0.2.2.2 which is shipped with 11i (11.5.8-10). In coming weeks I'll add 10G Application Server Integration with Oracle Applications 11i.
Oracle Web Server or OHS is entry point in applications, It processes requests coming from network over Client. There is Web Listener listening for Web Requests. If request is for simple html page core Apace returns HTML page but if URL refernced need further processing it will be forwarded to respective module like mod_pls, mod_jserv ... I will start with basics like how to start stop webserver, important directories & files , troubleshooting , tuning & important things to know in oracle applications E-Business suite Webserver.
How to Start/Stop Web Server All startup shutdown scripts in Oracle Applications 11i(11.5.10) are in $OAD_TOP/admin/script/$CONTEXT_NAME where OAD_TOP is also called as COMMON_TOP and CONTEXT_NAME is your SID_hostname , lets say your SID or Instance Name is VISION and its on machine with name machine1 so your CONTEXT_NAME will be VISION_machine1. Script to start/stop Web Server is adapcctl.sh where ad (application dba), apc(apache), ctl(control) and sh stands for shell script. (You will find this script only on node on which you have webserver, if you have multi mode Installation and node 1 is only for Concurrent Manager, Reports & Database and node2 is webserver & forms then adapcctl.sh should be on node2 & not node1) adapcctl.sh callsapachectl(shellscriptin $IAS_ORACLE_HOME/Apache/Apache/bin) which in turn calls HTTPD (in same directory which is executable called http daemon) it starts using configuration filehttpd.conf (from $IAS_ORACLE_HOME/Apache/Apache/conf directory , this file calls other configuration file. To know all important configuration file follow config files link)
 . If you can't access this check error_log at $IAS_ORACLE_HOME /Apache /Apache/ logs

  2. Check Jserv is Up : Check if you can access OAM (Oracle Application Manager) page. http://hostname.domainname:port_number/servlets/weboam/oam/oamLogin If you can't access login screen for OAM, check logs
  3. Check mod_pls is working fine : http://hostname.domainname:port/pls/$SID so if you installed vision instance with SID as VIS11I your SID here is VIS11II this sometime in documents also called as TWO_TASK. You should see output from procedure FND_WEB.PING . If not check that -
A) database listener is Up
B) GUEST User is not end dated & password is correct.
C) dbc (database connect descriptor) file is corerct at $FND_TOP/secure
D) apps password is correct in wdbsvr.app at $IAS_ORACLE_HOME/Apache/modplsql/cfg
  4. Login to Application : Login to application with Application User to confirm login is working fine , if not check mod_pls for any issues.
  5. Check Form Server is working : Click on any Core Responsibility like "System Administrator", If you are facing any problem opening form Applet
A) Check jinitiator is installed, if not do you have access to install jinitiator
B) Try clearing cache from jinitiator
C) Check forms server parameters are configured properly in appsweb_$CONTEXT_NAME.cfg
D) Check if form server listener is Up : netstat -an | grep form_server_port
  6. Check Concurrent Manager is Up & Running : Select "Administer Concurrent Manager" & check if "Actual" & "Target" are same. If not check $SID_MM_DD.mgr at $APPLCSF/$APPLLOG To know more about concurrent Manager 

Concurrent Manager
 as name suggest is Manager which manages your concurrent requests in Oracle Apps 11i. You can assign single request or batch job & let concurrent manager manage these request. Without going to technical definition & explanation about them I will go straight to the point which you might be interested in looking as Sysadmin or Oracle Apps DBA i.e. from startup/shutdown to configuration then finally to tuning. So tie up your belt & get ready for ride to CM.
 How to Start/Stop Concurrent Manager ? In Oracle Applications 11i as you might already be aware that all startup shutdown scripts in 11i(11.5.10) are in $OAD_TOP/admin/script/$CONTEXT_NAME where $OAD_TOP is also called as $COMMON_TOP and $CONTEXT_NAME is your SID_hostname , lets say your SID or Instance Name is VISION and its on machine with name machine1 so your CONTEXT_NAME will be VISION_machine1. So script to start/stop concurrent manager is adcmctl.sh where ad (In my view) is application dba , cm is concurrent manager & ctl means control. Well if you really want to explore more I'll suggest you open this file & go through it . If you open this script you will see syntax to run it in start or stop mode so I am not going to spoon feed you by giving syntax. When you run adcmctl.sh it records execution of this command in text file with name adcmctl.txt at directory $COMMON_TOP/admin/log/$CONTEXT_NAME. adcmctl.sh called another script startmgr which in turn calls batchmgr in $FND_TOP/bin and this is actual file which fire up Internal Concurrent Manager Process (Open these two files as they are text file and read content in these files.
Lets check important parameter in this file
  DISPLAY Set it to some valid XServer , if this is not set properly your reports display will fail (specially one with PDF or Bar code dislay)
  DIAG_PARAM Diagnosis parameter it will take value Y or N with default N, If you encounter some error while starting Concurrent Manager you change this to Y and then extra logging will be enabled then check Concurrent Manager log file.
  CONCSUB stands for CONC(Concurrent) SUB(Submit) , used to submit concurrent request like shut down concurrent manager.
  7. Check Reports can display properly : Change "Active Users" reports output from "Text file" to "PDF" from Define Concurrent Request Screen. Submit Active Users request & click on Output , check if you can se report in PDF format. If you encounter error like REP-3000 Oracle Toolkit , issue is with DISPLAY, To know more about this 
  SLEEP holds the number of seconds that the manager should wait between checks for new requests.
  MGRNAME is the name of the manager for locking and log purposes. So you can find if this is icm(Internal Concurrent Manager), sm(Service Manager)
  RESTART is set to N if the manager should not restart itself after a crash. Otherwise, it is an integer number of minutes. The manager will attempt a restart after an abnormal termination if the past invocation lasted for at least RESTART minutes.
  PMON is the duration of time between process monitor checks (checks for failed workers). The unit of time is concurrent manager iterations (request table checks). Default Value is 20 .
  QUESIZ is the duration of time between worker quantity checks (checks for number of active workers). The unit of time is process monitor checks.
Please pay attention to PMON, QUESIZ and SLEEP as these will be used during Concurrent Manager Tuning.
Where is Concurrent Manager Node ? If you don't know where is your CM here are few ways to find your it .
  SQLPLUS> select node_name from apps.fnd_nodes where support_cp='Y';
  In context file (SID_hostname.xml ) under $APPL_TOP/admin , check for variable < TIER_NODE oa_var="s_isConc" >YES< / TIER_NODE>, YES means this is Concurrent Manager Node.
  Check for Administer Concurrent Manager request under SYSADMIN responsibility.
  "ps -ef | grep FNDLIBR" under application User on all nodes where FND(Foundation) LIBR(Library) is executable.
  8. Check Apps Listener is Up : Apps listener is different from database listener, Apps Listener is used for various purpose one is to check output & log files in concurrent requests. Check if apps listener is fine by clicking on log file & output file for Active Users request you submitted in Step 7 on previous page. If you encounter any issues while trying to access Log & Output file issue might be with Oracle Apps Listener. Check if adalnctl.sh (where ad is apps dba, aln is apps listener and ctl is control) on application tier started successfully. If yes then check this query (select logfile_name, logfile_node_name, outfile_name, outfile_node_name from fnd_concurrent_requests where request_id=[your request id number].
  9. Check if Workflow is Up : Login to OAM (Oracle Application Manager) using url mentioned in step 2 in previous page. After login, On top right of page from "Nevigate to" select "Workflow Manager" from drop down menu and click on GO button. Check "Notification Mailers" is Up .
  10. Check if Discoverer 4i is Up : There are two type of discoverer 4i
Discoverer Viewer - Check http://yourhostname:port/discoverer4i/viewer
Discoverer Plus Check http://yourhostname:port/discwb4/html/english/welcome.htm
11. Custom applications : Any custom application configured with oracle apps 11i, oem any other monitoring software.


Being Apps. DBA, one should check the subsequent on Daily Basis in order to make sure that everything is working fine. This post is relating to ADMIN NODE.

Listener Status
$ lsnrctl status PROD

File System Space Utilization
$ df -h

Operating System - eBusiness DB and Concurrent Process/es
$ ps -ef |grep pmon|grep -v grep
$ ps -ef |grep FNDLIBR|grep -v grep

eBusiness DB Instance
SQL> SELECT instance_name,host_name,TO_CHAR(startup_time,'DD-MON-YYY HH:MM:SS AM') startedat FROM v$instance;

Invalid Objects
SQL> select owner,object_name,object_type from dba_objects where status<>'VALID';

Inactive Users
SQL> select username,command,status,server,osuser,machine,terminal,program,module,action from v$session where status<>'INACTIVE';

Active Sessions
SQL> select inst_id,count(*) from gv$session group by inst_id;

Active Users Detail
SQL> select username,command,status,osuser,process,machine,terminal,program,module,action from v$session where status<>'ACTIVE';

Data Files Status
SQL> select name,status from v$datafile;

Log Files Status
SQL> select * from v$log;

Archiving Errors
SQL> select error from v$archive_dest;

Tablespace Free Space
SQL> select tablespace_name,round (sum(bytes)/1024/1024) Free_Space_MB from dba_free_space group by (TABLESPACE_NAME);

Invalid Indexes
SQL> select index_name,table_name from dba_indexes where status ='INVALID' and owner='APPS';

eBusiness 11i/R12 - Applications Listener Status
$ sh $COMMON_TOP/admin/scripts/PROD_oracle2/adalnctl.sh status

Checking on the status of Report Server
$ sh $ORACLE_HOME/admin/scripts/PROD_oracle2/adrepctl.sh status

Database Connectivity via Apps. user without Connect String
$ sqlplus -S -M "HTML ON TABLE 'BORDER="2"'" "apps/****" @/oracle/appsprod/AOT/appschk.sql > /oracle/appsprod/AOT/output.html

One may send comments to rawishsiddiqui@yahoo.ca.

1 comment:

SYED JAVEED said...

PLEASE SEND ME THE IMPORTANT QUESTIONS ON CLONING APPS 11i,R12