Building a 5 nodes Oracle EBS using RAC and shared appl top 8.

Finish the configuration

What we have got now?

• Grid infrastructure
• RAC database software
• 2 node RAC EBS database
• A to be configured application tier

The already made steps:

1. Prepare the RAC database software
2. Prepare the source database for the conversion
3. Convert database into RAC
4. Post database conversion steps
1. turn on autoconfig
2. server parameter changes
3. create new EBS specific cluster listeners

So I have to configure the application tier to use the new RAC database. The remaining steps are:

1. Configure application tiers for using RAC database service
2. Post configuration steps

Configure application tier

So I have reached the 3.8 main point in the RAC configuration note. The first subpoint is the preparation  of the application tier. Let's start the configuration.

Prepare all nodes for autoconfig

On all 3 application tier modify the tnsnames.ora file. Do the following steps on the application servers in this order (1. conc01, 2. app01, 3. app02)

1. Login with applmgr to the node and modify the tnsnames.ora file
$ su – applmgr
# . bin/PROD.env
# cd $TNS_ADMIN
# cp tnsnames.ora tnsnames.ora.orig

# vi tnsnames.ora 

Modify and add the following lines - on the conc01 I have to modify the PROD entry

PROD=

        (DESCRIPTION=

        (ADDRESS=(PROTOCOL=tcp)(HOST=db01vip.company.local)(PORT=1551))

            (CONNECT_DATA=

                (SERVICE_NAME=PROD)

                (INSTANCE_NAME=PROD1)

            )

        )

Add tns entries for PROD1 and PROD2 instance too!

2. Modify the jdbc_url entry in the current context file. In the connection descriptor add the "vip" characters to the hostname, after the service name add the instance name too.

For example.

# vi $CONTEXT_FILE

After modification the jdbc_url xml tag should looks like this.

<jdbc_url oa_var="s_apps_jdbc_connect_descriptor">jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=db01vip.company.local)(PORT=1541)))(CONNECT_DATA=(SERVICE_NAME=PROD)(INSTANCE_NAME=PROD1)))</jdbc_url>

3 Run autoconfig

4. After succesful run of autoconfig run, check that the tnsnames.ora files are the same for 10.1.2 and for 10.1.3 OAS homes.

5. Check the APPS_JDBC_URL entry in actual tier's own dbc file too.

6. Go to the next application tier node.

Set database load balance for the database connection.

Now the system is working, but it don't use load balance feature when accessing the RAC database. I have to configure it.

Edit context file on each node an modify the s_tools_twotask, s_weboh_twotask  and the s_apps_jdbc_connect_alias entries to value: PROD_BALANCE.

After modifications run autoconfig on each node 

# cd $AD_TOP/bin

# ./adconfig.sh contextfile=$CONTEXT_FILE

Modification settings of the concurrent processing 

1. I have to set on the PCP. For this turn on APPLDCP entry in the context file on all nodes.

Run autoconfig on each node after modications.

2. I have to modify the APPLPTMP and APPLCSF enviroment's values. Because we have got 3 application tier and a 2 node RAC database, their have to point to shared directories. These directories has to shared for all nodes.

As I mentioned at the begin in this post series we have got a shared /u02 mount points on all nodes which is good for this purpose. So create directories for APPLPTMP and an other one for APPLPTMP under /u02. (You should only do it once :) )

3. Now edit context file on all application tier and modify the APPLCSF and APPLPTMP entries as this example:

# vi $CONTEXT_FILE

<APPLCSF oa_var="s_applcsf">--put full path of shared APPLCSF directory name here--</APPLCSF>

<APPLPTMP oa_var="s_applptmp" osd="UNIX">--put full path of shared APPLPMTP directory name here--</APPLPTMP>

4. Modify ult_file_dir in database, it should contain the APPLTMP directory and it should be the first! 

Login as oracle into db node, source the enviroment and:

# sqlplus / as sysdba

SQL> alter system set utl_file_dir='--put full path of shared APPLPMTP directory name here--',  '/usr/tmp' scope=spfile;

SQL> exit;

5. Modify the dbutilfiledir parameter in the context file on each database node. It should be the same as the above value what you have set for utl_file_dir parameter.

6. Run autoconfig on database nodes.

6. For correct working of Transaction Manager I have to set extra database parameters. Run these alter SQL command as sysdba in the RAC database:

SQL> alter system set "_lm_global_posts"=TRUE scope=SPFILE;

SQL> alter system set "_immediate_commit_propagation"=TRUE scope=SPFILE;

8. Restart RAC database on both node with srvctl command as grid user.

9. Now run autoconfig on all remaining application tier nodes in the "usual" order.

Test the reconfigured EBS

Now I have too start the whole application tier for test it's working and made the remaining changes.

Start the application tier on the nodes with this order (1. conc01, 2. app01, 3. app02)

Open a browser and test the system with one of web tier URL. For example: http://app01.apeh.local:4463

It will redirect to the second node, because I don't configure the load balancer yet and the autoconfig was runned on the second node at last. (theres is a common FND profile option, which's value is modified when you run the autoconfig, and every nodes put himself into this profile option)

Post configuration steps

Set up web Load Balancing.

Set up load balancing based on 380489.1 note, your original configuration and your goal.

Prerequisites for succesful configuration

1. certified load balancer
2. have a comman hostname for entry url, standard EBS users will reach it from outside of load loadbalancer
3. configure load balancer for EBS 
1. - for example set correct value for "sticky bit"
2. configure load balancer's "outside" port for listening for common hostname's IP address.
3. load balancing method (how to choose application servers at the login)
4. common entry url port number, it could be different
5. etc.

In this example I will use for common hostname: ebs.company.local, the new common url port number will the same as the EBS system port number.

Some configuration opportunity:
1. the simpliest way:  the new system will use http and the load balancer entry url is is http too.

In this case load balancer configuration is simple too, because you don't need to set up any special SSL configuration on load balancing and on EBS system.

Set these values for context variables:

Context Variable Name	Old Context Value	New Context Value
s_webentryurlprotocol	http	http
s_webentryhost	app01 app02 conc01	ebs
s_webentrydomain	company.local	company.local
s_active_webport	4473	4473
s_login_page	http://app01.company.local:4473/OA_HTML/AppsLogin http://app02.company.local:4473/OA_HTML/AppsLogin http://conc01.company.local:4473/OA_HTML/AppsLogin	http://ebs.company.local:4473/OA_HTML/AppsLogin
s_external_url	http://app01.company.local:4473 http://app02.company.local:4473 http://conc01.company.local:4473	http://ebs.company.local:4473

2. the next one, is when the new system will use https and the load balancer entry point will use https.
In this case use the above table, but at the "new context value" column change http to https at every row.
The load balancer should forward the whole packets to the app servers, it should configured for correct nating only - just like as at previous, it should not configured for SSL termination (or SSL offloading).

Of course you should configure the EBS system to use correct SSL certificates, never use the Oracle's example certificate for production system!

3. the EBS system will configured for http and the load balancer entry point will use only https.
In this case at you should configure the EBS system for http, the load balancer should configured for SSL termination and the terminated packets will forwarded to web tier http servers.

In my experience the customer's don't like this option. Usually I hear two objections:
- for SSL termination they need extra licences for the load balancer.
- the SSL termination need extra resources in the load balancer hardware.
I comprehend these objections, the SSL termination at application tier is much cheaper than at load balancer's hardware. Usually the application tier hardware's resources are more than enough for this purpose.

Optionally refresh certifications

If you choosed to configure the EBS system's for https you should setup it. Use 376700.1 for SSL configuration.

Configure database parameters

Configure database initialization parameters as your system wants to. At least set higher values for SGA, PGA, don't forget to set the parallel_force_local parameter to TRUE.

Apply latest rapid clone patches

It is hardly recommended to apply the latest rapid clone pathces. It could be very useful when you will create test and development system with cloning. Creating clone system from a RAC based Oracle EBS is not so simple, it will be an other post later :)

Configure archive log and backup

Never forget to configure backup for RAC system. Cause of using ASM you should use RMAN for backing up the database, online or "offline", scheduling, incremental level backup is depends on you, and your system's load.

Check some feature of the new system.

After you create the first system - which should be only a test system, never use the first implemented system as a production system - check some standard and none standard feature of your system.

For example:

- the login :)

- the load balancing - monintor the system, that the load balancer encumber the application tier as you choose.

- the concurrent program running - run some programs just for test the concurent manager's settings.

- the APPLPTMP configuration, for example this SQL command should create a test file under the common APPLPTMP directory: exec FND_FILE.PUT_LINE(FND_FILE.LOG, 'THIS IS A TEST');

- do functional testing, never forget it!

- etc.

Ready! 

No I am ready, the system should running well at this point. I hoped you liked this post series. If you have any comment, don't hasitate comment any post in this series, ask if you have a question.

Good luck!