Cloning a 12.2 EBS with rapid clone

Cloning 12.2 EBS 

The 12.2 EBS cloning has been dramatically changed in the 12.2 version. The main concept still the same, (preclone, copy files, post clone), but the underlying logic was changed.

First you will not find Rapid Clone note for 12.2 on Oracle's Support site (The following document mention the 1383621.1 note, but is just a "rapid clone bug list" now, not a complete step by step note as the earlier notes were). You should open the "Oracle® E-Business Suite Setup Guide Release 12.2" document and read chapter 17 "Cloning Oracle E-Business Suite with Rapid Clone".

Prerequisites

I will not discuss them please read the above document for prereq steps. 

Cloning tasks

The 3 main steps is still the same as in earlier releases

1. Prepare the Source System for Database Tier and Application Tier
2. Copy both Database Tier and Application Tier nodes from the Source System to Target System
3. Configure the Target System for both Database Tier and Application Tier

BUT!

High Level Standard Cloning Process for Application Tier In addition to cloning the database tier node, standard cloning process for Application Tier basically involves the following two key processes:

1. Copy Application Tier node from the Source "Run Edition File System" to the Target "Run Edition File System".

2. Copy the Application Tier node from the Target "Run Edition File System" to the Target "Patch Edition File System".

 What does it means? The document contain a good diagram example:

The document will later explain the above, but what effect will this? When you are ready with the preclone steps you should copy one of the fs directory (the actual run edition) to the target then copy it for the other fs directory. So a stanard cloning will takes much longer, just because the new dual file system. :(

Prepare the Source System for Database Tier

It is still the same as earlier, except the ORACLE_HOME and the data directory common place has been changed. You will find them directly under main base directory (11.2.0 directory for softwares and data directory for database files)

So just source the database enviroment with the database user (for example oracle) and

$ cd $ORACLE_HOME/appsutil/scripts/<CONTEXT_NAME>

$ perl adpreclone.pl dbTier

Prepare the Source System for Application Tier

The changes begin here. 

First you should login with application tier's user (for example applmgr) and source the enviroment. BUT you should be aware which one! You should run the preclone script "Run Edition File System". 
If you don't know which one is currently the run edition, then source the firts one check is with this command:

$ echo $FILE_EDITION

If it doesn't return with "run" value, then you should logout , login and source the other file system enviroment file.

This is crazy, isn't it??? I think a little bit better shell programmer could write a better solution for it...

Update: you could use the EBSapps.env enviroment file for determining the run edition file system. If you run
$ . <target base directory>/EBSApps.env run
command. Then it will set your enviroment to the actual run edition file system and prompt all important main directory.

Second after you source the correct enviroment you should run the preclone script as same as earlies like this example:

$ cd <INST_TOP>/admin/scripts

$ perl adpreclone.pl appsTier

BUT! you couldn't run the script on a fully shutdowned system. The script ask for Weblogic administration password and check for a running Weblogic Administration Server!!

So if you shutdowned the full system, now you should start the admin server before running the preclone script with these commands

$ cd <INST_TOP>/admin/scripts

$ adadminsrvctl.sh start

Why was it to hard to create a better script. For example if the script check for running state of admin server and it isn't found  a running one then the script start it just until the preclone script is running?

Now you could run the preclone script. Funny, because the script will reach 80 percent at the completion state line then falling back to 20%.

• It will stop until the "Creating FMW archive" step doesn't finish, the preclone script copying FMW files into the COMMON_TOP/clone/prereq and the underlying FMW directory.
• It will start the node manager and the forms,  oacore, forms-c4ws and oafm services - in this order - if they are in shutdown mode during "Staging Weblogic Server" phase
• After finishing the above phase the "Creating WLS config archive" is coming. Strange but it has emptied the whole log file.

The preclone script will use much more resources than the earlier releases. At least 1 CPU core, many memory and many many I/O operations will be used.

Now we have got a preprared 12.2 system, let's go and copy the files

Copying source files to target

Before begin to copy the source target, check whether what components is running.

After successfully running application tier's preclone script there will be running at least the admin server and the node manager. Stop them - and any other running apps tier process - then shutdown the database - if it is possible with the standard normal mode, if it was not possible, then start and shutdown with normal mode before copying the source database.

Instead of copying file to other server I mention to use the tar command. So first create a tar archive files for database, then application tier just like this example command.

$ tar czvf dbtier.tar.gz <source base directory>/11.2.0 <source base directory>/data

then create archive file for apps tier (if fs1 is the run edition file system)

$ tar czvf appstier.tar.gz <source base directory>/fs1/EBSApps

The EBSApps directory will contain all necessary directories what propose the cloning document: <APPL_TOP>, <COMMON_TOP>, <OracleAS Tools 10.1.2 ORACLE_HOME> 

When the archived files are ready copy them to the target system and unpack them.
If you packed files as above, during the untar will create the whole <source base directory> under your directory where you have run the tar process.

After succesfully "untaring" copy the database files and the fs directory into the new base directory. You should have something like this one - if fs1 was the run edition file system.

<target base directory>/11.2.0
<target base directory>/data

<target base directory>/fs1/EBSApps

Check the unpacked directories for these

• ownership, all file has to be got the right ownership, if you want to use the original unix users, then you should have on the target node the exactly same user as on the source system. Not only their names, their user and group id should have the same.
• check kernel parameters, system config, rpm packages whether they are EBS compliant or not. If not configure the target node before continue.
• if the target node already has an oraInventory, then check if the unix user of the EBS has got enough priviliges for that oraInventory directory. If not have oraInventory then prepare a new one before running the postclone scripts.

Configure the Target System

Run the post clone scripts for database

Log on to the Target System as the ORACLE user and enter the following commands:
$ cd <RDBMS ORACLE_HOME>/appsutil/clone/bin
$ perl adcfgclone.pl dbTier 

When is ask for Targe System directory give the value of <target base directory>.

Now run preclone for the first application tier file system as the document mention.

Log on to the "Run Edition File System" in the Target System as the APPLMGR user and enter the following commands:
$ cd <COMMON_TOP>/clone/bin$ perl adcfgclone.pl appsTierAt the prompt, "Target System Base Directory", enter the location of the <target base directory>.

Until this step it is same as in earlier version (except new directory structure).

From this point come the exactly new 12.2 extra cloning steps.

Run adpreclone.pl on the "Run Edition File System" in the Target System.

Source the enviroment file int the new file system and run the following command.

$ cd <COMMON_TOP>/clone/bin
$ perl adcfgclone.pl appsTier

Where <COMMON_TOP> is for example <target base directory>/fs1/EBSapps/comn.

New questions, changed features:

• "Weblogic Adminserver password"
• "Do you want to add a node" - default is no
• "Target System File Edition type" - default is run
• after setting"Target System Base Directory" it will default automatically 
• run file system directory
• patch file system directory
• fusion middleware home
• web oracle home
• Appl TOP
• COMMON TOP
• "Target System Instance Directory" - you should not give the full INST_TOP directory value. Instead of accept the defaulted value - the main base directory and the script will set to the correct value for the new INST_TOP.
• A new service could enable/disable after setting display parameter: "Web Administration"
• If you not set well oraInventory the post clone script will go to error at the FMW pre-requisite check step. The error in the log file will be: "FileNotFoundException: /etc/oraInst.loc"
• The precheck script will also check for rpm packages, kernel version, glibc version, memory, LD_ASSUME_KERNEL necessity.  
• You should have more than 1.5 GB free space in temp directory. If you don't have the preclone script will goes to error. (somewhy the prereq check script doesn't check for this :( ) In this case before you retry, don't forget to delete the newly created FMW_Home diretory under the fs1 or fs2 directory.
• If any error occured during FMW process, check log file under fs1/EBSApps/comn/clone/FMW/logs directory.

At the end, the post clone script ask for starting the application tier, if say yes you have wait for many minutes, the new version start much slower then the earlier was.

Now we have to have these directories:

<target base directory>/11.2.0
<target base directory>/data

<target base directory>/fs1/EBSApps

<target base directory>/inst

Before continue don't forget to check the unix links, for example the libgcc_s files under EBSapps/10.1.2/lib directory. (check 12.2 release notes of your platform for more information)

Run the preclone on the completed file system - with applmgr user and sourced the envoriment file.

$ cd <INST_TOP>/admin/scripts

$ perl adpreclone.pl appsTier

Copy Oracle E-Business Suite application directories from the "Run Edition File System" to the "Patch Edition File System" in the Target System.

In the document there is a special note for the copy step:

Note: Make sure you pay attention to symbolic links under <OracleAS Tools 10.1.2 ORACLE_HOME> and <COMMON_TOP> directories when copying files.

Without any detailed of the symbolic links. So everybody has to create yourself an own list? Why was it to hard to create a "duplicate_fs.sh", which copy the files and repair all symbolic links? Crazy...

By the way the document say only "Copy Oracle E-Business Suite application directories". I thinked this means the EBSapps directory only. Was it to hard to wrote down it's name?

I used the following example command for copying fs1 to fs2 as root.
$ cd <target base directory>
$ copy -RH fs1 fs2
$ cd fs2
$ rm -rf FMW_HOME inst
$ cd <target base directory>
$ chown <applmgr>:<applmgr group> fs2

The latest step is required because the copied files owned by root after the cp command.
Of course on a havily used system it is better if you copy only the EBSapps directory.

Login with applmgr, go into the new file system directory and search for symbolic links. On linux you could use this command for searching
$ find -type l -exec ls -lad {} \;

I have to repaired these files, all of them linked to the original file system instead of to new one:
EBSapps/10.1.2/bin/lbuilder
EBSapps/10.1.2/bin/webcachectl
EBSapps/10.1.2/sysman/webapps/emd/WEB-INF/lib/log4j-core.jar
EBSapps/10.1.2/webcache/examples/readme.examples.html

Run postclone on this newly copied and checked "patch edition" file system.

$ cd <COMMON_TOP>/clone/bin
$ perl adcfgclone.pl appsTier

Where <COMMON_TOP> is for example <target base directory>/fs2/EBSapps/comn.

The first 3 question will be the same as earlier (apps, weblogic password, "do you want to add node"). On the 4. question "File Edition type" say "patch" instead of "run".
On next parameter gave the new run edition context. in my example:
<target base directory>/fs1/<inst_top>/appl/admin/<context file>

The script will read all necessary information from the xml and ask only for these:

• port pool - give a different port pool than the run edition
• utl_file_dir - give the same as the run edition

Now we have to have these directories: 

<target base directory>/11.2.0
<target base directory>/data

<target base directory>/fs1/EBSApps

<target base directory>/fs2/EBSApps

<target base directory>/inst

And a halfly running system :) The script will not ask for starting the patch edition.

Finishing Tasks

Follow the instructions in the "Finishing Task" chapter of the document