Manage Mass Provisioning Using Oracle Enterprise Manager Grid Control

Learn how to provision literally hundreds or even thousands of systems using the mass provisioning techniques described here.

Let's assume that your company’s CTO has asked the IT department to deploy all software, applications, and databases in a centralized, automated fashion. Is that a tall order? Not as much as you might think.

By using Oracle Enterprise Manager's Provisioning and Patch Automation Management Pack, the entire software stack can indeed be provisioned - including the operating system, middleware, and Oracle Database Homes. You can also clone databases, whether clustered or non-clustered. In this approach, you can provision literally hundreds or even thousands of systems using these mass provisioning techniques in Oracle Enterprise Manager Grid Control.

Consider, step by step, the scenario of provisioning Oracle software on a new target server with an existing OS, which can be anything – Linux, Solaris, HP-UX, AIX or Windows.

The first step would be to install the Enterprise Manager Agent on a new Agent Home on the target server. Oracle allows a number of methods to install the agent on a new server, such as using the Agent Pull method using a supplied agent-download script that uses the wget utility to transfer files, or by using the newer Agent Deploy (Push) method that requires the setup of SSH equivalence between servers.

Other methods exist such as using the installation of an agent on a shared NFS filesystem and sharing the Agent Home binaries among different target servers, and the latest Agent Cloning method that involves a series of documented steps to copy an Agent Home to different servers.

When the installation of the Agent on the target server is completed using any of the above methods, the Agent starts collecting information about the component targets on the server, and uploads the information continuously to the Oracle Management Service (OMS) using http or https, this metric information is then stored in the Oracle Enterprise Manager repository.

The central Oracle Enterprise Manager is now fully aware of the target server and all the components therein, and the server appears as a target in the Oracle Enterprise Manager Grid Control Console.

The new Target server at this point of time contains an Agent Home where the Agent binaries are installed. However, because the Oracle Database Enterprise Edition (EE) software is yet to be installed on this server, there is no Oracle Home.

You could install the Oracle Database software in the old-fashioned way using the Oracle Installer in interactive or silent mode, but manual steps are involved in this process and it quickly becomes a tedious and error-prone process when the same task is to be repeated again and again on hundreds of servers – even if silent scripts were to be used.

Or you could simply use the Oracle Enterprise Manager Provisioning and Patch Automation Management Pack at this stage, to “clone” the Oracle Home from an existing Oracle Home, or from a “Gold Copy” – which is an interesting concept explained later in this article.

Clone Wizard versus the New Deployment Procedures

Oracle Enterprise Manager supplies a “Clone Wizard” which was the original way of cloning Oracle software and databases using Enterprise Manager. This functionality is not certified to work with Oracle Database 11.2 and will ultimately be superseded by the new Deployment Procedure technology that we will explore in a later section. However, the Clone Wizard is worth having a look at, to see how Enterprise Manager first started with cloning Oracle Homes in a simple, wizard driven way. We will use this to clone an Oracle 10g Database home.

The release used for illustration purposes in this article is Oracle Enterprise Manager 10g Grid Control Release 5. In this release, as well as in the current release (Oracle Enterprise Manager 11g Grid Control Release 1, at the time of writing), the steps you follow are generally the same. The specific enhancements in the 11g release are pointed out in a later section in this article. We have also placed some screenshots from the 11g release where applicable in this article.

Login to the Oracle Enterprise Manager Grid Control console, and move to the Deployments Tab (seen in Figure 1). Under the Cloning section, notice that there is an option to clone the Oracle Home as well as the Oracle Database. Click on the Clone Oracle Home link.

Figure 1

This starts the Clone Wizard and brings up the Clone Oracle Home: Source Home page (Figure 2). Now, you can select the Source Home to be cloned.

Notice that the default Source Type on this page is “Installed Oracle Homes”, and hence the table shows the installed Oracle Homes on different hosts that can be cloned by this provisioning procedure. The platforms, database versions and patchset levels are also visible as also the targets on each host (so it is possible to understand what software is present in the Oracle Home being cloned).

Figure 2

The page clearly mentions that the Clone Wizard can be used for cloning Oracle software such as the Oracle Database, Clusterware, Application Server, and so on. However, for advanced operations such as switching users with sudo or pbrun, or if you want to include database creation along with the software cloning, the use of the more sophisticated “Deployment Procedures” is recommended – these are a highly customizable series of pre-defined steps that we will look into in a later section in this article.

Note the different types of Oracle Database 10g and 11g homes available for cloning, and also the Oracle CRS and Oracle ASM homes – all of these can be cloned. You have selected the /app/oracle/product/db/sidb (OraDb10g_home1) on the tdsmaa04-d1.oracleads.com host as the Source Home. The exclamation mark signifies that there are critical patch advisories that apply to this home. This is obviously an Oracle Database 10g Home.

It is possible to change the Source Type on this page by selecting a different value from the dropdown box. If you select “Software Library” as the Source Type, then this will be treated as the source for the provisioning procedure. In the page that appears (Figure 3), an 11.1.0.6 Gold Database Oracle Home image can be seen in the list.

But what is this Software Library and Gold Image?

Figure 3

Software Library with Gold Copy or Image

A Software image library is used in the process of provisioning, as a means of storing standardized images that have been pre-installed and possibly patched upto an appropriate level. This is known as a “Gold” copy or image.

For example, a Gold copy could be an image of a fully patched Oracle Home (such as an Oracle 10g Home patched to 10.2.0.5 with the latest Patchset Update (PSU) or Critical Patch Update (CPU) applied), installed as per the corporate database standards and security standards, and with only the licenced options installed as per the coporate licence.

The options selected could be the Oracle Label Security Option and/or the Oracle Database Vault Option, if the corresponding licences have been purchased and if these options are considered necessary to be pre-installed with all new Oracle Database Homes as per corporate security policy. We should note here that the Oracle Installer allows these options to be selected when “Advanced Installation” and a “Custom” install of the Oracle Database software is being performed.

The Gold Copy could have these options pre-selected and pre-installed and therefore avoid any oversight on subsequent installations by the DBA, since the DBA would simply be using the Gold Copy to create a cloned Oracle Home.

The other possibility is that certain unlicensed options may be inadvertently installed during manual installations of the database, such as when Orace Database 11g is installed using the Oracle Installer and “Enterprise Edition” is selected under Advanced Installation. This would automatically preselect and install the Oracle Partitioning option and Oracle Advanced Security option, and if these database options are used in any way in the future, this usage would be detrimental to Oracle license compliance.

The Gold Copy would however prevent such an event by ensuring, in the first place, that such unlicensed options are not installed in the Gold Copy.

The Gold Copy or Image can be thought of as a managed set of software components (that can be the operating system software, Oracle software, or other third-party software), directives (executable instructions used for provisioning) and other information bundled together to construct an independent deployable unit. The Software Library in Oracle Enterprise Manager is used to maintain the software components that can be combined with other components as required so as to form the final software configuration (the Gold Copy) that can now be provisioned. The Gold Copy must also refer to staging, preinstall or postinstall directives.

Once the Gold Copy of an Oracle Home has been set up in this manner in the Software library, new Oracle Homes can be created on any other servers (on which the Oracle Enterprise Manager Agent has been installed) by simply running the Clone Oracle Home utility or the Database Provisioning Deployment Procedures (these are preferable to the former). The new Home will be an exact copy of the Gold image and as such will be of the same version, patchset level, and with all the appropriately licensed database options included in the Gold image. You no longer have to worry about the starter DBA in the application team installing the entire suite of options or patching the Oracle Home inappropriately, or not patching it at all.

Hundreds and even thousands of database servers can be provisioned in this way from the Gold copy in an automated matter and in much less time as it would take to perform the core software installation manually, patch to an appropriate level manually and also ensuring at the same time that all the database standards have been observed and only the licensed options have been included – a tall order for the manual method, no matter how scripted, when it comes to thousands of provisioned databases per year.

The Gold Copy Provisioning thus effectively reduces the implementation time and eliminates the overall human error inherent in a step-by-step manual process or in a manually scripted process.

Source Settings

You can now proceed to answer the rest of the Cloning Wizard questions.

In the Source Home page (Figure 2), you selected an existing Oracle Home and then clicked on the Next button. This brings up the Source Settings Page (Figure 4). On this page, select a working directory on the source host to stage the files for cloning.

Figure 4

At this point, you would want to exclude certain files from the copy process – specify these under the section “Files to Exclude”. A default list of the files to be excluded is displayed. You can change this list if desired:

  *.log,*.dbf,*.aud,*.trc,EMStagedPatches,sqlnet.ora,tnsnames.ora,listener.ora,oratab

Logically, this list exists since there is no need to copy log files, data files, audit files and trace files from the Source Oracle system. You also need not copy Oracle SqlNet files such as tnsnames.ora, sqlnet.ora and listener.ora that may contain server-specific details (the hostname or IP address of the source server).

The operating system directories where Oracle Enterprise Manager patches are staged, or the oratab file do not need to be included as well – the latter is usually to be rebuilt differenty on each server as per the databases existing on that server.

Select Next. If there is insufficient space in the working directory location, such as happened when we initially selected /tmp, then the following warning appears. Clear the space in /tmp or choose another working directory on the server before trying again.

Review the following warnings before continuing. The issues below must be addressed before submitting the Clone Home job.

 tdsgcadmin01-rep.oracleads.com - Insufficient space on /tmp to clone this home to a different host.

The estimated space required to archive this home is 3417 MB. If cloning to the same host, archive space is not required.

Destinations

In the next screen that appears (Figure 5), the destination host can now be selected. The space requirement for the new Home is also displayed on this page. In our case it is 3417 MB.

Figure 5

Click the torch icon next to the Host field on Figure 5. This brings up a list of target servers (Figure 6) with the same operating system as the source server – no other servers can be seen in this list, although they may be visible elsewhere as targets in the Oracle Enterprise Manager Grid Control Console.

Select tdsmaa04-d3.oracleads.com as the destination host. This already has an Oracle CRS (Cluster Ready Services) 11g Home, Oracle ASM (Automatic Storage Management) 11g Home, Oracle Database 11g Home and an Oracle Enterprise Manager 10g Agent home that are all marked as independent products.

Figure 6

After selecting the destination host, you need to enter the new Oracle Home location (Figure 7). The cloning will take place to this location. Also give a new name to the Oracle Home (in this case, the name OraDb10g_clonehome is used for illustration purposes).

Next, the host credentials for the destination host can be entered and the working directory is specified on the same page.

Figure 7

Pre-cloning and/or post-cloning operations (either scripts or host commands) to customize the cloning operation can be specified on the next page (see Figure 8). You can choose if sudo is to be used or not in these operations – this will not be required if the operations do not require root privileges.

However, the option “Running root.sh” does require root privileges. If and only if the oracle user you specified in the credentials page has sudo privileges, then you can choose to execute the root.sh script on this screen, otherwise the system administrator can be asked to run this script later. The warning displayed is that some functionality may not be available if the script is not run immediately after the install.

Figure 8

Click Next. This is followed by another page where you can enter a number of operating system specific parameters for the destination home, such as the names of the Oracle oper and dba groups and the asm admin group, the complete Oracle Base path and also the Oracle RAC node names if the destination is an Oracle RAC cluster of nodes. See Figure 9.

Figure 9

The following page (Figure 10) allows you to schedule the cloning process, either immediately or at a scheduled time. In this case, you decide that the job will be run immediately.

Figure 10

The Review page is now displayed – here, ensure that all details are correct. Warnings (such as lack of space), if they exist, may also appear on this screen (Figure 11).

Figure 11

Cloning Begins

When you click on the Finish button, a confirmation that the Clone Home job has been submitted appears along with the message: “If you have chosen not to run the root.sh script as part of the clone, be sure to run it on each of the target homes after the job has completed successfully.”

At this point, the cloning operation begins and its progress can be monitored by selecting View Job (Figure 12).

Figure 12

As the job starts executing its series of steps, the intial step is seen to be “Prepare Source Home”. If you click on this step, it displays the series of commands that are used to perform the preparation of the home. Essentially in this step, a Unix tar file is created in a temporary directory with the exclusion of certain files, and the tar file is then compressed. The output log of this step shows this process clearly and is reproduced below.

			 Output Log

chdir(/app/oracle/product/db/sidb)

/app/oracle/product/em/agent10g/perl/bin/perl ./clone/bin/prepare_clone.pl ORACLE_HOME="/app/oracle/product/db/sidb" 2>&1
mkdir /app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45

/usr/bin/find . -name "*.log" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "*.dbf" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "*.aud" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "*.trc" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "EMStagedPatches" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "sqlnet.ora" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "tnsnames.ora" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "listener.ora" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"

/usr/bin/find . -name "oratab" >> "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"
chdir(/app/oracle/product/em/agent10g/perl/bin/)

./perl -w "/app/oracle/product/em/agent10g/sysman/admin/scripts/osm/ecmHomePackager.pl"
 -export "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/homeArchive"
"/app/oracle/product/db/sidb" "*.log,*.dbf,*.aud,*.trc,
EMStagedPatches,sqlnet.ora,tnsnames.ora,listener.ora,oratab"
"/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt"
Use Zip Option :: FALSE

/bin/csh -cf '/bin/tar cpfXh - "/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/excludedFileList.txt" .
 | /usr/bin/compress -fv >
"/app/oracle/tmp/oemchs8627D0F9DD8A4EA6E040928D4B1A6F45/homeArchive"'
            

 C:\OracleHomes\agent10g\bin\zip -r -S -9 "C:\temp\oemchsED724047ACDB4DBCB7C2EE14AC386371\homeArchive"

			* -x "" "*.log" "*.log/*" "*.dbf" "*.dbf/*" "*.aud" "*.aud/*"
"*.trc" "*.trc/*" "EMStagedPatches" "*/EMStagedPatches" "EMStagedPatches/*" "*/EMStagedPatches/*"
 "sqlnet.ora" "*/sqlnet.ora" "sqlnet.ora/*" "*/sqlnet.ora/*"
"tnsnames.ora" "*/tnsnames.ora" "tnsnames.ora/*" "*/tnsnames.ora/*" 
"listener.ora" "*/listener.ora" "listener.ora/*" "*/listener.ora/*" "oratab" "*/oratab"
"oratab/*" "*/oratab/*" 1> NUL 

Figure 13

Another step in this job is to Update the Built-in Clonable Home. This step uses the Oracle Installer in the silent mode, and has the following output:

Output Log 
 
chdir(/app/oracle/product/em/agent10g/jdk/jre)

./bin/java 
 -cp "/app/oracle/product/em/agent10g/oui/jlib/xmlparserv2.jar":
 "/app/oracle/product/em/agent10g/oui/jlib/OraInstaller.jar":"/app/oracle/product/em/agent10g/jlib/emCfg.jar":
"/app/oracle/product/em/agent10g/oui/jlib/srvm.jar":"/app/oracle/product/em/agent10g/oui/jlib/share.jar":
"/app/oracle/product/em/agent10g/sysman/jlib/emd_java.jar":
"/app/oracle/product/em/agent10g/sysman/jlib/emagentSDK.jar":"/app/oracle/product/em/agent10g/sysman/jlib/emcoreAgent.jar":
"/app/oracle/product/em/agent10g/sysman/jlib/log4j-core.jar" 
-Doracle.installer.oui_loc="/app/oracle/product/em/agent10g/oui" oracle.sysman.emd.ecm.clone.DestEnvironment -isPreCloneNeeded 
"/app/oracle/product/10.2/db_7" |chdir(/app/oracle/product/10.2/db_7/clone/bin)

/app/oracle/product/em/agent10g/perl/bin/perl ./clone.pl -waitForCompletion ORACLE_HOME="/app/oracle/product/10.2/db_7" 
ORACLE_HOME_NAME="OraDb10g_clonehome"
 oracle.rdbms:11.1.0.6.0:s_nameForOPERGrp=oinstall
 oracle.rdbms:11.1.0.6.0:s_nameForDBAGrp=dba oracle.rdbms:11.1.0.6.0:s_nameForASMGrp=oinstall 
REMOTE_NODES={} ORACLE_BASE=/app/oracle/product LOCAL_NODE= CLUSTER_NODES={}   2>&1

./runInstaller -silent -clone -waitForCompletion  -waitForCompletion 
 "ORACLE_HOME=/app/oracle/product/10.2/db_7" 
"ORACLE_HOME_NAME=OraDb10g_clonehome" "oracle.rdbms:11.1.0.6.0:s_nameForOPERGrp=oinstall" 
"oracle.rdbms:11.1.0.6.0:s_nameForDBAGrp=dba" 
"oracle.rdbms:11.1.0.6.0:s_nameForASMGrp=oinstall" "REMOTE_NODES={}"
 "ORACLE_BASE=/app/oracle/product" "LOCAL_NODE=" "CLUSTER_NODES={}" -silent 


Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 1709 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2010-05-09_08-50-39AM.
 Please wait ...Oracle Universal Installer, Version 11.1.0.6.0 Production
Copyright (C) 1999, 2007, Oracle. All rights reserved.

You can find the log of this install session at:
 /app/oracle/product/oraInventory/logs/cloneActions2010-05-09_08-50-39AM.log
.................................................................................................... 100% Done.

Installation in progress (Sun May 09 08:51:06 CDT 2010)
.............................................................................         
                                          77% Done.
Install successful

Linking in progress (Sun May 09 08:51:25 CDT 2010)
Link successful

Setup in progress (Sun May 09 08:53:03 CDT 2010)
Setup successful

End of install phases.(Sun May 09 08:53:08 CDT 2010)
WARNING:
The following configuration scripts need to be executed as the "root" user.
#!/bin/sh
#Root script to run
/app/oracle/product/10.2/db_7/root.sh
To execute the configuration scripts:
    1. Open a terminal window
    2. Log in as "root"
    3. Run the scripts
    
The cloning of OraDb10g_clonehome was successful.
Please check '/app/oracle/product/oraInventory/logs/cloneActions2010-05-09_08-50-39AM.log' for more details.

rmtree(/app/oracle/tmp/oemchd8627D0F9DD8A4EA6E040928D4B1A6F45, 0, 0)
2 

At this point, the root script “root.sh” is executed. In this step the unix user oracle attempts to execute '/app/oracle/product/db/sidb/root.sh' as root on tdsmaa04-d3. To perform this activity, sudo rights are required for oracle on this server.

Next, the post-cloning scripts are executed and this is followed by the “Refresh Host Config” step. This step shows the following output:

  Output Log


 Starting collection...

 Successfully collected configuration for Host tdsmaa04-d3.oracleads.com.

 Capturing collected information into repository...

 Successfully recorded information into repository.

As can be seen in the log, the Oracle Enterprise Manager agent collects the configuration from the Host, this includes the information on the new Oracle Home which is now registered in the Oracle Enterprise Manager repository. After this, the source temporary files are removed from the working directory, and the overall Clone Oracle job has completed successfully (Figure 13).

A brand new Oracle Home has been provisioned on the destination server in this way.

New Database Provisioning Procedures 

As we have seen, the Clone Wizard performed a simple cloning of a 10g Oracle Home, but there are obvious limitations – it is not certified to work with Oracle Database 11.2. Also, it is not fully customizable.

So, instead of using the Clone Wizard, it is recommended to use the new functionality of “Deployment Procedures”. These are highly customizable sets of pre-defined steps used for database provisioning, allowing advanced operations such as using sudo or pbrun, and the possible inclusion of database creation along with the software cloning. We can now take a look at these procedures.

Move to the Deployments tab in Grid Control. Under the section “Deployment Procedure Manager”, select the Database Provisioning Procedures link. This displays available out-of-the-box and customized deployment procedures for deploying databases and replay clients (see Figure 14).

The Oracle Replay Client is a program which is available as part of the standard Oracle Client or the Oracle Instant Client. It is a multithreaded executable “wrc” – each thread responsible for submitting a captured session's workload. This is used for the Database Replay feature of the Oracle Real Application Testing Option (licensable) for Oracle Enterprise Edition.

Figure 14

Out-of-the-Box and Customized Procedures

As we have seen, Oracle Enterprise Manager supplies a number of out-of-the-box deployment procedures. Please note that it is not necessary to customize any procedure, in most cases the functionality provided by the out of the box procedures is enough. Customization provides an added benefit if you so desire.

On the Procedures page in Figure 14, it is possible to customize a procedure by simply selecting “Create like” (provided you are logged into Oracle Enterprise Manager as a user with the necessary rights) and create a new procedure with a new name. In a customized procedure, you can change the error handling of any steps (either stop on error, or continue on error) or you can disable or enable any steps.

The first procedure “Oracle Single Instance DB Provisioning” in the list above is such a customized procedure – this is obvious due to the different update dates and the different owner visible on the page (Figure 14). This is a copy of the out-of-box procedure “Oracle Database Provisioning” but with some minor steps disabled for the environment of the test. You will use this procedure for the purpose of this demonstration.

Select the “Oracle Single Instance DB Provisioning” procedure and click on the View button. This displays the detailed series of steps of this deployment procedure (Figure 15).

Figure 15

Ignore the fact that the “Register Database Targets in Enterprise Manager” step has been disabled, this step is normally enabled and is only disabled in this case because of the particuar testing environment that we have used. This step does not need to be disabled in a normal scenario and for all purposes you can consider it to be enabled, as it is in the normal out-of-box procedure.

Schedule Deployment

Back on the list of procedures in Figure 14, select the “Oracle Single Instance DB Provisioning” procedure and click on Schedule Deployment.

This brings up the Select Source and Destination page (Figure 16).

Figure 16

On this page, select the Software Library radio button. This will allow you to use a Gold image from the software library as the source for the provisioning operation. Alternatively, you could choose an existing reference installation as the source. The other possibility is to select “Staging Location” which will allow you to use components created using the installation media in the software library.

At this point, use the Gold image as the source. Click on the torch icon, this brings up the Components page in Figure 17. Simply select the existing “11.1.0.6_Gold_DBOH_Image” from the list.

It is also possible to upload a different component at this point if you wish, by clicking on the Upload link on this page. This will upload it to the Software library.

Figure 17

After selecting the Gold image, you will be brought back to the Select Source and Destination page (Figure 16). Click on the Add button under the Specify Destination Host Settings section to add the destination host. This displays the window in Figure 18.

By default, only suitable hosts best suited for provisioning the Oracle database are displayed in this window. Select Show all hosts to display all the hosts, and click on the Go button. Tick the host tdsmaa03-d2.oracleads.com and then the Select button (Figure 18).

Figure 18

You are now back on the Select Source and Destination page (Figure 16). Click on the button Customize Host settings if you want to provide additional information such as Agent credentials, Custom Oracle Home name and Custom Host name. You would do this if your agent home and Oracle Home credentials are different. This brings up the Customize Host settings window (Figure 19).

Figure 19

Tick Agent Home Credentials and click on Customize. This will take you back to the Select Source and Destination”page with the Agent Home credentials now appearing on this page as well (see Figure 20). Now, fill in more settings for the target server like the Oracle Base directory location, Oracle Home and the working directory, as well as the Oracle Home credentials and the Agent Home credentials.

Multiple Targets

We should note here that it is possible to add multiple destination targets on this page for the same provisioning session. Thus you can easily schedule a provisioning of multiple targets from a single Gold image, a reference installation, or the staging location.

You can change Paths to “Different for each host” and Credentials to “Override – Different for each host” once multiple targets are added on this page (Figure 20).

Figure 20

As the Additional Parameters on this page, you can provide any Oracle Installer parameters you want to be used, such as –debug. You can also specify a shared location to stage the binaries. This location should be shared across all destination hosts. Specifying a shared staging location in the case of multiple destination hosts will obviously reduce the provisioning time, since the binaries will not need to be copied to each destination host before installation can start. They only need to be copied to the shared location by the deployment procedure.

Tick Create and configure Database to also include a database in the provisioning operation along with the installation of the database software binaries. If this is not done, you will have to use the Oracle Database Configuration Assistant (DBCA) to manually create and configure a database.

At the bottom of the page, you can chedule the provisioning operation, either to run immediately or at a future scheduled time. Go to the Next page when you are done.

Configuring the Database

The Database Configuration page now appears (Figure 21).

Figure 21

On this page, you can specify the Database Configuration details. Type in the Data File location, Oracle SID (instance name) and domain for all the destination hosts you have specified, as well as the database password. The path and the password can either be the same for all the hosts or different.

Under the Advanced Database Parameters section, you can specify Oracle Database Configuration Assistant (DBCA) arguments in the Additional Database Parameters box, such as the characterset to use for the database.

In the other check boxes, you can specify if you want DBCA to use the Custom Database Response File and/or the Custom Database Template File, these can be used to create and configure databases as per corporate database and security standards. For example, it will be possible to exclude or include certain database options, set certain parameters and so on. If no templates are specified, default out of the box templates will be used by DBCA.

It is also possible to create a database that uses Oracle Automatic Storage Management (ASM) for data storage. Simply use a response file with the appropriate answers and settings for ASM.

Move to the Next page. Figure 22 appears.

Figure 22

On this page, you can specify an Email address and your Oracle Support password to receive security updates from Oracle Support. Select Next to move to the next page.

The Review page is now displayed (Figure 23). Check all the settings on this page and make sure they are correct. Then click on the Finish button.

Figure 23

Launching Provisioning

The Database Provisioning operation is launched at this point. It appears immediately in the Procedure Completion Status screen (Figure 24) as status “Scheduled” and then starts.

Figure 24

Drill down on the procedure you just launched by clicking on the corresponding link on this page. This enables you to see the progress of each of the steps as they are being executed (Figure 25).

Figure 25

You can now sit back, relax and wait for the steps to complete.

Some steps may be skipped depending on the prior selections in the question and answer screens. The procedure checks all the required software and hardware pre-requisites and makes sure that the database is provisioned using the Oracle recommended best practices. Some of the failed requirements are also fixed automatically, such as kernel settings.

After some time, the steps complete successfully and the deployment procedure is completed and appears as status “Succeeded” (Figure 26).

Figure 26

After the database targets are registered in Oracle Enterprise Manager, the new database “newdb.tdsmaa03.oracleads.com” along with its newly provisioned Oracle Home “/app/oracle/newdb” appears in the list of Oracle Enterprise Manager targets, such as the Host Targets screen seen in Figure 27. The status is also seen as “Up”.

We should note here that there may be a gap of a few minutes after registration, before the new target appears in Enterprise Manager. Simply refresh the screen after a few minutes.

Figure 27

You have seen the ease and flexibility of the powerful Oracle Database provisioning deployment procedures. Using one of these procedures, the benefits are obvious – a new Oracle Home and Oracle Database has been set up and configured in a fraction of the normal manual DBA timeframe. Human error has also been greatly reduced due to the seamless automation of the provisioning procedure.

It is now your turn to perform mass provisioning – using the advanced Deployment Procedures supplied by Oracle Enterprise Manager Grid Control, you can easily provision hundreds and maybe thousands of such new Oracle Homes and Databases in your corporate space.

Provisioning Techniques versus Manual Copy of Home

Oracle DBAs have known for years that manual copying of Oracle Homes using Unix-level utilities such as tar or windows utilities such as zip is not supported by Oracle Support as a valid method of installing software. As such, this method is not recommended in the corporate space.

Yet, there are some companies that persist in this method, by using complicated scripts that have been maintained by teams of DBAs over the years and need to be updated by new DBAs for every new version of Oracle. These methods also do not update the Oracle Inventory on the target server, nor do they update the oratab file. As such these manually created Oracle homes will not be discovered by the Oracle Enterprise Manager agent.

Hence, the better alternative is obvious: Simply use the Provisioning Procedures provided by Oracle Enterprise Manager. These deployment procedures are highly sophisticated and are fully certified by Oracle Support and also enable you to provision Oracle Clusterware besides Oracle clustered and non-clustered databases. As Oracle database versions change, the procedures are intelligent enough to cater for cloning all aspects of the Oracle database or Oracle Clusterware.

Of course, remember that the Oracle Enterprise Manager Provisioning and Patch Automation Pack license is required to use the Deployment procedures.

Gold Copy of Oracle Database

As we have seen, an Oracle database itself can also be used as a Gold Copy, in the sense that you can construct a database as per the corporate database standards – ie. with directories using Oracle Flexible Architecture (OFA) directory structure or not, specific naming of datafiles and tablespaces, extent management, suitable initialization parameters, and with licenced options appropriately installed.

Once this master database is setup and tested and is certified as functional by the Database Engineering department, it can be uploaded to the Software library so that a Gold Image of the database is created. This then can be cloned to any server which needs a new Oracle database.

So, cloning Oracle Homes is not the only use of this powerful pack; it also allows you to clone Oracle Databases fast and seamlessly either to the same server or to totally different servers. You can easily create test or development instances from production, as an example. Remember to combine this feature with the Oracle Enterprise Manager Data Masking pack, which helps you to mask confidential data in production at the end of the cloning process.

A Gold image of an existing Oracle Database (clustered or non-clustered) can be created using the Create Component page in Oracle Enterprise Manager Grid Control, on the Deployments Page and Provisioning Tab. Full details on how to create the Gold image can be seen in the reference documention listed in the last section in this article.

We should note here that from Oracle Enterprise Manager 10g Grid Control Release 5 (10.2.0.5) onwards, due to customer requests, new Oracle Enterprise Manager access control restrictions were introduced. This means that only Administrators – such as sysman or other super administators of Oracle Enterprise Manager are now able to edit entries in the Software library and Deployment procedures

Bare-metal Provisioning

Want to automate the deployment of the operating system? The process of provisioning the operating system on a totally blank box is known as Bare-metal Provisioning. This is normally accomplished using a standardized PXE (Preboot Execution Environment) booting process and kickstart mechanism to boot new hardware machines and install the Linux operating system or Oracle VM server software.

Out-of-box support is provided to provision 32-bit and 64-bit versions of Oracle Enterprise Linux and Oracle VM for x86, RedHat Enterprise Linux (RHEL) and Suse Linux (SLES).

A boot server is required to perform the network boot for the initial startup of the hardware, and a stage server is required to stage the OS image that is to be deployed, besides an RPM repository configured as a YUM repository and the Software Library for software components and directives, this will be added on top of the stack.

When Linux is provisioned, the Oracle Enterprise Manager Agent is automatically installed after the Linux install completes. When Oracle VM servers are provisioned, Oracle VM server software is automatically installed. You then need to register the new Oracle VM servers in Oracle Enterprise Manager to start managing the Oracle VM servers using Oracle Enterprise Manager.

Using the Oracle Ops Center Provisioning and Patch Automation pack, it is also possible to provision Oracle Solaris SPARC 8,9 and 10, and also Oracle Solaris x64/x86 9/10 in addition to the above Linux versions.

Provisioning and Patch Automation Enhancements in Oracle Enterprise Manager 11g

The new 11g release of Grid Control is based on the foundation of Oracle WebLogic Server (WLS) 11g Release 1 Enterprise Edition (10.3.2) instead of Oracle Application server used by the 10g release of Grid Control. Therefore, the 11g release can potentially benefit from the performance and memory management features associated with Oracle WebLogic Server. Note that at this point of time you can install Enterprise Manager Grid Control on an Oracle WebLogic Server Cluster, but you cannot take advantage of the cluster configurations.

What’s new for the Provisioning and Patch Automation pack? The new release delivers full configuration management and provisioning for large-scale Oracle SOA Suite and Oracle WebLogic Server environments via new deployment procedures. You can provision WebLogic Server – such as, clone from a Weblogic “Gold Copy”, and Weblogic cluster scale out and scale back is enabled. Oracle WebLogic Server is often used in an “application grid” environment, or it can also be used as the foundation for a private cloud. Oracle Enterprise Manager 11g can therefore greatly simplify the management and provisioning of such deployments, including the private cloud. It is also possible to provision the Database Machine of Exadata V2 using a special deployment procedure.

There is now full integration with My Oracle Support (MOS), with complete patching lifecycle automation, SR integration, detection of patch conflicts, the ability to create merge requests automatically and to interact with the community of Oracle users and benefit from their experience with the patches. A new feature called a “Profile” is used in deployment features for RAC provisioning, this allows all inputs to the deployment screens to be recorded and used to further automate succeeding deployments. The pack supplies four out-of-the-box profiles to be used for RAC provisioning.

So far as patching plans are concerned, the 11.2 database version is now supported for the first time. Existing patching procedures have also been enabled to patch the Oracle Grid infrastructure aspect of RAC as well as non-RAC databases. In the case of provisioning – in earlier releases, the wizard “Oracle Home Clone” could be used to clone Oracle Homes, but this has been deprecated and should no longer be used – it is not certified to work with Oracle Database 11.2. (Certification details for this wizard as well as deployment procedures are in My Oracle Support (MOS) Note 737939.1.) No new versions of databases or middleware will be certified with this wizard. The only alternative is to use Deployment Procedures – for example, in the case of Oracle 11.2 non-RAC databases, use the “Oracle Database Provisioning” deployment procedure.

In Oracle Enterprise Manager 11g Grid Control Release 1, a new deployment procedure “Oracle Grid Infrastructure Provisioning for Standalone Servers” has been provided for the first time. You can use this to provision the new Grid Infrastructure in the case of 11.2 non-RAC Databases. On the Deployments Tab, select “Database Provisioning Procedures” under the “Deployment Procedure Manager” section. This displays the new Grid Infrastructure Provisioning procedure (Figure 28).

Figure 28

Another new procedure “Oracle Grid Infrastructure / RAC Provisioning” is provided to provision 11.2 RAC databases and their corresponding Grid Infrastructure. Whereas, the existing procedures “One Click Extend Cluster Database” and “Delete/Scale down Oracle Real Application Clusters” have now been certified with Oracle 11.2 RAC Clusters for the first time. On the Deployments Tab, you can select “RAC Provisioning Procedures” under the “Deployment Procedure Manager” section. This displays the new Grid Infrastructure / RAC Provisioning procedure (Figure 29).

Figure 29

Pre-requisites are now dynamic for the first time – as in the case of the RAC provisioning, the software can automatically download the latest pre-requisites for the purpose of RAC provisioning, which can then be used. Bare-metal provisioning is now supported for Oracle VM 2.2 and Oracle Enterprise Linux 5. Provisioning of Linux, Solaris and Windows as Guest Virtual Machines (VMs) can also be performed on Oracle VM servers. You can also provision your Oracle Automatic Storage Management (ASM) installations, as also easily provision your Clusterware. Grid Naming Service (GNS) support is provided.

Additional Details and Conclusion

Oracle Enterprise Manager Targets can be patched at the Oracle Database, Oracle Automatic Storage Management (ASM) or Oracle Clusterware level using customizable Deployment Procedures that automate the process of patching multiple targets with multiple patches in a highly efficient manner, including RAC Rolling upgrades. Besides the tight integration with My Oracle Support, there is also a Critical Patch facility that connects to My Oracle Support (this could be a direct connection or in an offline mode) and discovers the latest patches available for the Oracle Homes in your installation – such as the most recent Critical Patch updates (CPU) released quarterly by Oracle. Linux, Solaris and Windows operating systems can also be patched using this pack.

To automate the patching of all your Oracle RAC or non–Oracle RAC databases, Oracle Automatic Storage Management (ASM) instances, and Oracle Clusterware in your environment, read the article by the author titled “Patch a Thousand Databases, Using Oracle Enterprise Manager Grid Control”.

You can refer to more details on Provisioning in Oracle Enterprise Manager Administrator's Guide for Software and Server Provisioning and Patching 11g Release 1 (11.1.0.1.0) which is available on the Oracle Technical Network via this link.

It is obviously necessary that the Oracle Enterprise Manager Grid Control site should be properly architected to perform different Database Administration tasks, including the Provisioning and Patching from the Enterprise Manager Software library. For an overview of a recommended scalable architecture for Grid Control.

For easily setting up Oracle Recovery Manager (Oracle RMAN) backups for your corporate databases using Oracle Enterprise Manager Grid Control, read “Oracle RMAN Backups: Pushing the Easy Button”. You may also be interested in data integration, which you can read about in “Using Oracle GoldenGate for Real-Time Data Integration”. For understanding how Oracle Enterprise Manager Grid Control can save a lot of time and resources when used for Oracle Data Guard setup, management (including switchover or failover), and monitoring, read another article from the author, “Easy Disaster Proof Production with Grid Control”. And to see how easily the Data Masking Pack of Oracle Enterprise Manager Grid Control can obsfucate your confidential production data as an integral part of the database cloning process, read the article "Mask Your Secrets Using Oracle Enterprise Manager."

Porus Homi Havewala works as a Senior Manager (Enterprise Technology) at Oracle Corporation Singapore, and was previously a Principal Consultant at S&I Systems Singapore (Oracle Platinum Partner). He was awarded the Oracle ACE Director title by Oracle HQ in 2008 and is currently an Oracle Employee ACE. He has extensive experience in Oracle technology since 1994, including as a Senior Production DBA, Principal Database Consultant, Database Architect, E-Busines. He has published numerous articles on Grid Control on OTN, and created the world's first blog dedicated to Grid Control. Porus is also the author of the forthcoming book Oracle Enterprise Manager Grid Control Advanced Techniques for the Real World.