Configuring and using Coherence Persistence

 

Before You Begin

Purpose

In this tutorial you will learn how to install, configure and work with Oracle Coherence Persistence 12.2.1.

Time to Complete

Approximately 2-3 hours.

Background

The Oracle Coherence Persistence feature enables Coherence to be a system-of-record and improves time-to-recovery from planned or unplanned, partial or total cluster outages. Persistence can be configured to work in an on-line mode persisting changes as they are made to the cluster, or to take a snapshot of a data grid. Local disk configuration maximizes scalability, while network-attached storage maximizes availability. Both data and metadata (indexes, listeners, expiry, locks, etc.) are stored, and recovery is completely automatic.

Scenario

Oracle Coherence Persistence is a set of tools and technologies that manage the persistence and recovery of Coherence distributed caches. Cached data is persisted so that it can be quickly recovered after a catastrophic failure or after a cluster restart due to planned maintenance. Persistence and federated caching can be used together as required. 


Persistence-Modes

Persistence can operate in two persistence modes:
  • On-Demand persistence mode – a cache is manually persisted and recovered upon request using the snapshot manager. The snapshot manager is exposed as an MBean interface that provides operations for creating, archiving, and recovering snapshots of a cache. 
  • Active persistence mode – In this mode, cache contents are automatically persisted on all mutations and are automatically recovered on cluster/service startup. The snapshot manager can be used in active persistence mode to perform on-demand snapshots of a cache.

Both the Coherence Query tool and the Oracle Coherence plugin for JVisualVM are aware of and expose the snapshot manager MBeans via various GUI and command line operations.

Disk-Based Persistence Storage
The persistence store is used to store the backing map partitions of a partitioned service. The locations of the database files can be stored on the local disk of each cache server or on a shared disk on a storage area network (SAN).

This image shows two Coherence clusters. On the left is a three computer Coherence cluster beneath which is a single disk, showing a Coherence cluster sharing a disk. On the right is a three computer Coherence cluster beneath which is three disk images, each associated with an individual computer. The computers themselves represent Coherence instances.
Coherence Clusters showing shared and individual disks for persistence


The figure shows two options:
  • The local disk option allows each cluster member to access persisted data for the service partitions that it owns. Persistence is coordinated across all storage members using a list of cache server host addresses. The address list ensures that all persisted partitions are discovered during recovery. Local disk storage provides a high throughput and low latency storage mechanism; however, a partition service must still rely on in-memory backup (backup-count‬ value greater than zero) to remain machine safe.‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬
  • The shared disk option, together with active persistence mode, allows each cluster member to access persisted data for all service partitions. An advantage to using a shared disk is that partitioned services do not require in-memory backup (backup-count‬ value can be equal to zero) to remain machine-safe; because, all storage-enabled members can recover partitions from the shared storage. ‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬ Disabling in-memory backup increases the cache capacity of the cluster at the cost of higher latency recovery during node failure. In general, the use of a SAN can potentially affect throughput and latencies and should be tested and monitored accordingly.


  • Both the local disk and shared disk approach can rely on a quorum policy that controls how many cluster members must be present to perform persistence operations and before recovery can begin. Quorum can ensure that you have enough memory to load the cache contents on cluster startup.

    Persistence Configuration
    Persistence is declaratively specified using Coherence configuration files and requires no changes to application code. Operational overrides are used to configure the underlying persistence implementation if the default settings are not appropriate. Cache configuration is used to set persistence properties on a distributed cache.

    Management and Monitoring
    Persistence can be monitored and managed using MBean attributes and operations. Persistence operations such as creating and archiving snapshots are performed using the Persistence‬ MBean or a tool such as the Coherence Query tool.‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬ Persistence attributes are included as part of the attributes of a service and can be viewed using the Service MBean‬. Persistence attributes and statistics are aggregated in the persistence‬ and persistence-details‬ reports. Persistence statistics are also aggregated in the Java VisualVM plug-in. Both tools can help troubleshoot possible resource and performance issues

    What Do You Need?

    Before starting this tutorial you will need the following:

    Requirement More Information
    Supported Operating System Oracle Coherence runs in any of OS X, Windows, Linux or Solaris. All example commands are given using Windows syntax.
    4 GB of diskspace Coherence, the Java JDK and sample applications require 3-4 GB of disk space.
    4GM Memory This tutorial runs multiple instances of Oracle Coherence and requires an absolute minimum of 4GB of memory.  While its possible to run the tutorial with less memory, performance will suffer and out of memory errors may occur.
    Java SE Development kit 8, for your operating system. Oracle Coherence 12.2.1 requires the Java JDK 1.8 or later. The Java JDK 1.8 can be downloaded here.

    For this tutorial we will assume the Java JDK is installed and pointed to by the JAVA_HOME variable. On Windows this is assumed to be something similar to:

    set JAVA_HOME=c:\"Program Files"\Java\jdk1.8.0_101
    On Linux the variable assignment might resemble:
    export JAVA_HOME=/opt/jdk/jdk1.8.0_101
    Oracle Coherence 12.2.1 Oracle Coherence, as a stand-alone product can be downloaded here.
    Coherence JVisualVM Plugin The plugin may be installed from within JVisualVM and is found in the coherence/plugins/jvisualvm directory of the Coherence install.

 

Installing Coherence and the Coherence Plugin for JVisualVM

The JVisualVM plug In the following two sections you will install stand alone Coherence and update JVisualVM to include the Coherence JVisualVM plugin.

 

Unpacking and installing Oracle Coherence

  1. If required, download and install the Java JDK 1.8.
    Refer to the prior table for the location of the download for your operating system.  Refer to the JDK installation instructions for your operating system.
  2. Open a command prompt or terminal as required by your operating system.
  3. Coherence is provided as a zipped Java jar file. Unzip using a command similar to those below unpack Coherence:
    $ cd c:\downloads
    $ unzip fmw_12.2.1.0.0_coherence_Disk1_1of1.zip
    Archive:  fmw_12.2.1.0.0_coherence_Disk1_1of1.zip
    inflating: fmw_12.2.1.0.0_coherence.jar
    $                                  
  4. Install coherence using a command similar to:
    $ %JAVA_HOME%\bin\java -jar fmw_12.2.1.0.0_coherence.jar
    Note: For the remainder of this tutorial Windows 7 style commands will be shown.
    Linux and OSX users will need to modify these commands to match OS requirements.
    In certain instances both Windows and Linux style commands will be shown.
  5. In the Welcome dialog click Next.
  6. In the Auto Updates dialog click Next.
  7. In the Installation Location dialog enter a fully qualified path to install Coherence into and click Next.
    This image shows a portion of the Installation Location dialog. The portion shown is the field for entering the entering of the fully qualified path to the Coherence home directory. Which for this tutorial is c:\fmw\coherence12.2.1 and is refered to as COHERENCE_HOME.
    Coherence Home

    For the remainder of this tutorial will refer to this location as COHERENCE_HOME.
    Consider using an environment variable set to the coherence directory for ease of use. For example:
    $ set COHERENCE_HOME=c:\fmw\coherence12.2.1\coherence
  8. In the Installation Type dialog select Coherence with Examples and click Next.
  9. In the Prerequisite Checks dialog ignore any warnings and click Next.
  10. In the Installation Summary dialog click Install. Additionally you may save the response file for later review.
  11. The Installation Progress dialog will display and show each stage or the installation. Click Next to view a log of the install or Finish to complete the install and exit the installer.
 

Installing the Coherence and MBeans JVisualVM Plugins

In this section of the tutorial we will install the Coherence JVisualVM Plugin.

  1. At the command line start JVisualVM using a command similar to:
    $ jvisualvm –J-Dcom.oracle.coherence.jvisualvm.refreshtime=10
    Note: The –J-Dcom.oracle.coherence.jvisualvm.refreshtime=10 property changes the data refresh interval from the default 30 seconds to 10 seconds and limits wait time when JVisualVM needs to refresh a panel or pane.
  2. Select Tools> Plugins and then the Available Plugins tab,
  3. Select the VisualVM-MBeans plugin.
  4. Click Install. The Plugin installer wizard will display.
  5. Click Next.  The installer will download and install the latest version of the plugin. 
  6. When complete click Finish.
  7. Select  the Downloaded tab.
  8. Click Add Plugins... and navigate COHERENCE_HOME/coherence/plugins/jvisualvm directory to and select the coherence-jvisualvm.nbm file.
  9. Click Install. The Plugin installer wizard will display.
  10. Click Next.
  11. Read and then accept the terms of the licensing agreement and click Install
  12. If presented with a validation warning click Continue.
  13. The plugin will be installed. Click Finish when complete.
  14. Close the Plugins dialog. The Coherence JVisualVM plug in is now installed.
  15. Exit JVisualVM.
 

Getting Started with Coherence Persistence

By default all distributed and federated Coherence services are automatically configured with ‘on-demand’ persistence.The objective of this section is to use persistence commands from within JVisualVM to create and recover snapshots using Coherence standard defaults.

 

Starting Cache Servers and Populating Data

These next steps all require the use of COHERENCE_HOME and JAVA_HOME variables.
As a convenience a sample script is provide here. This script is shown in step 2 below and used to set up the appropriate values for the Coherence and Java home variables. Use of this script is optional and is not shown through the remainder of this tutorial.

If using the provided Windows script update the script for the COHERENCE_HOME and JAVA_HOME locations in your environment.

  1. Open four command windows, two will be used for Coherence cache servers, one for JVisualVM and one for CohQL commands.
  2. Using two of the terminals, start two Coherence cache-servers using a commands similar to (Windows and Linux examples given):
    $ setup_env.cmd
    $ cd %COHERENCE_HOME%
    $ bin\cache-server.cmd -jmx
    Linux/Unix users use a command similar to:
    $ cd $COHERENCE_HOME
    $ bin\cache-server.sh -jmx
    Note: The -jmx argument adds several parameters to the cache server in support of JMX system management.
  3. In a third command window, start the Coherence Query tool (CohQL) using a command similar to:
    $ bin\query.cmd 
  4. Insert data into a cache using commands similar to:
    CohQL> insert into 'test' key(1) value('value1')
    CohQL> insert into 'test' key(2) value('value2')
    . . . 
    CohQL> insert into 'test' key(10) value('value10')
  5. verify that the data was added using a select command.
    CohQL>  select key(),value() from 'test'
    The command should return results similar to:
    Results
    [1, "value1"]
    ...
    [10, "value10"]
  6. In a command window start JVisualVM using a command similar to:
    $ %JAVA_HOME%\bin\jvisualvm –J-Dcom.oracle.coherence.jvisualvm.refreshtime=10
  7. In the JVisualVM Applications pane select one of the DefaultCacheServer instances by clicking its name.
    THis images shows the JVisualVM Application pane with the first of two Coherence instances selected.
    JVisualVM Application Pane
    The application's information will display in the right hand pane.
  8. In right hand pane select the Oracle Coherence > Caches tab.
    Notice that the cache shows the entries previously added using CohQL.
    This image shows the right pane in JVisualVM after a Coherence DefaultCacheServer was selected. The image specifically shows the selection of the Oracle Coherence tab and the Caches sub-tab. The caches sub-tab should show that a cache exists and that 10 elements were added.
    JVisualVM Caches subtab
  9. In right hand pane select the Oracle Coherence > Persistence tab.
    Coherence Cluster
    JVisualVM Persistence subtab
 

Creating and Recovering On Demand Snapshots

In the previous section we populated a cache using the Coherence query tool. In this section we will

  1. In right hand pane select the Oracle Coherence > Persistence tab.
  2. From the Persistence tab, select the PartionedCache Service, right click and select Create snapshot.
    This image shows the right pane in JVisualVM after a Coherence DefaultCacheServer was selected. The image specifically shows the selection of the Oracle Coherence tab and the Persistence sub-tab.
    Persistence subtab snapshot menu
  3. In the input dialog, enter the name "snapshot.test" and click OK.
    In the Please Confirm Operation dialog, click Yes.
    In the informational Result dialog, click OK.
    Note: To preserve data consistency Coherence will pause while the snapshot is written.
  4. Select the PartionedCache Service, right click and select List snapshots. Click OK.  The newly created snapshot should be listed.
    Coherence Cluster
    List of snapshots
  5. Return to the terminal window running the Coherence Query tool and delete the data in the test cache using a command similar to:
    CohQL>  delete from 'test'
  6. Confirm that the cache is now empty using a command similar to:
    CohQL>  select count() from 'test'
    Which should show a result:
    Results
    0
    CohQL>
  7. Return to JVisualVM and in the Persistence tab, Right click the Partitioned Cache Service and select Recover Snapshot.
  8. From the drop down list select the previously created snapshot and click OK. In the confirmation dialog click Yes. In the results dialog click OK.
  9. Return to the Query terminal and display the restored data using a command similar to:
    CohQL>  select count() from 'test'
    Which should show a result similar to:
    Results
    10
    CohQL>
  10. Exit the Coherence Query too using the byecommand.
    CohQL> bye
    $
  11. Using [ctrl][c] or the corner X close the terminal windows, shut down both instance of Coherence, and exit JVisualVM.
    Consider keeping these windows open for reuse in later tutorial sections.  However the instances themselves should be shut down as described above.

You have now successfully used the default Coherence persistence mechanisms to create and restore simple snapshots. In the next section you will explore using Coherence Active Persistence.

 

Configuring and Using Coherence Active Persistence

In this section we will build the provided examples and explore 'active' persistence. Active persistence differs from the default persistence in that when enabled all changes to data are persisted automatically. In addition to data changes, metadata on indices, triggers, listeners and locks is also persisted.

For more information on configuring active persistence see the Coherence documentation here.

In this section we will:
  1. Build the Coherence examples
  2. Configure and start cache servers enabled for ‘active’ persistence
  3. Simulate Failure in the cluster and observe Active Recovery
  4. Guard Against Potential Data Loss
You will again need four terminal windows:
  • Two terminals for cache servers
  • And one each for JVisualVM and the Coherence Query tool (CohQL)
 

Build Examples

There are two aspects to setting up active persistence:

  • Defining the persistence-environment in the operational override file
  • Associating a persistence-environment with a service
Note: Active persistence can be enabled for all services and caches by specifying the persistence-mode=active override for example:-Dcoherence.distributed.persistence-mode=active
The override uses the default persistence directories unless otherwise configured.

The Coherence Persistence examples  come preconfigured and ready for using ‘active-default’ persistence.
  1. Ensure that all cache servers from the prior section are shutdown.
    Also ensure that JVisualVM and the Coherence query tool are shut down
  2. Open a terminal window and change directory to the COHERENCE_HOME\examples\java directory
    $ cd %COHERENCE_HOME% 
    $ cd examples\java
  3. Examine the override and cache configuration files:
    1. Using a command similar to:
      $ type resource\persistence\tangosol-coherence-override.xml
      Search for the persistence-environments element
      Notice that element is wrapped in comments and resembles:
      <!--
         <persistence-environments>
         . . .
         </persistence-environments>
      -->
       
    2. Using a command similar to:
      $ type resource\persistence\examples-cache-config.xml
      Search for the persistenceelement for the ExamplePartionedPofScheme.
      Notice that the examples cache configuration uses the default-active persistence definition
       

      This image shows a portion of the resource\persistence\examples cache configuration file with the persistence section highlighted showing the use of the default-active persistence configuration.
      A portion of the examples cache configuration
  4. Build the examples using commands similar to:
    $ bin\build.cmd contacts
    $ bin\build.cmd persistence
    Wait for each command to complete then proceed to the next step.
  5. Start a cache server instance using a command similar to:
    $ cd %COHERENCE_HOME%\examples\java
    $
    bin\run-cache-server.cmd persistence
    Remember to ensure that the COHERENCE_HOME and JAVA_HOME variables are correctly set in each window.
  6. Open a second terminal window and start a second cache server instance.
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  7. Open a third window and start JVisualVM.
  8. Select either of the DefaultCacheServer instances and in the right pane select Oracle Coherence and then the Persistence subtab.
    Note: that topmost pane was expanded to show all services.
  9. Notice that the persistence mode of the Partitioned cache services is shown as active.
    This image shows JVisualVM with the Coherence > Persistence tab selected for a running instance of a DefaultCacheServer.  In the image the persistence mode for the various partitioned cache services is shown to be active.
    The Persistence Subtab of a Cache Server in JVisualVM
  10. Start the Coherence Query tool using a command similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run.cmd persistence cohql
  11. Insert data into a cache using commands similar to:
    CohQL> insert into 'test' key(1) value('value1')
    . . . 
    CohQL> insert into 'test' key(10) value('value10')
  12. Return to JVisualVM and examine the examine the Oracle Coherence > Caches tab. Note that the cache now contains entries.
    Examining the Oracle Coherence > Persistence > Storage and Latency Details tab will also show an increase in Total Active Space Used.
 

Simulate failure and Observe Active Recovery

  1. Kill all running cache servers and exit JVisualVM.
    Under Windows use the [x] in each terminal window. 
    Under Linux use a command similar to ps augx | grep DefaultCacheServer and then capture the process ids and use kill -9 %pid to kill each process. 
  2. Open two terminal window and start two cache server instances.
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  3. As each cache server starts you may notice messages indicating recovery is in progress.
    Statements such as those shown below represent Coherence recovering the cache:
    2016-08-05 12:29:31.983/37.098 Oracle Coherence GE 12.2.1.0.0 (thread=DistributedCache:PartitionedPofCache2, member=1): Recovering 257 partitions
    . . .
    2016-08-05 12:30:47.278/112.372 Oracle Coherence GE 12.2.1.0.0 (thread=DistributedCache:PartitionedPofCache2, member=1): Recovered PartitionSet{0..256}
    from active persistence
    . . .
  4. Restart JVisualVM, connect to a default cache server and examine the cache.
    Notice that the original objects have been restored.
  5.  Optionally, create and recover a snapshot of an active persistence cache using JVisualVM:
    1. Within JVisualVM select a DefaultCacheServer instance.
    2. Open the Oracle Coherence > Caches tab and note the name of the Service associated with the test cache, for example PartitionedPofCache.
    3. Open the Oracle Coherence> Persistence tab.
    4. Select the matching Partitioned cache row.
    5. Right click and select Create snapshot. Name and otherwise complete the snapshot.
    6. Using Coherence Query tool, drop all entries in a cache.
      For instance delete from 'test', where testrepresents the name of the originally created cache.
    7. Use a command such as select count() from 'test' to verify the cache contains no data.
    8. Returning to JVisualVM and using Recover snapshot to restore the snapshot.
    9. Use a CohQL command such as select key(),value() from 'test' to show that the cache data was reloaded.
  6. Leave both cache servers running but exit JVisualVM and the CohQL tool.

Notes:

  • Persistent data is stored under the directory COHERENCE_HOME/examples/java/persistence-data in four sub-directories.
    • active – Contains active persistence data.
    • snapshots – Contains snapshot data.
    • trash –Contains incomplete persistence data.
    • archives – Contains archiving snapshots (covered in the next section)
    • All of the above directories can be configured in the operational override file tangosol-coherence-override.xml. The archive directory must be a shared directory but the others may be local or shared.
    • In this tutorial section we are using the default-active environment and relying on setting the coherence.distributed.persistence.base.dir property (in function_set_env or set_env.cmd) to a specific directory rather than defaulting to users home
    • There are special considerations on multi-machine environment using local active directories.
      Covered in a later section on recover quorums.
 

Guarding Against Data Loss

By default, Coherence is configured with one backup, which means Coherence can suffer the loss of one instance without data loss. Where multiple machines are involved, Coherence can also suffer a single machine loss without data loss. Increasing the number of backups guards against data loss when multiple nodes or machines fail at the same time, however each additional backup adds latency for write operations. An alternative strategy is to use Active Persistence, with a shared active persistence directory. This alternative strategy prevents data loss from the simultaneous failure of multiple nodes or machines. In addition active persistence can be configured to use zero backups and not lose data.

In this section we will demonstrate active persistence using three cache servers, and simultaneously kill two of them and suffer no data loss.

  1. Open a third terminal window and start a Coherence server using commands similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  2. Start the Coherence Query tool using a command similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run.cmd persistence cohql
  3. At the CohQL command line examine the current test cache to determine its count and/or contents of objects using a command such as those shown below:
    CohQL> select count() from 'test'
    Results
    10
    CohCQL>
    or
    CohCL> select key(),value() from 'test'
    Results
    [1, "value1"]
    . . .
    [10, "value10"]
    CohQL>
  4. From another terminal window, list the running java processes and kill any two cache servers.
    $ jps
    7872 DefaultCacheServer
    5284 Jps
    6436 DefaultCacheServer
    6756 Driver
    7016 DefaultCacheServer
    
    or
     $ ps augx | grep [D]efaultCacheServer
    oracle      5335  0.0  0.0   3852   368 tty7     Ss+  May25   0:00 DefaultCacheServer
    oracle      5336  0.0  0.0   3852   368 tty2     Ss+  May25   0:00 DefaultCacheServer
    oracle      5337  0.0  0.0   3852   368 tty3     Ss+  May25   0:00 DefaultCacheServer
    Note: jps is part of the Java Developers Kit. The pscommand is built into the Linux OS.
  5. Using the Windows taskkill commands stop two of the Coherence cache servers.
    $ taskkill /PID 7872 /PID 6436 /F
    Or the Linux kill -9
    $ kill -9 5335 5336
  6. Examine the remaining cache server output. Search for recovering active partition messages.
    For example:
    2016-08-08 11:17:34.615/254843.115 Oracle Coherence GE 12.2.1.0.0  (thread=DistributedCache:PartitionedPofCache, member=1): Recovering 86 partitions
    2016-08-08 11:18:03.455/254871.956 Oracle Coherence GE 12.2.1.0.0  (thread=DistributedCache:PartitionedPofCache2, member=1): Recovered PartitionSet{0, 42..2, 128..171} from active persistence
  7. Return to the Coherence Query tool. Examine the cache contents. Does the cache still contain the expected count and content of objects?
    CohQL> select key(),value() from 'test'
    Results
    [1, "value1"]
    . . .
    [10, "value10"]
    CohQL>
  8. Using [ctrl][c] stop any running instances of the Coherence Query tool and any running cache servers.
    Consider keeping these windows open for reuse in later tutorial sections.
  9. Optional: As an additional exercise consider:
    1. Setting the backup count in resource\persistence\examples-cache-config.xml to zero.
    2. Start 3 cache server (as described above).
    3. Start a Coherence Query tool instance (as described above)and add data.
    4. Kill two of the three servers.
    5. Question: Does the result without backup count of zero match the prior result with backup count one?
 

Archiving and Validating Snapshots

Snapshots can be current, or archived.  Archived snapshots can be saved, retrieved and restored. Both active and archived snapshots can be validated to determine if they are corrupt.
The objective of this section is to exercise archiving and validating snapshots.
In this section you will:

  1. Populate the contacts cache with data.
  2. Create and validate the snapshot.
  3. Archive the snapshot and validate the archived snapshot.
  4. Retrieve and recover the archived snapshot.
 

Populating the Contacts Cache

To populate the Contacts cache with entries:

  1. Open two terminal window and start two Coherence server using commands similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  2. Open a new terminal window and populate the cache with data using commands similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run.cmd persistence populate
    The populate operation will add 10,000 entries to the contacts cache
  3. Start JVisualVM and select one of the DefaultCacheServer instances
  4. In the right pane select the Oracle Coherence > Caches tab.
  5. Examine the service/Cache name instances, one of which should show a 10,000 entries.
    This image shows JVisualVM with the Coherence > Cache tab selected for a running instance of a DefaultCacheServer.  In the image the PartitionedPofCache cache is selected and shows a size of 10,000 entries.
    The Cache Subtab of a Cache Server in JVisualVM
 

Creating and validating snapshots

Create and validate a snapshot:

  1. In JVisualVM navigate to the Persistence subtab.
  2. Select the cache with the larger active space used.
    Note:You may need to expand the services portion of the Caches tab.
  3. Right click and select Create Snapshot. Complete the wizard to finish creating the snapshot.
    Alternatively you could use the create snapshot command in Coherence Query tool. See the Coherence Query Tool documentation for more information,
  4. Start the Coherence Query tool using a command similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run.cmd persistence cohql
  5. Using a command similar to the one shown below, list all snapshots.  Note that two snapshots are listed.
    CohQL> list snapshots
    Results
    "PartitionedPofCache": ["contacts", "test-snapshot"]
    "PartitionedPofCache2": []      
  6. Validate the snapshot using a command similar to:
    CohQL> validate snapshot 'contacts' 'PartitionedPofCache'
    Validating snapshot directory 'c:\. . .\contacts'
    Results
    "Success"
  7. Perform a verbose validation using a command similar to
    CohQL> validate snapshot 'contacts' 'PartitionedPofCache' verbose
    Validating snapshot directory 'c:\. . . \contacts'
    Results
    Attribute                    Value
    ---------------------------- ---------------------------------------------------
    Partition Count              257
    Directory                    c:\. . .\PartitionedPofCache\contacts
    Storage Format               BDB
    Storage Version              0
    Implementation Version       0
    Number of Partitions Present 257
    Is Complete?                 true
    Is Archived Snapshot?        false
    Service Version              14
    Statistics                   contacts
    Size=10000, Bytes=1627109, Indexes=0, Triggers=0, Listeners=0, Locks=0 test Size=10, Bytes=180, Indexes=0, Triggers=0, Listeners=0, Locks=0
 

Archiving and validating snapshots

An archived snapshot is a full copy of all snapshot parts, from all members, stored in a central location and in a storage-independent format.  To archive and validate a snapshot:

  1. Using JVisualVM archive the snapshot by:
    1. Selecting the Persistence subtab
    2. Select the cache with the larger active space used.
      Note:You may need to expand the services portion of the Caches tab.
    3. Right click and select Archive Snapshot.
      Select the contacts snapshot.
      Complete the wizard to finalize the archive process.Complete the wizard to finish creating the snapshot.
      Alternatively, you could use the archive snapshot command in the Coherence Query Tool.
      For example:
      CohQL> archive snapshot 'contacts' 'PartitionedPofCache'
      Are you sure you want to archive a snapshot called 'contacts' for service 'Parti
      tionedPofCache'? (y/n): y
      Archiving snapshot 'contacts' for service 'PartitionedPofCache'
      Results
      "Success"
  2. Snapshots are contained in the directory defined by the <snapshot-archivers> element of the tangosol-coherence-override.xml file. Which for this tutorial is COHERENCE_HOME/examples/java under the persistence-data/archives directory.
    Examine the COHERENCE_HOME/examples/java/persistence-data/archives directory to find snapshot data.
    Snapshot data is typically broken down into individual files based on the number of partitions specified for a cache which defaults to 257.  How many snapshot files appear to exist for the contacts snapshot?
 

Retrieving and recovering archived snapshots

In this section we will use the Coherence Query Tool to work with snapshots. All the operations shown can also be completed within JVisualVM.   Note: When retrieving an archived snapshot, the local snapshot should first be removed.
You may also use the OVERWRITE option of the RETRIEVE SNAPSHOT command to to replace an in existing snapshot.

To retrieve and recover a snapshot:

  1. Return to or open the Coherence Query Tool in a terminal window.
    Use command similar to those below if you need to start an instance of the Coherence Query Tool.
    CohQL>  remove snapshot 'contacts' 'PartitionedPofCache' 
    
    When asked for confirmation answer 'y'.
  2. Delete the the existing snapshot
    CohQL> retrieve archived snapshot 'contacts' 'PartitionedPofCache' 
    
    When asked for confirmation answer 'y'.
  3. Drop all results in the contacts cache:
    CohQL> delete from 'contacts'
    Results
    CohQL> select count() from 'contacts'
    Results
    0
  4. Recover the snapshot:
      CohQL> recover snapshot 'contacts' 'PartitionedPofCache' 
    When prompted enter 'y' to confirm.
    Note if the recover snapshot command returns the error Error: (Wrapped: Unable to complete operation recoverSnapshot for service PartitionedPofCache) sleep interrupted the operation will continue in the background.
    Use the command list services to show the status of the PartitionedPofCache service.  When the service shows idle continue with the instructions below. 
  5. Examine the content of the cache to verify the recover operation restored data:
    CohQL> select count() from 'contacts'
    Results
    100000
  6. Leave all processes running as they will be used in the next section.
 

Monitoring Persistence

The objective of this section is to understand how to manage and monitor a Persistence Environment.

In this section you will:

  1. Identify the PersistenceCoordinator MBean
  2. Work with JMX Notifications in JVisualVM
  3. Run the NotificationWatcher example
  4. View Service MBean attributes

Those interested in more information about Coherence and JMX monitoring are encouraged to review the documentation here.
The complete reference for Oracle Coherence MBeans can be found here.

 

Identify the Persistence Coordinator MBean

The PersistenceCoordinator MBean is a Coherence “Responsibility MBean”. What this means there will only ever be a single instance of this MBean. The singleton instance is responsible for coordinating the Persistence operations within the cluster.

To find the Persistence Coordinator MBean:

  1. If required, Start  JVisualVM and select one of the running DefaultCacheServer instances.
  2. In the right pane select the MBeans tab.
    Note: If the MBeans tab is not displayed review the section on installing the Coherence and MBeans JVisualVM plugins.
    Note:Only one of the DefaultCacheServer instance is configured for JMX. If you see the error
    Data not available because JMX connection to the JMX agent could not be established continue with the other instance.
  3. Navigate to Coherence > Persistence > PartionedPofCache > PersistenceCoordinator.
    This image shows JVisualVM with a Coherence default cache server selected. In the right pane the MBean browser tab is selected and open to Coherence > Persistence > PartitonedPofCache folder. Within the PartionedPofCache folder the PersistenceCooridator MBean is selected.
    The MBeans browser tabCache Server in JVisualVM with the Persistence Coordinator MBean selected
  4. Examine the attributes of the Persistence Coordinator MBean which include:
    Field Description
    CoordinatorID The Member Id of the current coordinator
    Idle Coordinator is idle flag
    Operation Status The current operation being executed, or idle if no operation in progress.
    Snapshots A string array of know snapshots for this service.
  5. Select the Operations tab. The operation tab shows the operations that are callable against the PersistenceCoordinator MBean. These are primitive operations that both the Coherence Query tool and the Coherence JVisualVM plug execute.
    This image shows JVisualVM with a Coherence default cache server selected. In the right pane the MBean browser tab is selected and open to Coherence > Persistence > PartitonedPofCache folder. Within the PartionedPofCache folder the PersistenceCooridator MBean is selected and the Operations tab is displayed.
    The MBeans browser tabCache Server in JVisualVM with the Persistence Coordinator MBean selected
  6. Using the createSnapshot operation, create a new snapshot. Enter a name, for example new snapshot and click the createSnapshot button:
    This image shows the right pane of JVisualVM with the MBean browser tab is selected and open to Coherence > Persistence > PartitonedPofCache folder with the PersistenceCoordinator been selected. Specifically this image shows the Operations sub-tab and the use of teh createSnapshot operation to create a snapshot named "new snapshot".
    Creating a new snapshot
  7. Return to the Coherence Query tool, or use the list snapshots operation in the JVisualVM Persistence tab to list all known snapshots.
    Does the newly created snapshot exist?
    CohQL> list snapshots
    Results
    "PartitionedPofCache": ["contacts", "new-snapshot"]
    "PartitionedPofCache2": ["test"]
    
 

Explore JMX Notifications with JVisualVM

JMX Clients can subscribe to Persistence JMX notifications to obtain information about operations that are executed for a particular service. Notifications for operations exist for start, stop, and a variety of failures. In this section we will examine JMX notifications using JVisualVM. 

To enable JMX notifications:

  1. Start or return to JVisualVM and select a DefaultCacheServer instance. You may reuse an existing instance if one is already open.
  2. In the MBeans browser tab navigate to Coherence > Persistence > PartionedPofCache > PersistenceCoordinator.
  3. Select the Notifications tab.
    Note that the Collect Notification checkbox is checked by default.   When checked, Oracle Coherence will subscribe to notifications and calculate operation duration based on begin/end times.
    Note that you may need to expand the window to see the Collect Notifications check box and Clear Notifications button.
    This image shows the notifications tab of the Oracle Coherence plugin. Specifically the Collection Notifcations 
check box (which is enabled by default) and the Clear Notifications buttons are high lighted.
    Notifications tab highlighting Collection Notifications
  4. Click the Subscribe button. (you may need to scroll to the bottom of the window).
  5. Perform several operation on a snapshot. For example, remove, recreate, archive the contacts snapshot:
    CohQL> list snapshots
    CohQL> remove snapshot 'contacts' 'PartitionedPofCache'
    Are you sure you want to remove snapshot called 'contacts' for service 'PartitionedPofCache'? (y/n): y
    CohQL> create snapshot 'contacts' 'PartitionedPofCache'
    ...
    CohQL> remove archived snapshot 'contacts' 'PartitionedPofCache'
    ...
    CohQL> archive snapshot 'contacts' 'PartitionedPofCache'
    ...
    CohQL>
  6. Return to the Notifications tab and examine the result of the prior operations.  You should see something resembling that below:
    This image shows the notifications tab of the Oracle Coherence plugin when the PersistenceCoordinator MBean was selected. THe notifications tab show the results of performing four operation, remove snapshot, create snapshot, remove archived snapshot and archive snapshot after having clicked the subscribe button.
    Notifications tab

    Note:Snapshot operations typically generate two sets of notifications: The begin/end pairing signifies the beginning and end of the entire snapshot operation and is used to calculate operation duration.
  7. Select the Oracle Coherence > Persistence tab.
    You can clear existing notifications, or right click the table and save as a CVS file if desired.

The following section reuses the existing Coherence instances and you can leave them running.

 

Exercising the NotificationWatcher Example

A notifications example is provided which details how to programmatically determine the durations of operations.
To run this notifications example:

  1. Open a terminal window and start the notifications example using commands similar to:
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run.cmd persistence notifications
    Which will produce output resembling:
    redirecting Coherence logging to persistence.log...
    . . . 
    ------Getting MBeanServer...------
    ------Retrieving Registry...------
    ------Registering listener for PartitionedPofCache------
    ------Registering listener for PartitionedPofCache2------
    ------Waiting for notifications. CTRL-C to interrupt.------
    
  2. Return to the Coherence Query tool and using commands similar to those shown perform some snapshot operations.
    Note:Confirmation requests removed for clarity.
    CohQL> list snapshots
    CohQL> remove snapshot 'contacts' 'PartitionedPofCache'
    CohQL> create snapshot 'contacts' 'PartitionedPofCache'
    CohQL> remove archived snapshot 'contacts' 'PartitionedPofCache'
    CohQL> archive snapshot 'contacts' 'PartitionedPofCache'
    CohQL>
  3. Return to the terminal where the notifications application is running. 
    Examine the output which should show notifications being generated for each of the operations, one each for begin and end. 
    Thu Aug 11 10:36:52 EDT 2016 : PartitionedPofCache (remove.snapshot.begin) Removing snapshot "contacts"
    Thu Aug 11 10:36:57 EDT 2016 : PartitionedPofCache (remove.snapshot.end)   Successfully removed snapshot "contacts" (Duration=5143ms)
    Thu Aug 11 10:37:08 EDT 2016 : PartitionedPofCache (create.snapshot.begin) Building snapshot "contacts"
    Thu Aug 11 10:37:50 EDT 2016 : PartitionedPofCache (create.snapshot.end)   Successfully created snapshot "contacts" (Duration=41933ms)
    Thu Aug 11 10:39:40 EDT 2016 : PartitionedPofCache (remove.archived.snapshot.begin) Removing archived snapshot "contacts"
    Thu Aug 11 10:39:41 EDT 2016 : PartitionedPofCache (remove.archived.snapshot.end)   Successfully removed archived snapshot "contacts" (Duration=539ms)
    Thu Aug 11 10:39:51 EDT 2016 : PartitionedPofCache (archive.snapshot.begin) Archiving snapshot "contacts"
    Thu Aug 11 10:39:58 EDT 2016 : PartitionedPofCache (archive.snapshot.end)   Successfully archived snapshot "contacts" (Duration=6776ms)
  4. Return to the terminal window where the notifications application example is running and stop it using [ctrl][c].

Learners interested in how the notifications examine registers for notifications are directed to the NotificationWatcher.javaexample found in: COHERENCE_HOME\examples\java\com\tangosol\examples\persistence\NotificationWatcher.java.

 

Exploring Service MBean Attributes

The information displayed on the Persistence tab in JVisualVM is obtained from a number of sources including:

  • PersitenceCoordinator MBean
  • ServiceMBean values aggregated for all Coherence instances
In this section you will examine these areas and explore the data provided by JVisualVM
  1. Return to or restart JVisualVM
  2. Select a DefaultCacheServer instance.
  3. In the right pane  select the MBeans tab.
    Reminder: Only one of the DefaultCacheServer instance is configured for JMX. If you see the error
    Data not available because JMX connection to the JMX agent could not be established continue with the other instance.
  4. Navigate to the Coherence > Service > PartitionedPofCache and select instance 1"
    This image shows MBean browser path of a DefaultCacheServer to the Service MBean under the PartionedPofCache service for the first Coherence instance.
    Service MBean for the first instance under PartiotionedPofCache

  5. In the right pane examine the attributes tab of the Service MBean. Persistence attribute are prefixed with Persistence. Not all are shown.
    Given the image, can you answer determine the persistence mode for this instance and service combination?
    Service MBean Attributes

    All the attributes are explained in the Coherence documentation, several outlined below, the complete reference for Oracle Coherence MBeans can be found here
    • PersistenceActiveSpaceUsed – The total space used be active persistence for this node
    • PersistenceEnvironment – the details of the active and snapshot directories
    • PersistenceLatencyAverage – the average latency added by Persistence
    • PersistenceLatecyMax – the maximum latency added by Persistence
    • PersistenceSnapshotArchiver – the snapshot archiver used by the service
  6. Shutdown any cache servers, Coherence Query tool instances and JVisualVM


 

Using Recover Quorum

The partitioned cache recover quorum is used to define the number of cluster storage members that must be available before persistence recovery starts. The use of the quorum allows time for a cluster to start and ensures that partitions can be recovered gracefully without overloading too few storage members or without inadvertently deleting orphaned partitions.

The objective of this section is to show how to use the recover quorum to control when services begin active recovery.
In this section you will:

  1. Enable the recover-quorum
  2. Execute he NotificationWatcher example and view active recovery in progress
 

Enabling Recover Quorum for Services

The recover-quorum controls the number of members that must be present before recovery can begin. This is important for a number of reasons:

  1. When using ‘active’ mode, there must be enough service members running to be able to recover the data without running out of memory.
  2. If the recovery-hosts parameter is not set correctly then there may be partitions that are not recovered because not all persisted data is not available. The recovery-hosts parameter is shown below:
    <partitioned-quorum-policy-scheme>
       <recover-quorum>2 </recover-quorum>
      <recovery-hosts></recovery-hosts>
    </partitioned-quorum-policy-scheme>

Since only a single machine is running all instances the recover-hosts element is left empty.

To configure Recover Quorum:

  1. Ensure that all Coherence instances, Coherence Query Tool instances and JVisualVM are shut down.
  2. Using notepad or gedit or a similar editor, update the file COHERENCE_HOME/examples/java/resource/persistence/examples-cache-config.xmland un-comment out the following lines: (Note: There is two entries, ExamplesPartitionedPofScheme and ExamplesPartitionedPofScheme2)

    $ cd %COHERENCE_HOME%\examples\java
    $ notepad resource\persistence\examples-cache-config.xml 
    Which will resemble:
    . . . 
    <distributed-scheme>
    <scheme-name>ExamplesPartitionedPofScheme</scheme-name>
        . . .
    </persistence>
          <!-- <partitioned-quorum-policy-scheme>
    <recover-quorum>2</recover-quorum>
    </partitioned-quorum-policy-scheme>
    -->

    <autostart>true</autostart>
    . . .        </distributed-scheme>
  3. In a new terminal window or a window previously opened start a single Coherence instance.
    Note:Consider redirecting the output of the run-cache-server.cmd command to a file to more easily search for specific output.
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  4. Examine the output of the window searching for disallow messages.
    For example:
    2016-08-12 10:57:00.534/19.401 Oracle Coherence GE 12.2.1.0.0 <warning> (thread=DistributedCache:PartitionedPofCache, member=1): Action recover disallowed; all-disallowed-actions: recover(2)
    2016-08-12 10:57:00.544/19.408 Oracle Coherence GE 12.2.1.0.0 <warning> (thread=DistributedCache:PartitionedPofCache2, member=1): Action recover disallowed; all-disallowed-actions: recover(2)
    
    Assuming the run command was redirected to quorum.log under Window use a search command similar to:
    $ type quorum.log | findstr /C:"disallow"
    Under Linux use a command similar to:
    $ grep disallow quorum.log
  5. Open a second terminal window and start a second instance of Coherence
    $ cd %COHERENCE_HOME%\examples\java 
    $ bin\run-cache-server.cmd persistence
  6. Examine the output of either instance and you will see messages indicating recovery in progress.
    2016-08-12 11:05:39.531/46.461 Oracle Coherence GE 12.2.1.0.0  (thread=DistributedCache:PartitionedPofCache, member=2): Recovering 128 partitions
    2016-08-12 11:05:44.255/51.184 Oracle Coherence GE 12.2.1.0.0  (thread=DistributedCache:PartitionedPofCache2, member=2): Recovering 128 partitions
  7. When recover is complete an all actions allowed message will be generated similar to:
    2016-08-12 11:05:12.935/511.789 Oracle Coherence GE 12.2.1.0.0 <Warning> (thread=DistributedCache:PartitionedPofCache, member=1): All actions allowed
    2016-08-12 11:05:13.600/512.453 Oracle Coherence GE 12.2.1.0.0 <Warning> (thread=DistributedCache:PartitionedPofCache2, member=1): All actions allowed
    
 

Running the notification watcher and observing active recovery

The notification example can be used to watch for quorum based recovery notifications. In order to be notified of such operations, the notification recipient must be registered before the notifications are sent. 

To use the notfications example to watch for recovery notifications perform the following steps.

  1. Shutdown/stop all running instances of Coherence and Coherence clients.
  2. In one of the existing Coherence terminal windows execute:
    $ bin\run-cache-server.cmd persistence
  3. In a second Coherence terminal window start the notifications example:
    $ bin\run.cmd persistence notifications
    When the window displays ------Waiting for notifications. CTRL-C to interrupt.------ continue to the next step.
  4. In a third Coherence terminal window execute:
    $ bin\run-cache-server.cmd persistence
  5. Return to the window where the notifications example is running and you should see messages similar to:
    Fri Aug 12 11:22:58 EDT 2016 : PartitionedPofCache (recover.begin) Active recovery in progress
    Fri Aug 12 11:22:59 EDT 2016 : PartitionedPofCache2 (recover.begin) Active recovery in progress
    Fri Aug 12 11:23:38 EDT 2016 : PartitionedPofCache2 (recover.end) Recovery Completed (Duration=38799ms)
    Fri Aug 12 11:23:38 EDT 2016 : PartitionedPofCache2 (recover.begin) Active recovery in progress
    Fri Aug 12 11:23:41 EDT 2016 : PartitionedPofCache (recover.end)  Recovery Completed (Duration=42924ms)
    Fri Aug 12 11:23:41 EDT 2016 : PartitionedPofCache2 (recover.end)  Recovery Completed (Duration=3451ms)
    Fri Aug 12 11:23:42 EDT 2016 : PartitionedPofCache (recover.begin) Active recovery in progress
    Fri Aug 12 11:23:42 EDT 2016 : PartitionedPofCache (recover.end)  Recovery Completed (Duration=512ms)
    You may notice that there are multiple begin/end notifications for the recovery. This occurs because recovery may require one or more passes to recover all data.
  6. Leave the two instances of Coherence running, they will be reused in the next section.

 

Using CohQL

The objective of this section is to provide more detail on how to use CohQL to carry out Persistence operations. In this section you will:

  • CohQL commands from a file.
  • Responding to errors.
  • Setting timeout for CohQL commands.
 

Running CohQL Commands from Files

To use Coherence Query tool, you can use the query[.sh|.cmd] files located in the coherence/bin directory. The examples come with a script to run CohQL, which is the equivalent to running query.sh. To run commands from a file you specify the -f option followed by the file name to execute. You can also use the -c option which will exit CohQL once the commands have executed.

  1. Ensure that you have two Coherence cache servers running.
  2. Create a file example.cohql which contains the following commands, notice the semi-colon ";" at the end of each line. For this tutorial the script name is assumed to be example.cohql.
    list services;
    create snapshot 'test' 'PartitionedPofCache';
    list snapshots;
    recover snapshot 'test' 'PartitionedPofCache';
    remove snapshot 'test' 'PartitionedPofCache';
    
  3. Run the Coherence Query tool, but using the newly created command file using a command similar to that shown below:
    $ bin\run persistence cohql -f example.cohql
    Note that if the file is not in the same directory as where the command was run, enter the fully qualified path name.
    Which should produce results similar to:
    . . . 
    redirecting Coherence logging to persistence.log...
    Results
    Service Name         Mode   Quorum Policy                                                       Current
    -------------------- ------ ------------------------------------------------------------------- -------
    PartitionedPofCache  active allowed-actions=distribution,restore,recover,cache-read,cache-write Idle
    PartitionedPofCache2 active allowed-actions=distribution,restore,recover,cache-read,cache-write Idle
    Creating snapshot 'test' for service 'PartitionedPofCache'
    Results
    "Success"
    Results
    "PartitionedPofCache": ["contacts", "test"]
    "PartitionedPofCache2": ["test"]
    Recovering snapshot 'test' for service 'PartitionedPofCache'
    Results
    "Success"
    Removing snapshot 'test' for service 'PartitionedPofCache'
    Results
    "Success"
    Coherence Command Line Tool
    CohQL>
  4. Using the byecommand exit CohQL.
 

Responding to Errors

CohQL supports controlling error behavior using:
WHENEVER COHQLERROR THEN (CONTINUE | EXIT);
When specified CohQL will either ignore the error and CONTINUE or EXIT without processing additional commands.
EXIT is useful when you want to validate a snapshot or archived snapshot before you archive or retrieve. 

To add error support to your script:

  1. Open the previously created script in an editor.
  2. Add an exit command command on error and a suffix to the create snapshot command which should result in an error and the termination of the script.
    The updated script should resemble:

    whenever cohqlerror then exit;
    list services;
    create snapshot 'test' 'PartitionedPofCacheXYZ';
    list snapshots;
    recover snapshot 'test' 'PartitionedPofCache';
    remove snapshot 'test' 'PartitionedPofCache'
    
  3. In the Coherence Query Tool window execute the following.  Note the addition of -c which will cause the query tool to execute its commands and then exit.
    $ bin\run persistence cohql -f example.cohql
    Which should produce results similar those below.
    redirecting Coherence logging to persistence.log...
    Results
    Service Name         Mode   Quorum Policy                                                       Current Operation
    -------------------- ------ ------------------------------------------------------------------- -----------------
    PartitionedPofCache  active allowed-actions=distribution,restore,recover,cache-read,cache-write Idle
    PartitionedPofCache2 active allowed-actions=distribution,restore,recover,cache-read,cache-write Idle
    Service 'PartitionedPofCacheXYZ' does not exist
 

Setting CohQL Timeounds

The default CohQL command timeout is 60s. Many commands, such as archiving or validating a snapshot, can take considerably longer to complete. To specify a longer timeout use the alter session command.

For example:
ALTER SESSION SET TIMEOUT [timeout in MS]

To explore timeouts:

  1. Start the Coherence Query tool using a command similar to:
     $ bin\run persistence cohql 

  2. Specify a very short timeout and then executing a long query:
    CohQL> alter session set timeout 300000
    
    Which should result in:
    Results
    "CohQL statement timeout set to 5m"
    
  3. Reset the timeout to 10ms:
    CohQL> alter session set timeout 10
    
    Which should result in:
    Results
    "CohQL statement timeout set to 10ms"
    
  4. Rerun the select statement:
    CohQL> select count() from "contacts"
    Which should show results similar to:
    Error: (Wrapped) null
  5. Optionally, reset the session timeout to 60 seconds and re-run the query.
    CohQL> alter session set timeout 60000
    Results
    "CohQL statement timeout set to 1m"
    CohQL> select count() from "contacts"
    Results
    10000
    
  6. Exit CohQL and the two Coherence cache instances running, they will be used in the next section.
 

Restoring and Recovering Indices and Listeners

The objective of this section is to showcase metadata recovery and custom FTP archiving. In this section you will:

  • Explore recovering listeners
  • Explore recovering indices
 

Recovering Cache Metadata

In the past sections we have seen how data is recovered when using active mode. In this section we will show how the following can be recovered:

  • Listeners
  • Index definitions
  • Locks
  • Triggers
To demonstrate how listeners are persisted and recovered:
  1. Ensure that you have two instances of Coherence running. If need be, start the instances using commands similar to:

    $ cd COHERENCE_HOME\examples\java
    $ bin\run-cache-server.cmd persistence
    
  2. Return to CohQL and delete all data from the contacts cache using a command similar to:
    CohQL> delete from "contacts"
    
  3. Start JVisualVM and select one of the DefaultCacheServer instances.
  4. In the right pane select the Oracle Coherence > Caches tab.
  5. Within the Caches tab select the contacts PartionedPofCache/Contacts row and then the Cache Storage Details sub-tab for that how. Notice that the Listener Registrations count is zero.
    This image shows jVisualVM with a Coherence Default Cache Server Selected. In the right pane the Oracle Coherence tab was selected, then the caches subtab. The contacts cache was then select and its Cache Storage details pane of the contacts cache.  The storage details sub-tab shows zero listener registrations.
    Contacts Cache Storage Details
  6. Open a new terminal window and run the following command to add a listener to the contacts cache.
    $ cd COHERENCE_HOME\examples\java
    $ bin\run.cmd persistence add-listener
    The following will be displayed.
    ------ObserverExample begins------
    ------ContactChangleListener added------
    Press RETURN to continue
    Do NOT press RETURN but rather let the listener application continue to run.
  7. Return to JVisualVM. Notice that there are now two listeners registered, one per instance.
    This image shows jVisualVM Cache Storage details sub-pane of the contacts cache with two listeners registered, one per instance.
    Contacts Cache Storage Details with listeners registered
  8. Run the following command to add content to the contacts cache.
    $ cd COHERENCE_HOME\examples\java
    $ bin\run.cmd persistence populate 1
  9. While leaving the listener process running, shut down both of the Coherence cache servers and JVisualVM.
  10. Restart both Cache server instances.
  11. Start JVisuaVM and re-connect to a DefaultCacheServer instance.
  12. In the right pane select the Oracle Coherence > Caches tab.
  13. Within the Caches tab select the contacts PartionedPofCache/Contacts row and then the Cache Storage Details sub-tab for that how. Notice that the Listener Registrations count is greater then zero, showing a listener was restored.
  14. In a terminal window rerun the populate command
    $ cd COHERENCE_HOME\examples\java
    $ bin\run.cmd persistence populate 1
  15. Examine the window running the listener notice that even though the two cache servers were terminated the listener was recovered.
    entry updated
    old value:
    John Smith-0
    Addresses
    . . . 
    
    new value:. . . 
  16. Press return in the listener window to exit the listener.

To demonstrate how index definitions are recovered, complete the following:
  1. Ensure you have 2 cache servers started and JvisualVM running.
  2. In JVisualVM select a DefaultCacheServer instance and then the MBeans tab.
  3. Navigate to Coherence > StorageManager > PartitionedPofCache > contacts > any instance
  4. In the right pan examine the indexInfo attribute, note that it shows 0 entries.
  5. Use CohQL to add an index to the contacts cache using a command similar to:
     CohQL> create index on 'contacts' lastName  
  6. Return to JVisualVM and double click the value next to the indexInfo attribute is should now some something similar to the image below.
    This image shows the indexInfo attribute of the the current instances of PartitionedPofCache after adding an index.
    IndexInfo attribute populated with an index
  7. Exit JVisualVM and shut down both cache server instances.
  8. Restart both instances and JVisualVM.
  9. In JVisualVM select a DefaultCacheServer instance and then the MBeans tab.
  10. Navigate to Coherence > StorageManager > PartitionedPofCache > contacts > any instance.
  11. In the right pan examine the indexInfo attribute, note that it shows an entry which was restored.
  12. Shut down the JVisual VM, the Coherence cache instances and any other terminal windows.
 

Congradulations you have completed the Coherence tutorial on Persistence

 

Want to Learn More?