Using Coherence Portable Object Format Annotations

Overview

    Purpose

    Serialization, or marshalling,  is the process of encoding an object into a binary format. It is a critical component of working with Oracle Coherence.


    The Portable Object Format, or POF, is a language-agnostic binary format designed to be efficient in both space and time and has become a cornerstone of Oracle Coherence. Oracle Coherence 3.7 introduced the concept of POF annotations, enabling simple objects to support POF serialization without complex coding.

    Time to Complete

    Approximately 45 minutes 

    Introduction

    One of the more useful aspects of Coherence is its support for simple interoperability between various platforms.  While most of the code accessing Coherence is cluster-side and written in Java, Coherence clients can be written in Java, .NET, or C++.  There are no special constraints on the client. Instead, code is written in the language of choice, compiled against a client library, and then run. 

    What makes this possible are two things: Coherence*Extend and a specialized version of serialization known as Coherence Portable Object Format (POF). POF is a platform-independent mechanism for serializing objects into and out of a binary format. An object is marshalled into binary using POF, stored in a Coherence cache, and then unmarshalled to whatever format is appropriate for the client application.

    In general, you should choose POF instead of standard Java serialization because POF is:

    • Language independent: POF supports Java, C#, and C++.
    • Small: POF objects are often 1/5 the size of objects serialized with Java.
    • Fast: POF objects can be serialized in less than 1/4 the time of their Java counterparts.
    • Versionable: POF objects can evolve over time but maintain backward compatibility.
    • Controllable: Developers can control how POF objects are serialized and deserialized.

    When working with POF, a developer typically extends a domain object to implement the com.tangosol.io.pof.PortableObject interface. The readExternal and writeExternal methods of this interface then do the work of marshalling and unmarshalling data into and out of a POF data stream. Although it is not exceedingly complex, implementing these interfaces had a variety of advantages and disadvantages. Coherence 3.7 provides POF annotations, a new feature that can be used to simplify the POF process.

    This tutorial covers the use of Coherence 3.7 POF annotations to extend and register objects that support serialization.

    Hardware and Software Requirements

    • Coherence 3.7.1, which can be downloaded here
    • Coherence POF OBE sample code, which can be downloaded here

    Prerequisites

    Before starting this tutorial, you should have: 

    • Basic working knowledge of Linux commands and command-line syntax
    • Coherence 3.7.1 installed on your machine
    • The OBE sample code downloaded and unpacked. This code is designed to run on Linux and similar operating systems but can be adapted to Windows if required.
    • Oracle Java JDK version 1.6.0_26 (or later) downloaded and installed. The Oracle JDK kits and associated products can be found here.
      Note: The installation of Java is outside the scope of this OBE. For more information, see the JDK download and installation pages. 
    • A working knowledge of Java annotations

    Before running this tutorial, you should download and install Oracle Coherence 3.7.1 and the associated example source code. The following steps review the installation process of Coherence 3.7.1, as well as the expected installation locations. 

    Note: You can skip this step if you have already installed Coherence 3.7.1.

      Click the link under "Hardware and Software Requirements," accept the license agreement, and download Coherence 3.7.1. Note that, for this tutorial, it is assumed that files are downloaded to/tmp/.  and installed into /opt.

      Open a command prompt and change directory to the location where Oracle Coherence will be installed.

      Unzip the contents of the Coherence zip file by using a command that is similar to:

      unzip /tmp/coherence-java-3.7.1...zip 

      Note that the exact name of the file may vary.

      For this tutorial, all files will be unzipped to /opt/coherence


      Unzip the contents of the OBE example code by using a command that is similar to:
      unzip /tmp/POF.OBE.src.zip


Modify Classes to Support Serialization Using Portable Object Format Annotations

    Annotations

    Coherence POF is based on the concept of an annotation. Annotations are:


    Here is an example:

    Definition
         public interface Animal { String type; }
    Use
         @Animal(type="mammal") public class Bird {. . .}

    Entity classes are annotated to support POF using the Portable and PortableProperty annotations, which are found in the com.tangosol.io.pof.annotation package.

    Portable is defined as:
      package com.tangosol.io.pof.annotation;
      @Target(value=TYPE)
      @Retention(value=RUNTIME)
      public interface Portable extends Annotation {}

    It is applied to types (@Target (value = TYPE) as a whole, indicating that they should be serialized using POF. 
    For more information about the Portable annotation, click here.

    PortableProperty is defined as:
      package com.tangosol.io.pof.annotation;
      @Target(value={FIELD,METHOD})
      @Retention(value=RUNTIME)
      public interface PortableProperty extends Annotation {
          public abstract int value();
          public abstract Class codec();
      }

    PortableProperty is applied to fields or methods, and takes value or codec attributes. The value attribute indicates which index value is used to identify the field.
    For more information about the PortableProperty annotation, click here.

    Configure the Environment

      A variety of scripts have been provided to simplify the testing of POF. These scripts depend on certain environment variable settings. Appropriate defaults have been provided for these variables; the defaults do not need to be changed if they are correct.

      The scripts assume the following:
      • Coherence has been installed into /opt/coherence.
      • The OBE source has been unzipped to /opt/POF.OBE.

      If either of these values is different, perform the following steps:

      In a command prompt window, ensure that you are in the directory where the OBE source was unpacked.

      cd /opt/{obe source directory}

      In a text editor, open the bin/set-env.sh script.

      gedit bin/set-env.sh

      Near the top are two export statements specifying the locations of both the OBE source and Coherence.
      Uncomment the two variables and set them to the appropriate values for your environment.

      Note: If the default values /opt/coherence and /opt/POF.OBE are used, no changes are required.

      #!/bin/bash

      export OBE_HOME=/opt/my.pof.obe.location
      export COH_HOME=/opt/my.coherence.location

      Save your changes and exit the editor

    Test the Out-of-the-Box Classes for Size and Serialization Performance


      This tutorial provides the following two sample entity object classes that are used as examples of serialization and deserialization:

      • example.entity.Person.java: A representation of a person, defining such characteristics as name, sex, age, and address
      • example.entity.Address.java: A representation of an address, including street, city, and state

      These two classes implement java Serializable by default and can be used with Coherence caches without change. However, both can be easily modified via POF annotations to support POF serialization.


      Additionally, the following two classes are provided that can be used to test the speed and size of serialized data:
      • example.entity.test.TestPerson.java: Creates instances of the two entity class and then uses com.tangosol.util package classes to exercise serialization
      • example.entity.test.TestPersonPOF.java: Identical to TestPerson but assumes that the entity being serialized supports POF annotations
      These two classes are used to return speed and size information with respect to serialization operations.
      Open a command prompt and change to the directory where the OBE source was unpacked.

      The bin/run.sh command is provided, which can be used to run either of the test classes. This script simply sets the correct classpath and then runs the provided Java class.

      bin/run.sh example.entity.test.TestPerson


      Examine the results of running the test. Note the size of a serialized object and the time taken to serialize and deserialize objects. These values will be used for comparison purposes later.


    Optional: Examine Test Sources

      The classes used to test the entity objects, TestPerson.java and TestPersonPOF.java, use several interesting classes to test serialization. The packages and classes include:

      • com.tangosol.io.pof.PofAnnotationSerializer: Specifies to Coherence how to serialize these objects
      • com.tangosol.util.ExternalizableHelper: Converts back and forth between serialized and unserialized format
      • com.tangosol.util.Base: Provides basic sizing information and formatting services for serialized data
      (Optional) Examine the test application code.

      Assuming the root of the OBE source tree, open either or both sources.

      gedit src/java/example/entity/test/TestPerson.java src/java/example/entity/test/TestPersonPOF.java




      The highlighted sections point out various methods of interest.

    Modify Classes to Support POF Annotations

      Open the Person class. Note that the sources for all classes used in this OBE can be found in the src/java subdirectory.

      gedit src/java/example/entity/Person.java
      Modify the class to specify the class-level @Portable annotation.

      Annotate the class so that it specifies the @Portable annotation. Likewise, add an import for both annotations, which can be found in the com.tansosol.io.pof.annotation package.

      When complete, the updates should resemble those below:


      Annotate the class variables with the @PortableProperty annotation.
      Portable property requires that an index be provided for each property. Assign property indices in ascending order. Note that the transient field is not annotated.


      Save the changes
      Similarly, open and update the Address class.

      gedit src/java/example/entity/Address.java


      When complete, the updates should resemble those below:
      Save the changes and exit the editor

    Test POF Annotations

      Rebuild the sources by using the bin/build.sh script. Correct any errors.
      $ bin/build.sh

      Run the POF version of the test application.

      bin/run.sh example.entity.test.TestPersonPOF

      The result should look like the following:


      Note the much smaller object size and serialization speeds.


Configure Coherence to Support POF Serialization

    In the preceding section, we modified entity objects to support POF serialization via POF annotations. In this section, we complete the picture by configuring Coherence to use POF as a serialization mechanism. To use POF serialization, we will perform the following tasks:

    • Create and populate a POF configuration file.
    • Configure Coherence to use the newly crated POF configuration.

    Creating a POF Configuration File

      POF configuration files are XML files that define the ID and mapping of entity objects so that they can be manged by Coherence.

       Entity objects that support POF are generally specified via one or more user-type elements contained in the user-types element of a POF configuration file.
      Each user-type element specifies two required sub-elements and other optional elements. The two required elements are:
      • type-id: A developer-assigned integer ID >= 1000. Note that <= 1000 is reserved for Coherence.
      • class-name: Fully qualified name of a Java class that implements the portable object interface must be available on the classpath.
      • An example might be:
        <user-types>
            <user-type>
               <type-id>
        1001</type-id>
               <class-name>com...MyEntityImplementingPOF</class-name>
            </user-type>

            . . .
        </user-types>

      In the command prompt window, copy the source POF configuration to the configuration directory.

      cp src/config/pof-configuration.xml config/.


      Open the file in a text editor.

      gedit config/pof-configuration.xml

      Add a user-type element.
      Find the comment containing:
        <!--
               Enter user defined types below this comment.
               User types must be above 1000
         -->

       Directly beneath this comment, add a user-type element.
      The newly added element should look like the following:
       <!--
           Enter user defined types below this comment.
           User types must be above 1000
       -->
       <user-type>
       <user-type>
      Complete the entry by adding an ID of 1001 and a class name of example.entity.Person

      The completed entry should look like the following:

       <user-type>
          <type-id>1001</type-id>
          <class-name>example.entity.Person</class-name>
      <user-type>


      Add a second user-type element for the example.entity.Address class.
      The two completed elements should now look like the following:
      <user-type>
          <type-id>1001</type-id>
          <class-name>example.entity.Person</class-name>
      </user-type>
      <user-type>
         <type-id>1002</type-id>
          <class-name>example.entity.Address</class-name>
      </user-type>

      Save the changes and exit the editor.

    Specifying a POF Configuration File and Enabling POF

      There are two ways to specify a POF configuration. Each has its benefits and drawbacks. The two methods are:
      • Using classpath: The simplest way to specify a POF configuration file is to create a file with name pof-config.xml and place it into the classpath. The pof-config.xml POF configuration deployment descriptor file is used to specify custom user types when using Portable Object Format (POF) for serialization. At run time, Coherence uses the first instance of pof-config.xml that is found in the classpath.

      • Using a system override: -Dtangosol.pof.config={pof-file-name}. When using a system override the file is either found via the classpath, in which case only the file name needs to be specified, or the file can be fully qualified, in which case the complete path to the file and its name should be specified.
     
        Note: In either case, the POF must be enabled either via a override file or via a system property.
        POF can be enabled using -Dtangosol.pof.enabled=true.

      Rename the POF configuration file.

      In a command prompt, ensure that you are in the root of the OBE source. Then rename the POF configuration file.

      $ PWD
      /opt/POF.OBE
      $ mv config/pof-configuration.xml config/pof-config.xml


      In a text editor, open the bin/start-server.sh script.

      gedit bin/start-server.sh

      Find the SYS_OPT section.

      Beneath the line ending in wka=localhost, add a new line that resembles the following:

      SYS_OPT="$SYS_OPT -Dtangosol.pof.enabled=true"

      The update should resemble:


      Save your changes and exit the editor.

    Test the POF Configuration

      Start a Coherence server.

      bin/start-server.sh > server.log 2>&1 &

      Use tail to confirm that the server started correctly.

      tail -f server.log

      Wait until the server shows that it has started, which is indicated by  Started DefaultCacheServer... Then exit tail.
      Use grep to determine that the POF configuration was loaded.

      grep -i pof-config server.log

      The result should look like the following:

      Start a simulation.
      A simple simulation application is provided that interacts with Coherence, adding entities, accessing entities, and otherwise operating on the cache.
      The provided script will start several simulation instances that generate a number of entries. Access those entries and exit.
      The simulation will run for two minutes and then exit.

      bin/start-sim.sh


      After starting the simulation, wait approximately 30 seconds and then examine the simulation logs. The tail command can be helpful to watch the simulation progress.

      tail -f sim.logs/sim.0.log

      You will see person entities being regularly added to a coherence cache.
      Search the simulation log for POF artifacts.

      grep -i pof.config sim.logs/*.log

      Shut down the Coherence server.
      The simulation exits automatically after two minutes. Determine the job ID of the Coherence server by using the jobs command and then shut down the instance by using kill.



Summary

    In this tutorial, you should have learned:

    • What Portable Object Format (POF) annotations are
    • How POF annotations are used in Java entity classes
    • How to test classes that implement POF annotations
    • How to configure Oracle Coherence to use POF for serialization

    Resources

    • The Oracle Coherence 3.7.1 documentation (click here)
    • Various Oracle University classes on Oracle Coherence development and administration (plus advanced training) that can be found here
    • Additional OBEs in the Oracle Learning Library

    Credits

    • Lead curriculum developer: Al Saganich
    • Other contributors: Jason Howes, Tim Middleton, Noah Arliss, and others

To help navigate this Oracle by Example, note the following:

Hiding Header Buttons:
Click the Title to hide the buttons in the header. To show the buttons again, simply click the Title again.
Topic List Button:
A list of all the topics. Click one of the topics to navigate to that section.
Expand/Collapse All Topics:
To show/hide all the detail for all the sections. By default, all topics are collapsed
Show/Hide All Images:
To show/hide all the screenshots. By default, all images are displayed.
Print:
To print the content. The content currently displayed or hidden will be printed.

To navigate to a particular section in this tutorial, select the topic from the list.