Integrated Real-Time Intelligence Using Oracle WebCenter, Oracle Coherence, and Oracle Business Activity Monitoring

Providing next-generation access to critical data.

Part of the Oracle Fusion Middleware Patterns article series.

Download:

• Oracle Fusion Middleware

Introduction

Application users in today's world demonstrate an ever-increasing need for real-time intelligence as part of their user experience, a variety of aggregate information along with key performance indicators (KPIs) available in real time when they need it and where they need it, without having to search for it. A bank teller processing transactions typically needs real-time intelligence for the customer in front of him. Without that information, the teller is in no position to offer sound advice on savings or changing investment patterns, or to advise that to act on new information. This represents a lost opportunity to cross-sell and up-sell new products and services. If the bank teller had real-time intelligence on current mutual fund interest rates, the customer's investment pattern, and the health of the customer's investments, he could, for example, advise the customer to move funds from a low-interest account into a better-performing money market account for a short term.

However, intelligence alone is not sufficient. Once the intelligence is gathered, the next issue to address is accessibility. How does a user gain access to this critical data? Traditionally, there are many ways employed, including printed paper reports.

Traditional applications have always separated data entry or data management functions from reporting and analytics. They are handled by two different applications, two different information portals, or two different technologies.

Let's imagine the world of Enterprise 2.0 (E2.0), in which every information worker is empowered to be a decision-maker, cutting out key bottlenecks upfront. In this scenario, nuggets of information are treated like gold and made available to the users when and where they need them. Why should business-critical analytical information be treated any differently? Users are constantly in need of these nuggets of business intelligence (BI) either historical or real-time which help them to create and manage their transactions much better.

A combination of BI and E2.0 allows us to combine information management and analytics in the same context and transaction. In this article we will put together a reference architecture for contextual, real-time business insight. Using as an example DJO, a leading global provider of high-quality orthopedic devices, we will walk through a real-life example of how this is accomplished using Oracle WebCenter, Oracle Business Activity Monitoring, and Oracle Coherence.

Understanding Real-Time Intelligence

So, what is real-time BI? To understand this, let's slide BI within an enterprise across three different categories:

• Historical intelligence. Information aggregated over a period of years or months and typically stored in a data warehouse environment. Reports are run over minutes or hours to produce deep intelligence, collecting vast amounts of data.
• Near-real-time intelligence. Data aggregated over the last few weeks or days, and typically stored in transactional databases. Information is produced by reporting tools close the transactional applications.
• Real-time intelligence. Information aggregated over the last few hours or days of data and typically stored in memory for fast access. Information is produced by analytical engines, and has short-term importance or relevance.

The need for real-time intelligence is becoming more critical because traditional enterprise resource planning applications and homegrown portals are focused mostly on just transactions intake creating an order, managing customer information, updating financial information, and so on. Gone are the days when transactions were viewed as a point of data management. You have to look at the process as a whole. The ability to create an order while you update customer and financial information during the transaction is critical. Businesses are grooming users to be decision-makers across a process in order to avoid a multistep transaction process. As more and more businesses adopt this practice, real-time intelligence becomes key to any decision-making process.

Building a contextual, real-time BI application requires a combination of four key technologies: a Web 2.0 framework, business activity monitoring (BAM), grid caching, and service-oriented architecture (SOA), as illustrated in Figure 1.

Figure 1: Building blocks of a real-time BI application.

• Web 2.0 Portal. This portal provides content, presence, and social networking capabilities to create a highly interactive user experience. End users will use this portal to gain insight into real-time business activity.
• Business Activity Monitoring. This will allow the application to capture key events occuring during a business activity. These events are aggregated, analyzed, and presented via a Web 2.0 portal in easy-to-understand KPIs. Keep in mind that BAM is distinct from the dashboards used by BI. BI provides historical intelligence, whereas BAM processes events in real time or near real time.
• Grid Caching. Data grid caching will boost the performance of the application by caching non-dynamic events into memory. This will eliminate the need for the Web 2.0 portal to talk to the BAM/SOA layer for relatively static data and will provide extreme performance and scalability.
• Service-Oriented Architecture. This layer, along with the BAM layer, provides a platform for event-driven architecture. SOA is responsible for orchestrating the key business processes by integrating with a variety of enterprise applications. During the execution of the processes, triggered events can be captured by the BAM layer, or services can be activated by triggers fired on incoming events. SOA and BAM integration typically takes place through the Java Message Service (JMS).

In the next section, we will put together a reference architecture and present an approach to build a contextual, real-time intelligence application.

Reference Architecture

As previously discussed, this application requires integration between the Web 2.0 portal, SOA, BAM, and a grid caching solution (Figure 2).

Figure 2: Real-time BI architecture.

The processing sequence involves six steps:

1. Event generation and process orchestration. All process activities requiring analytics are modeled through a loosely coupled orchestration process. This helps in efficiently altering the pattern and any information captured as part of the application event, if required. Oracle BPEL Process Manager (PM) can help here by allowing the user to model standards-based (BPEL) business processes. Using Oracle BPEL PM, an analyst can add activity sensors to monitor the execution of activities within a BPEL process, or add fault sensors to catch any failures. During this step, the process orchestration layer generates various events, which are passed through JMS or a similar medium and the enterprise service bus (ESB).
2. Event detection and absorption. The JMS bus provides the means to transport the events through a process into a BAM engine. The JMS messaging queue allows multiple endpoint systems to consume the business-generated events.
3. Processing and filtering. All the captured events are filtered, correlated, and analyzed to gauge their impact on critical KPIs and service-level agreements (SLAs). The end user is notified of any new, pertinent information. The BAM engine continually updates the active report while it is being reviewed by the user. These updates continue until the user closes the report.
4. Real-time availability. BAM engines generally come with a built-in data cache. However, for large-scale enterprise solutions, there can be no trade-off between speed and scalability. In this situation, in-memory data caching solutions can significantly improve application performance. During this step, based on the frequency of data change, the data grid cache is refreshed and subsequent requests from the Web 2.0 portal are processed by the cache. Oracle Coherence provides this capability through an in-memory data grid layer, which can cache as much as 1TB of data. This approach not only improves application performance, but makes the application less hardware intensive.
5. Representation and visualization. During this step, the user is presented with visually powerful information. Developers can use declarative frameworks to model dynamic data objects to capture underlying intelligence. Oracle Application Development Framework Data Visualization (ADF DVT) components are a set of rich interactive Oracle ADF Faces components that provide significant graphical and tabular capabilities for visualizing and analyzing data.
6. User interaction and personalization. The Web 2.0 portal (Oracle WebCenter) allows the user to aggregate a variety of information into a personalized dashboard. BAM charts, workflow activities from business processes, an interface to tweak KPIs, and the ability to define rules and filters all come together on a single screen. By providing additional integration and runtime customization options, control is placed directly in the hands of end users to slice, dice, and analyze data the way they want.

All six steps are highlighted in Figure 3.

Figure 3: Six steps to real-time BI.

Now that we have reviewed the reference architecture and the six steps to build such an app, we will focus on the real-life implementation of a similar application at DJO Global.

Building a Real-Time Call Center App at DJO

DJ Orthopedics (DJO), headquartered in Vista, California, designs, manufactures, and distributes a line of technically advanced products and services for the prevention, treatment, and rehabilitation of acute and chronic orthopedic and spinal conditions. DJO, as part of its reimbursement business, runs call centers processing private and government medical insurance claims from all regions of the United States. The call centers also supply patients with healthcare devices and process the claims with insurance companies. Clinics and patients supply critical healthcare information as part of their claims. Call centers interact with healthcare insurance providers through electronic data interchange (EDI) transactions.

Information about customers and orders is distributed across custom databases, third-party systems, and Oracle E-Business Suite. However, in this configuration call center agents had no access to real-time aggregated customer information.

It is imperative for agent workings on claim processing to view in real time the daily or weekly call activity associated with a specific claim. Call center managers require real-time forecasting and load distribution to process claims, recognize trends, and address cash flow in real time.

In order to provide access to the necessary information, DJO decided to build a real-time BI application that would support call center user interaction and showcase real-time information on:

• Call activity
• Order activity
• Financial activity
• Personal productivity goals

Architecture

DJO Global chose the combination of Oracle ADF, Oracle Coherence, Oracle BPEL PM, and Oracle BAM to build this solution (see Figure 4), with Oracle E-Business Suite at the core to provide all processes, from order management, contract management, fulfillment, and shipping to financials and order-to-cash.

Figure 4: Call center real-time insight application.

Let's review these architecture pieces using the invoice reconciliation DJO use case. DJO organizes all patient claims based on the insurance provider, and sends the claims invoices to the insurance companies for reimbursement. These external transactions are supported by business-to-business processes through Oracle SOA Suite. The insurance companies perform direct payment to the banks, and the banks send back EDI files to DJO with explanations of the payments. This information has to be reconciled with the source information sent to the insurance companies to ensure and sign off on records paid. If information can't be reconciled, DJO processes records that are, for various reason, not paid.

In some cases, pending transactions from past months may still be awaiting payment. There is a need for real-time intelligence as new files are received and new claims are adjudicated. It is necessary to get a real-time view of the records processed today, total records sent for reimbursement, total payments received, and pending reconciliation.

As part of this reconciliation, multiple steps occur:

1. Original source records are retrieved.
2. Source records are dispatched to insurance companies.
3. New payments are received and reconciled.
4. Agents are notified of claims that they need to process and finalize by working with the patients.
5. If paid by the insurance companies, information needs to be finalized in AR and receipts should be acknowledged.

Therein lies the need for a real-time dashboard that can be viewed by both call center agents and invoice agents to collaborate, process, and rectify.

Design Goals	Solution Approach
Call center agents and invoice agents need visually rich dashboards to review invoice and order activity.	Leverage Oracle ADF DVT for visualization
Reconciliation information should be instantly available upon request (1k transactions/second expected).	Oracle Coherence to act as in-memory data grid to collect and display data.
Dashboards should be updated as soon as new reconciliation information is available. Agents should be able to analyze data in a variety of formats (charts, graphs).	Oracle BAM
The process of integrating order and invoice data should be automated. Processes and policies should be easy to adapt to new healthcare regulations.	BPEL engine as part of Oracle SOA Suite managed with Oracle BPEL PM.

Now that we understand how all the pieces fit together in the context of the invoice reconciliation use case, let's walk through the six steps that were outlined earlier.

1. Event Generation and Process Orchestration

   As soon as new invoice reconciliation data is received, Oracle E-Business Suite triggers new events. Many Oracle E-Business Suite products leverage the Oracle Workflow Business Event System for business process integration. Although this is not the only method available for integrating Oracle E-Business Suite into a business process, it does allow an ESB or BPEL process to be event driven using standard Oracle E-Business Suite functionality.

   Oracle E-Business Suite posts real-time data to the "invoice monitor" BPEL process. An invoice schema (XML Schema Definition, or XSD) is created as a template for data validation from Oracle E-Business Suite.



Figure 5: Invoice schema (XSD) for validation.

Leveraging the invoice XSD, the Oracle E-Business Suite event posts the data to the invoice monitor BPEL process for orchestration.

2. Event Detection and Absorption

   The invoice monitor BPEL process takes the invoice reconciliation data from Oracle E-Business Suite, enriches it, and sends it to the BAM layer. Figures 6 and 7 offer two views of the invoice monitor process. Figure 6 is a Service Component Architecture (SCA) view of the BPEL process, and Figure 7 is a traditional view of the process, showing how it transforms data and sends it to BAM.



Figure 6: SCA indicating a BPEL process orchestrating the information to BAM.



Figure 7: BPEL orchestration for data transformation and storage.

3. In this scenario, Oracle BPEL PM integrates with Oracle BAM using Oracle BAM Web services. This can alternatively be done by sending the information on a JMS bus. Oracle BAM Web services allow users to build applications that publish data to the Oracle BAM server for use in real-time charts and dashboards. Any client that can talk to standard Web services can use these APIs to publish data to Oracle BAM.

   The data objects in the Oracle BAM server are available using Oracle BAM Web services in this case, using the ICommand Web service. An ICommand Web service partner link is configured as shown in Figure 8.

4. 

Figure 8: Sample iCommand service configuration to BAM.

5. Once BPEL publishes this data to the BAM server, it is then processed and filtered.
6. Processing and Filtering

   Oracle BAM receives invoice reconciliation data from the BPEL process and presents this data in the BAM dashboard with the help of invoice data objects. Invoice data objects reflect the business data that is captured in BAM for presentation, analysis, alerts, and so forth. After execution of the invoice monitor BPEL process, an ICommand Web service call populates the invoice data object. In Figure 9, you can see the definition of the invoice data object as well as the populated data.

   

   Figure 9: Invoice data object definition in BAM.

7. Filters help in limiting the subset of data and displaying the required business information. Filters can also be used to define certain KPIs or SLAs and display content matching (or not matching) these criteria. In Figure 10, data is filtered so that only invoice reconciliation data older than one week is displayed in the dashboard.

8. 

Figure 10: Filtering rule for displaying invoice data older than one week

9. Once the data is stored and filtered, processed information is rendered live in the BAM dashboard. The Oracle BAM Architect process guides you in how to set up the charts and render the data in the dashboard.

10. 

    Figure 11: Filtered information displayed in the BAM dashboard.

11. However, the BAM dashboard is not enough for invoice and call center agents. Agents want to see this information right within their dashboards as they are processing the patient or insurance company. So as a next step, this information needs to be integrated into the overall dashboard.
12. Real-Time Availability

    The next step is to cache important BAM data in Oracle Coherence so that it can eventually be surfaced in the Oracle ADF user interface. Let's review how this is done.

    A scheduled Store Invoices BPEL process retrieves the data from BAM and posts it to the Oracle database tables (Figure 12).

13. 

Figure 12: Store Invoices process to store invoice data in database, eventually cached by Oracle Coherence.

The Store Invoices BPEL process uses the ICommand Web service to retrieve the data from the BAM data object to an XML file. After the Invoices.xml file is generated, it uses the file adapter service to retrieve the contents of the XML file.

The Store Invoices BPEL process then prepares a collection object comprised of all the invoice contents present in the Invoices.xml file by using XSL transformation. Finally, it calls a PL/SQL procedure by passing the generated collection object. The PL/SQL procedure stores the data to the Invoices database table. It updates the existing records and creates a new record, if none already exists.

The next step is to configure the Oracle Coherence cache to reflect this data store for the application. The application is integrated with Oracle Coherence by configuring the application in Oracle JDeveloper. In Oracle JDeveloper, we refer to the Oracle Coherence cache configuration file with the workspace definition.

14. 

Figure 13: Configuring Oracle Coherence in Oracle JDeveloper.

Add the following lines to the Java Configuration of the ADF Application:

• Dtangosol.coherence.distributed.localstorage=false
• Dtangosol.coherence.log.level=3
• Dtangosol.coherence.cacheconfig=[ConfigLocation]\invoice-cache-config.xml

With this logic, the information is now available to the Oracle ADF Model layer to be visualized with ADF View components.

For more information see Creating Oracle Coherence Caches in Oracle JDeveloper.

15. Representation and Visualization

    Now we are ready to leverage the information available in the cache to render the information on a Web application.

    To enhance the visualization of the rendered event information in Oracle Coherence, we leverage the Oracle ADF DVT components that are available as part of Oracle ADF 11g. DVT components in Oracle ADF can be used to build graphical representations of data, such as bar charts, gauges, Gantt charts, and geographical maps. The best feature of DVT components is the declarative model in which the cache can be wrapped into a model layer as part of the model-view-controller pattern within Oracle ADF 11g. This model can help the view objects hold the real-time data retrieved from the Oracle Coherence cache.

16. 

Figure 14: Oracle Coherence data integrated with Oracle ADF 11g DVT, wrapped into a task flow.

17. These charts are wrapped into an ADF task flow as reusable user interface components within a portal or an application. Task flow components are also made available to the Business Dictionary feature of Oracle WebCenter.
18. User Interaction and Personalization

    Now let's look at how these components are available for a user to interact with and also personalize the data as well as control the visibility of the information.

    The dashboard in Figure 15 displays a view of the entire application. Invoice agents and call center agents get personalized views where different KPIs (work lists, orders, patient details, payors, patient notes, physician details) come together. The result is a centralized view of their customers and an opportunity to have intelligent discussions with them.

19. 

Figure 15: Oracle ADF 11g rendering of the dashboard view including the BAM graphs (only sample data for graphs rendering).

Conclusion

The combination of BI and E20 enables us to combine information management and analytics in the same context and transaction. This solution could easily evolve into an architecture that combines complex event processing with online analytical processing to further expand the capabilities to leverage business intelligence.

Acknowledgments

Many thanks to Gana Duraiswamy and Sandeep Reddy, solution leads at Keste, for putting the solution together and making this a reality.