Open topic with navigation

Processes

Processes are configured sequences of processors, used within a project to process a given set (or sets) of data.

Each process normally has a specific purpose, such as to profile a table, to audit data transactions as they occur, or to match data.

A process begins with a Reader processor. Any other available processors may be added to the process from the Tool Palette.

Process States

There are four possible states for a process. The background color of the Canvas changes depending on the state. The following table shows the four states and how they change the background color:

Note that a process with a gray background color may be read-only because it is running (but was opened after the run started) or because it is not the current version of a process.

Running and Scheduling Processes

There are four ways of running a process:

• Using the Quick Run button on the Toolbar
• Right-clicking on a process, and selecting Run
• Using the Process Execution Preferences on the Toolbar, and running the process
• Defining and running (or scheduling) a Job containing the process as a task

Whichever of these options are used, processes are always run on the connected OEDQ server. Provided the server stays running, you can therefore set a process running, close your client session and reopen it later when the process has completed, or you can schedule the process to run at a more convenient time (for example, overnight). Alternatively, if you are still developing the process, you can run it and watch its progress before assessing results.

Quick Run

The Quick Run button runs the process immediately, using the current execution preferences for the process. If the process has never been run before, it uses the default options for process execution. The default options will not refresh any snapshots, publish any results to the Dashboard, or run any exports. Intelligent execution will be used to minimize the amount of processing required on the server to deliver the most up-to-date results.

To use the Quick Run option, simply open the process that you want to run, and click on the button in the toolbar.

Process Execution Preferences

The Process Execution Preferences button allows you to define options for process execution, such as:

• Running the process on a sample of the input data, rather than all of it
• Limiting result drill downs in order to speed up the process
• Adding data interface mappings for processes that read from or write to data interfaces so that you can run the process interactively
• Running in interval mode so that real time processes write their results on a regular basis
• Publishing results to the Dashboard

You can then run the process directly from this screen.  The settings you make here will be saved and used whenever you use the Quick Run option in the future.

To change the Process Execution Preferences, open the process, click on the button in the toolbar, and change the options. Separate tabs exist for Readers, Process and Writers, to allow you full access to the different execution options in OEDQ - for example so that you can choose not to copy the source data into the repository, but to write your staged data both into the repository and externally.

It is possible to promote a Process Execution Preference as a Job, ('Save As Job'), which can be scheduled in the future, see 'Defining and scheduling a job'.

For a full guide to the available options when running a process, see Execution Options.

Defining and scheduling a job

Defining and scheduling a job provides additional functionality:

• To run a job in the future and on a recurring basis
• To define a job consisting of many tasks
• To run external tasks before or after these tasks.

Canceling a Process

 It is possible to cancel a running process through the User Interface.  This can be done via the right click option on the process or by pressing the stop icon at the bottom left hand side of the canvas (which appears when the process is running).  

It is also possible to cancel running tasks from the Task Window.

The options on canceling a process are to:

• Cancel as quickly as possible
• Cancel but save results generated so far

For real time processes running as a Web Service, a third option is available:

• Shutdown Web Services

The Shutdown Web Services option offers a clean way of closing down Web Services, closing any open results intervals, and keeping results.

Processes which have been canceled will appear in the Project Browser with the amber stop icon to indicate that the process was canceled (explaining why its results may be missing or incomplete). See Project Browser Object States.

Process Change Control

Director allows you to control changes to your processes by allowing you to create versions of processes.

When a new version of a process is created, Director automatically saves the previous version and makes it Read-only.

If necessary, you can revert to a previously saved version of a process if you have changed something in error.

Creating a new process version

To create a new version of a process, right-click on the process in the Project Browser, and select New Version...:

This creates a Save Point of the process. The Save Point is logged with the user, the date and time, and the comment for the Save Point that you enter in the dialog box. For example:

Opening previous process versions

You can open previous process versions, and view their configuration on the canvas. Results for old process versions are also persisted and stored until purged (by purging all results for the process or project, or by deleting the process).

Reverting to a previous process version

If you have changed the configuration of a process in error, you can revert to a previous saved version of the process. To do this, right-click on the process, and select Revert to Version. This actually creates a new version of the process, using the selected previous save point.

Deleting or Purging a process

A process may be deleted by right-clicking on the process in the Project Browser, and selecting Delete (or pressing the Delete key).

The results for a process may be purged by right-clicking on the process, and selecting Purge Results. In some cases, there may be a large amount of results data for a process. In this case, you will see the purge operation running on the server in the Task Window.