You are here: Processor Library > Audit > Pattern Check

Pattern Check

The Pattern Check processor checks the pattern of data in an attribute against reference lists of valid and invalid patterns.

Use

Use the Pattern Check processor to ensure that the data in an attribute conforms to one of the valid patterns for that attribute. Data may need to conform to a set of valid patterns for either technical or business reasons. For example, when migrating data, the target system may require that all data for a given attribute consists of numeric characters only, and with minimum and maximum length restrictions. Alternatively, for business reasons, it may be that you want to tag as invalid records that have bad data, or data in the wrong attributes, for example, numeric values in Name fields, malformed product codes etc.

The lists of valid and invalid patterns can be created from the data itself using the Patterns Profiler.

Pattern Check allows the use of up to two reference lists - a list of Valid patterns for the attribute, and a list of Invalid patterns.

You may choose only to use one of these two lists. For example, if you discover from profiling that there are many different valid patterns for an attribute, you may wish only to check the attribute for invalid patterns, and consider the non-matching values as either Valid, or Unknown.

If, however, the attribute has a small number of valid patterns, you may wish simply to check the data against a list of valid patterns, and consider the non-matching values as Invalid, or Unknown.

Finally, you can use both lists, and recognize both valid and invalid patterns, with values that do not match either list categorized as Unknown.

Configuration

Inputs

A single attribute that you wish to check for valid or invalid patterns (or both).

Options

Option	Type	Purpose	Default Value
Character Map Reference Data	Reference Data (Pattern Generation Category)	Maps each character to a pattern character	*Character Pattern Map

Note:The default *Character Pattern Map is designed for use with Latin-1 encoded data, but you can create new character pattern maps that are suited to the character-encoding of your data, including multi-byte Unicode (hexadecimal) character references.

Valid patterns

Option	Type	Purpose	Default Value
Reference Data	Reference Data (Patterns Category)	List of valid patterns for the attribute	None
Categorize unmatched as	Selection (Unknown/Invalid)	How to categorize values that do not match the list of valid patterns	Unknown

Invalid patterns

Option	Type	Purpose	Default Value
Reference Data	Reference Data (Patterns Category)	List of invalid patterns for the attribute	None
Categorize unmatched as	Selection (Unknown/Valid)	How to categorize values that do not match the list of invalid patterns	Unknown

Outputs

Data attributes

None

Flags

Flag attribute	Purpose	Possible Values
Pattern	Indicates the pattern of the selected attribute.	The Pattern of the attribute
PatternValid	Indicates which data passes the Pattern Check: Valid Patterns, Invalid Patterns and Unknown Patterns.	Y/N/-

Publication to Dashboard

A Pattern Check's results may be published to the Dashboard.

The following interpretation of results is used by default:

Result	Dashboard Interpretation
Valid	Pass
Unknown	Warning
Invalid	Alert

Execution

Execution Mode	Supported
Batch	Yes
Real time Monitoring	Yes
Real time Response	Yes

Results Browsing

The Pattern Check produces a summary view of its results, showing the following statistics:

Statistic	Meaning
Valid records	The records that were categorized as Valid by the Pattern Check.
Unknown records	The records that were categorized as Unknown by the Pattern Check.
Invalid records	The records that were categorized as Invalid by the Pattern Check.

Drilling down on any of the above statistics reveals a count of the distinct patterns that were found to be Valid, Unknown or Invalid. You can then drill down again to see the records themselves.

Output Filters

The following output filters are available from a Pattern Check:

Valid records
Unknown records
Invalid records

Example

In this example, Pattern Check is used to verify values in an Account Number attribute (CU_ACCOUNT) using lists of valid and invalid patterns collated from Patterns Profiling.

Note that values that did not match either list of pattens were categorized as Unknown.

Summary View

Drilldown on Invalid Records