You are here: Processor Library > Audit > List Check

List Check

The List Check processor checks the data in an attribute against reference lists of valid and invalid values for the attribute.

The List Check allows case sensitive or insensitive matching, and can match the reference lists in one of a number of different ways:

Contains (the value must contain a matching list entry)
Whole value (the value must match the list exactly)
Starts with (the value must begin with a matching list entry)
Ends with (the value must end with a matching list entry)
Delimiter match (match the list after splitting the data using the specified delimiters)

Note:The List Check processor does not support the use of External Reference Data for the valid or invalid values. Attempting to do so will result in an error message during processing.

Use

The List Check is an important processor, used in auditing to find valid and invalid values in a data attribute. Use the results of the Frequency Profiler or Phrase Profiler to create lists of valid and invalid values, and use them in a List Check to audit the data against the lists on an ongoing basis.

List Check allows the use of up to two reference lists - a list of Valid values for the attribute, and a list of Invalid values.

You may choose only to use one of these two lists. For example, if you discover from profiling that there are many different valid values for an attribute, you may wish only to check the attribute for invalid values, and consider the non-matching values as either Valid, or Unknown - for example simply to look for suspect words such as 'Test' in a Surname attribute.

If, however, the attribute has a small number of valid values, you may wish simply to check the data against a list of valid values, and consider the non-matching values as Invalid, or Unknown - for example to check Title values against a small set of valid titles.

Finally, you can use both lists, and recognize both valid and invalid values, with values that do not match either list categorized as Unknown.

Configuration

Inputs

A single attribute that you wish to check based on lists of valid or invalid values (or both).

Options

Valid values

Option

Type

Purpose

Default Value

Valid values Reference Data

Reference Data

Note:The List Check processor does not support the use of External Reference Data.

List of valid values for the attribute

None

Categorize unmatched as

Selection (Unknown/Invalid)

How to categorize values that do not match the list of valid values

Unknown

Invalid values

Option

Type

Purpose

Default Value

Invalid values Reference Data

Reference Data

Note:The List Check processor does not support the use of External Reference Data.

List of invalid values for the attribute

None

Categorize unmatched as

Selection (Unknown/Valid)

How to categorize values that do not match the list of invalid values

Unknown

Match options

Option	Type	Purpose	Default Value
Ignore case?	Yes/No	Drives whether or not to ignore case when matching the list(s)	Yes
Match list by	Selection (Whole Value/Contains/Starts With/Ends With/Delimiter Match)	Drives how to match against the list(s)	Whole Value
Delimiters	Free text entry	When matching values to the list(s) by splitting the data using delimiters (Delimiter Match), this allows you to specify the delimiter characters to use.	[space]

Outputs

Data attributes

None

Flags

Flag attribute	Purpose	Possible Values
ListValid	Indicates which records pass the List Check. A value of '-' means that the outcome was Unknown.	Y/N/-

Publication to Dashboard

A List Check's results may be published to the Dashboard.

The following interpretation of results is used by default:

Result	Dashboard Interpretation
Valid	Pass
Unknown	Warning
Invalid	Alert

Execution

Execution Mode	Supported
Batch	Yes
Real time Monitoring	Yes
Real time Response	Yes

Results Browsing

The List Check produces a summary view of its results, showing the following statistics:

Statistic	Meaning
Valid records	The records that were categorized as Valid by the List Check.
Unknown records	The records that were categorized as Unknown by the List Check.
Invalid records	The records that were categorized as Invalid by the List Check.

Drilling down on any of the above statistics reveals a count of the distinct values that were found to be Valid, Unknown or Invalid. You can then drill down again to see the records themselves.

Output Filters

The following output filters are available from a List Check:

Valid records
Unknown records
Invalid records

Example

In this example, the List Check is used to check the values in a Business attribute using lists of valid and invalid Business values collated from Frequency Profiling.

Note that values that do not match either list are categorized as Unknowns.

Summary View

Drilldown on Unknown Records

Drilldown on Invalid Records