How It Works
DataRight IQ processes data in batches, moving each record through a configurable pipeline from raw input to clean, structured output.
The processing model
DataRight IQ works through your data one record at a time, applying a sequence of stages. You choose which stages to apply and how to configure them.
| Stage | What happens |
|---|---|
| Read input | A record is read from one of your input sources. |
| Select for processing | Only the records you want are processed - chosen by criteria (such as geography or demographics) or as a representative sample. |
| Modify on input | Incoming values can be adjusted before parsing - for example, converting coded values or removing unwanted content from a field. |
| Parse | Names, titles, firms, addresses, email, phone numbers, dates, and custom patterns are identified and broken into individual components. |
| Standardize | Values are made consistent - casing, common business words, acronyms, and city/state/ZIP last-line data. |
| Generate new data | Gender codes, prenames, personalized greetings, match standards, and quality scores are added. |
| Split records | A single record can be split into several - for example, separating a combined "John and Mary Smith" into individual people. |
| Select for output | Each output target receives only the records you choose, again by criteria or sampling. |
| Write output | Clean, structured data is written to your target format and layout. |
Inputs
DataRight IQ is built for consolidation. It can read from many input sources at once - even when they use different formats - and bring everything together into a single, consistent target.
- Process multiple input sources in a single run and consolidate them into one output.
- Handle structured (fielded) data as well as free-form data that "floats" across fields.
- Identify and isolate individual elements from unstructured input so they can be placed exactly where you want them.
Outputs
You control exactly what goes into each output target. DataRight IQ can output four kinds of data, in any combination:
- Standardized, parsed data - components and lines that have been cleaned and made consistent.
- Unstandardized, parsed data - components and lines that have been identified but left in their original form.
- Raw input data - values passed straight through from the source, untouched.
- Generated data & codes - new values created during processing, such as gender codes, greetings, and quality scores.
Output is written in the structure and layout you define, so the results drop cleanly into your target system.
Repeatable jobs
A DataRight IQ job captures a complete configuration: the input sources, the processing to apply, and the output targets. Because a job is a saved, self-contained definition, the same process can be run again and again - on a schedule or as part of an automated pipeline - for consistent, repeatable results.