Data Extraction & System Configuration

IDP System Deployment

Configuration & Deployment

While you can compare the solution capabilities for data extraction in terms of the supported techniques, systems will differ in performance even when both use, for instance, keyword/value pairs.

This is because systems use different means of internal validation of field output. This affects both the accuracy of the data itself as well as the confidence scores associated with the data.

When comparing systems, the evaluation of the amount of output and amount of accurate output is insufficient. If your organization wishes to achieve optimal straight through processing, you will need to evaluate the amount of accurate answers along with the reliability of confidence scores for both accurate and erroneous data.

System Configuration

Traditionally, configuring systems required labor-intensive manual rule creation and coding using text parsing algorithms. However, contemporary solutions have revolutionized this process, simplifying it so non-programmers can build comprehensive classification and data extraction workflows. This is valuable when the staff lacks programming skills, both during the initial setup and subsequent adjustments in a production environment.

Streamlined systems for classification and extraction, achieved through the reduction of necessary steps and the provision of flexible rule adjustments, not only save time but also provide users with a clear understanding of the entire project. Moreover, the simplified configuration can reduce project costs and mitigates risks associated with improper setup.

Capture System

Classification & Extraction Configuration

Some systems require the use of more than one application to configure rules, so it is critical to understand what is involved in getting your IDP up and running.

The use of simple-to-use User Interfaces (UIs) enables efficient and low-risk configuration. Systems that support a full configuration across different document types (e.g. structured forms vs. unstructured documents) with one application are the easiest to use as it interface reduces the complexity involved in configuring and supporting different classification and data extraction rules.

Configuring automation rules is only part of the process. You will need to consider how those rules perform on production-level documents.

Solutions enabling users to test document classification and data extraction results without executing the entire capture workflow significantly simplify the iteration process for maximizing system accuracy. These solutions allow testing results at the configuration stage rather than requiring the user to completely configure the entire IDP workflow.

Another process improvement is solutions that include the option to automate rule creation. Some solutions take user feedback during production (such as data entry operator or actual machine learning) and use it to improve data extraction and classification. This is called ‘online learning’

Alternatively, offline learning allows a user to teach a system how to classify documents or extract data, all without programming knowledge.

System Precision

Organizations stand to gain efficiencies from adopting solutions offering automated, machine-learning-based document classification and separation. However, the most accurate method of comparing these systems is by evaluating the quality of their output, specifically assessing the system's ability to correctly classify and separate documents within a sample.

To do this, one must gather a representative set of documents, not just a few, run them through the system, and then directly compare the precision of one system to another.

This is critical when documents vary in layout, scale, or when there are differences in image input resolutions (e.g., images from faxes versus scanners). This will ensure the system can process different configurations and variances between documents.

Having a diverse range of location and extraction capabilities ensures that you can optimize the effectiveness of implementation without incurring additional labor costs.

Capture System Deployment

IDP is a complex process often involving many different steps and underlying technologies. Vendors approach their solution offering in a myriad of ways. They range from a modular approach that involves installing and configuring many different applications to databases or workflow engines. Other vendors adopt a more simplified single application strategy that involves fewer 'moving parts'.

Many systems use third-party applications to present a full solution. These applications might be included at no additional cost, or they are prerequisites that must be purchased and configured separately.

Understanding what is part of the standard software package vs. what is not is essential to understanding the total cost and requirements.

In addition to IDP, you have choices regarding the automation of the installation and configuration process itself. Some systems support a very simplified single-computer installation to enable the ability to quickly test the software without incurring a lot of effort.