3.2 - Kofax Partner Connect 2013 - Transformation Modules - Advanced Track and What Is New in KTM

PartnerConnect Deutschland Frankfurt, 31.
Januar 2013
Kofax Transformation Modules Expertenforum & Neuigkeiten in KTM?

Stephan Mayer / Stefan Skrok Presales EMEA
Agenda
How to build a successful KTM project?
New features in KTM
Advanced Track & Whats new in KTM - Overview

On Ramp KIC - PDF and Color Documents Productivity Enhancements Design Time Benchmarking

KTM for Forms Kofax Capture Add On Features Technology Enhancements Trainable Document Seperation LimLoc Enhancements Kofax Search and Matching Server Mix Print Detection Clustering Utility
Separation Classification Extraction
Knowledge Base Conflict Management Project Merge Tool Project Builder Test Documents New Xdoc Browser
Productivity Enhancements Users Localisation Thin Client Enhancements Field and Table Drop Down Lists Sticky Notes (Annotations) Advanced Routing Docking and Zooming
This and That Recostar 5 Normalization Format Locator Enhancements Locator Dialog - Testing Script Rotation
The Golden Rule of KTM
Automation?
(i.e. how much data is extracted automatically)
User productivity?
(i.e. how many docs can a user process per hour)
4
The Fallacy of OCR Accuracy

What OCR accuracy do you have? What is the straight-through processing rate? How much can we automate? 85% straight-through processing 23 fields 99.29% field accuracy 6 chars/field 99.89% character accuracy What is the cost of the other 15%? You will lose this deal against an OCR Provider because this
deal is being fought over features and tech, and not business value
User Productivity Example
Groe Supermarktkette in der Trkei
Pan European Wholesaler Before Kofax After 3 months of Automation After 2 weeks of user productivity
invoices/person/day 800 1200 >2000
Improvement 50% 66% (150%!)
What are the goals of a KTM Project?

Every KTM project can be reduced to the following goal Increase documents/person/hour. Decrease clicks/document. Can a user correct a problem faster than your complex
solution?
The goal is not Perfect OCR Perfect UI
Be guided by simplicity, order, speed Do not chase accuracy, chase docs/person/hour.
Anyone can do KTM
Classify Separate Folder Extract Validate Learning

8
All you need is paper and highlighters
Classify Separate Folder Extract Validate

9
Build a Benchmark
Add the Fields you need Classify (F5) Validate (F8) Save Xdocs (
Train Xdocs (F10) Train Project (
Tools/ExtractionBenchmark
/AllClasses
Save Benchmark Open in Microsoft Excel
10
Goals of every KTM Project
1. Human Productivity 2. Eliminate False Positives
bad data leaving Kofax 3. Reduce False Negatives user pressing ENTER 4. Few True Negatives OCR Accuracy, Database problems & learning
11
Benchmark Before
12
Benchmark During
13
Benchmark After
14
How to improve extraction quality beyond just OCR?

We probably have much of the
information to be extracted in our dictionaries/databases already!
15
Database Matching
Slide 16
Kofax Clustering Utility
New Utility for Clustering Unknown Documents

What it does Requirements Step-by-step Importing into KTM
18
What does the Kofax Clustering Utility do?

When configuring KTM content classification, the customer needs
to provide samples for each class.

What KTM requires:
19

When configuring KTM content classification, the customer needs
to provide samples for each class.

What customers usually provide:
20

presorting a document set into clusters of similar documents User labels some of these clusters Utility learns from labeling and pre-sorts again Several iterations of labeling and pre-sorting Export of sorted documents as learn-set for KTM content
classification
21

new KTM project Customer uses Utility to provide KPSG or partner with sorted
documents
KPSG or partner uses Utility to sort documents from customer Understanding what are the biggest subsets of documents in a
customers monthly mailroom volume
enhancing a KTM project Customer adds new classes to project and needs samples for
classification
22
Requirements
Kofax Clustering Utility works with XDocuments XDocuments must be created with KTM OCR Server tool KTM (5.5 SP2) must be installed to use Clustering Utility.
23
Requirements
Using the KTM OCR Server reduces the KTM base volume count Eval licenses supported Hardware requirements same as for KC/KTM Files to be clustered should be local for performance Need write access to file location
24
Step by Step KTM OCR Server

Configuring the KTM OCR Server: Select path to unsorted images Enable Save XDoc files and
Save text files

Under OCR Settings, select
proper language
Leave rest at default Running the KTM OCR Server: Simply press the Start button
25
Step by Step Kofax Clustering Utility

1. Import Point Import directory to same directory of unsorted documents For each document, an .xdc file and a .txt file must exist Select Start Discovery. Takes a while, ~0.5 sec per document Converts XDocs into internal format Identifies initial clusters
26

2. Discovery Label initial 3 clusters You see the most representative document of each cluster Provide a name for each cluster, will be used as class name in
KTM
27

2. Discovery
28

2. Discovery You can stop discovery when 80-90% of the documents are
discovered or continue until all documents are discovered

At 80-90% the most common document types are often known,
remaining documents are likely in very small clusters

Click Review to continue to next step
29

3. Review Sort by categories (labels) Examine the categories for consistency Confirm some documents if you want to cluster again
30

3. Review
31

4. Export Select any directory for export Sub directories will be created for each category/label .txt files (and tif/xdoc for reference) will be exported, since only .txt
files are used to train KTM content classification later
32
Importing into KTM

In Project Builder, point New Project dialog Content Classifier
settings to exported directory

Select Discovered documents sub directory
33
Importing into KTM

A class is created in Project Builder for each category Training documents are imported Select Train in Project Builder main menu Verify in Classification Benchmark (Result Matrix)
34
Importing into KTM
Setting this up manually and finding/organizing the proper training documents takes hours or days. With the Kofax Clustering Utility, this example took 20 minutes.
35
KTM for Forms
Kofax Transformation Modules vs Xtrata

Kofax Transformation for fixed-form and free-form documents, Xtrata limited to fixed forms only. KTM key applications: Automatic Indexing for Archive. Workflow (Mailroom) Automation. Forms Processing. Accounts Payable Automation. Automatic Document Separation. Records management. Xtrata key applications forms processing only.
37
Kofax Transformation Modules vs Xtrata

Advantages of using KTM for forms processing applications
Ability to perform database matching to improve extraction rates -
improves ROI.
More powerful and flexible validation interface (with Xtrata you have to
use the KC validation module) - improves productivity.

More classification methods, i.e. layout and context vs layout only in
Xtrata improves classification accuracy (requires Full Base license)

Scripting for more advanced applications - improves flexibility.
38
KTM for Forms

Basic Information http://www.kofax.com/downloads/datasheets/ds-kc10-license-
update-en.pdf
Features Layout-based classification Unlimited extraction fields Advanced Zone Locator Barcode Locator ABBYY FineReader OCR Document Review (thick client) Validation, Verification, Correction (thick and thin client)
39
Not supported in KTM for Forms

All locators not mentioned in the previous question Content based classification Any OCR besides Abbyy Trainable Document Separation
40
KTM 5.5 On Ramp
KTM 5.5 On Ramp

Kofax Capture Scan
Scan
Business Processes
Fax
Email
Kofax Capture Import Connector
Kofax Transformation Modules Native PDF Support
Export Connector
Folder
Web service
Original Format
42
KTM 5.5 PDF
Supports Color Supports Advanced Binarization for full compatibility with all KTM functions
Supports PDF
Advanced Binarization settings at project level
43
KTM 5.5 PDF
Extracts perfect original PDF text layer. No OCR required!
Image layer is ignored!
Page 44
Productivity Enhancements Design Time
KTM 5.5 Benchmarking

Separation Benchmarking Classification Benchmarking Extraction Benchmarking
46
KTM 5.5 Separation Benchmarking

Separation Benchmark Document Separation Test
47

Golden Files Extraction Benchmarking
Separation Benchmark
Golden Batch
48

Separation Benchmark How can a Golden Batch be created? Kofax Capture (before Export Connector) KTM Project Builder
49

50

Separation Benchmark Quality?
Correct Documents Rejected Documents Incorrect Documents
Incorrectly classified Additional splits Missed splits
False Postive
But confidently Document Review...? The worst of all three categories
51
KTM 5.5 Classification Benchmarking

Classification Benchmark
52

53

54

55

56
KTM 5.5 Extraction Benchmarking

Extraction Benchmark
Slide 57 57
KTM 5.5- Extraction Benchmarking

Extraction Benchmark
EV = Extracted Value EV = GFV EV = GFV EV GFV EV GFV
GFV = Golden File Value (perfect file) Super Work Work False positives
Project quality Project design
Slide 58 58
KTM 5.5 Extraction Benchmarking

Extraction Benchmark - Comparison
59
KTM 5.5 Extraction Benchmarking Extraction Benchmark - Enhancements Selection List
Sorting
By Column Content By Status
Open in Document Viewer Re-arrange columns

60
Productivity Enhancements Conflict Management
KTM 5.5 Conflict Management

Resolve conflicts...
62

Resolve Conflicts Dialogue
63

Toolbar Navigate between conflicts Synchronize Zoom Show All Fields
Page 64

A Conflict Document Delete document Delete field File name Page navigation Conflicting field Field area on document
Page 65

Progress / Summary
66

Delete field
67

Resolve conflict Select the correct value with the mouse
68

Resolve Conflicts
69
Productivity Enhancements Project Merge Tool
KTM 5.5 Project Merge Tool

Multiple Users One Project
71

Project Master
72

Copy the Project Master for each aditional user
73

Project Master
74

Copy 1
75

Copy 2
76

Merge Copy 1
77

Source and Destination projects
78

Select Classes
79

Summary
80

Save changes to destination project (Project Master)
81

Merge Copy 2
82

Source and Destination projects
83

Select Classes
84

Summary
85

Save changes to destination project (Project Master)
86

Project Master after merging
87

Elements that can be merged...
Classes Validation Forms Fields Locators Validation Rules Script Localization
88
89

Elements
90
91
92
93
94
95
96

Summary
97

Save changes
98

The merged project
99
Productivity Enhancements xDoc Browser
KTM 5.5 xDoc Browser

Design Time Enhancement Shows more xDoc Structure Representation
Boxes (used for Mixed Print recognition)

Annotations (Sticky Notes) XValues Helps the user to learn the KTM COM object Useful assistance for scripting
101
KTM 5.5 Project Builder Test Documents

Additional Project Builder Enhancements Open xDoc in Xdoc Browser from Project Builder
102

Additional Project Builder Enhancements Show selected document(s) in Windows Explorer
103

Additional Project Builder Enhancements Choose Details...
104

New look
105
Refresh Document List Previous / Next Document Previous / Next Page Zooming Highlighting On/Off Annotations On/Off
106

Refresh
107

Highlighting
108

Annotations
109

XValues
110

XValues
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22.
AC_BATCH_NAME - Batch Name (read-only) AC_BATCH_CLASS_NAME - Batch Class Name (read-only) AC_BATCH_PRIORITY - Priority (read/write) AC_BATCH_DIRECTORY - ImageDirectory (read-only) AC_BATCH_EXTERNAL_BATCHID - ExternalBatchID (read-only) AC_BATCH_GUID - BatchGUID (read-only) AC_BATCH_CREATIONDATETIME - BatchCreationDateTime (read-only) AC_BATCH_CREATIONSITENAME - CreationSiteName (read-only) AC_BATCH_CREATIONUSERID - CreationUserID (read-only) AC_BATCH_OPERATORUSERID - OperatorUserID (user ID of last batch history entry) (read-only) AC_BATCH_USERID - UserID (read-only) AC_BATCH_USERNAME - UserName (read-only) AC_BATCH_WINDOWSUSERNAME - WindowsUserName (read-only) AC_FIELD - Kofax Capture document fields (read only) AC_TABLE - Kofax Capture table fields (read only) AC_FORMTYPE - Kofax Capture form type (read only) AC_CSS - Kofax Capture Custom Storage String at document level AC_CSS_PAGE<n> - Kofax Capture Custom Storage String at page level AC_REJECTED_DOCUMENT - Indicates if the document has been rejected in Kofax Capture AC_REJECTED_DOCUMENT_NOTE - The rejection note AC_REJECTED_PAGE<n> - Rejected page AC_REJECTED_PAGE_NOTE<n> - Rejected page note
111

Copy Text to Clipboard
pXDoc.Locators.ItemByName("LineItems").Alternatives(0).Table
112
Productivity Enhancements Localisation
KTM 5.5 Localisation

KTM Languages
English
German
114

Additional KTM Languages # 1 2 3 4 5 6 7 8 9 Language Pack Brazilian Chinese Czech French Italian Japanese Polish Russian Spanish Language ID pt-BR zh-CN cs fr it ja pl ru es sv-SE
10 Swedish
Page 115

Additional KTM Languages Graphic User Interface Project Builder and runtime modules Component based messages KTM Server
Documentation
(runtime modules and Userguide.pdf)
1. 2. 3. 4.
Document Review Correction Validation Verification
116

Project Settings - Localization
117

Project Settings - Localization
118

.Net concept
Primary language
English English (United Kingdom) English (United Stated)
en en-UK en-US
Secondary language
119

Fall back principle
120

Fall back principle
Localise
Primary Secondary language translation?
Yes
No
Primary language translation?
Yes
No Use default value for display name Use translation value for display name
End
121

KTM GUI, Server and Active Language
122

KTM GUI Language, Server and Active Language
The Project.ActiveLanguage overrides the Region and Language settings
123

Summary KTM Graphic User Interface language KTM Server language Project language (Project.ActiveLanguage)
124

What can be localised? KTM Element Fields Table Columns Formatting Methods Validation Methods Validation Form Yes/No Component messages used Regular Expression only Component messages used Tab captions Field label Simple label Button captions DB button captions Group captions Script Resources Note
125

Fields
126

Tables
127

Project Script Resources
128

Project Script Resources
Project.Resources.GetString("Error_Example")
129

KTM project folder structure Default language in *.fpr file
Additional languages
Document Review Localised languages
Default language
130

Localisation.xml
External editor Language ID Example: Field Default value Localised translation

131

XML Update
Page 132

Project design language
133
Productivity Enhancements Thin Client
KTM TC 5.5 Improvements

New and Improved Functionality Inside KTM TC 5.5
Validation Form Layouts Annotations Additional Batch Editing Operations User Settings Advanced Login Capabilities Combo-boxes With Descriptions Combo-boxes Inside Tables Other Small Things
135

Support Validation Form Layouts
Different font types and sizes Mini-viewers Custom buttons Location of fields Anchoring Layout localization
136

Support Annotations
Display annotations created by KTM modules Create new annotations inside Thin Client Edit annotations Delete annotations Move annotations Hide/Display annotations
137

Additional Batch Editing Operations Delete pages Move, merge, delete documents Move, merge, delete, split, create folders
138

Preserve User Settings
User name at login screen Batch Open dialog box: size, columns, sorting settings Panels: size, expanded states Zoom settings: fit width, fit height, custom zoom Annotation settings: hide/display annotations
139

Advanced Login Capabilities Domain login for linked users Single sign-on support for Active Directory users
140

Combo-boxes Inside Tables, Items With Descriptions
Display descriptions, values or both Support empty strings consistently for all combo-boxes Paging control for over 100 items Type-ahead filtering capabilities New script events to initialize scripted combo-boxes
141

Other Small Things
Batch loading performance improvements (project caching) PDF support Reject/Unreject documents support scripting on the server Allow to install Thin Client Server on top of previous version Propagate user changes in config files to a new version
142
Productivity Enhancements Advanced Routing
KTM 5.5 Advanced Routing

Batch routing was new in KTM 5.0 with KC 9.0 Kofax Catpture Service Packs allow more functionality: KTM 5.0 Batch Routing (routing of documents) is available in KC 9 Batch Routing (routing of folders) is available with KC 9 SP1 KTM 5.5 Assigning a new batch class to the child batches. KC 9 SP2 required
144
KTM 5.5 Advanced Routing

Setting an XValue assigns a new batch class to a child batch:
KTM_DOCUMENTROUTING_NEWBATCHCLASS_<PlaceHolder>
Page 145
Technology Enhancements TDS / Project Planner
KTM 5.5 TDS Enhancements

KTM AFC Documents
AFC or SVM
KTM 5.5 AFC - Pages
TDS Separation
Algorythm unchanged Re-use training sets
SVM Last AFC
1st Middle
Re-build model
147
KTM 5.5 TDS Enhancements

SVM vs AFC
100,000 pages
Training Set
30,000 docs 100 doc. types
SVM
AFC
Similar accuracey, but the AFC produces fewer missed splits AFC allows for more frequent benchmarking
Page 148 148
Technology Enhancements Line Item Matching
KTM 5.5 Line Item Matching Locator

Use cases for new features: 1. 2. 3. 4.
Multi PO discovery Online Learning Release Matching information to ERP Getting more data
150
Multi PO discovery

151

KTM 5.0
KTM Server
Marked for Learning
Validation clerk
KTM KB Learning Server

Learned
KTM 5.5
KTM Server
Marked for Learning
Validation clerk
Marked for Learning
KTM KB Learning Server

Learned
Slide 152
152

Match Remarks Information about Under-/Over delivery, Ambiguous matches,
etc. are now stored in new global column for Match Remarks
153

Additional columns Table Locator can be used to find additional columns on the
invoice (e.g. Supplier Article code)
LIM Loc as input to Table Locator Table Header pack for column detection
154

Additional columns Additional Database columns (e.g. Cost Center ID) can be copied to
the XDoc Table
155
Technology Enhancements Kofax Search and Matching Server (KSMS)
KTM 5.5 Search and Matching Server
Business Value Faster client startup time Instant feedback Access large enterprise DBs Fast response time Industry standard connectivity Low Maintenance
New in KTM 5.5 Client Server instead of local copy (No Loading Delay No Local Memory Usage) Unlimited DB Size due to 64 bit support (50 Mio Records Tested) Multithreaded design with full support of multi core architecture MS SQL, Oracle, ODBC and CSV Automatic DB Update Scheduler in background
157
Technical Background
KTM Validation / KTM Server KSMS
Administrator
158
Technical Background

Instant access, no loading time Automatic update Direct access to databases Made for 64 bit systems and big databases Load balancing available Multiple KSMS Server Security Active Directory support Secure communication Administration through KTM remote or KTM local client possible Separate installer
159
Kofax Search and Matching Server

Enterprise Customer DB KSMS Search Speed - 1 million records
40,00
35,00
30,00 Search Operations / second
25,00
20,00
Server (8 cores + HT) Server (24 Cores - no HT)
15,00
10,00
5,00
0,00 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
160
Technology Enhancements Mixed Print Detection
KTM 5.5 Mixed Print Detection

Use cases 1. Mixture of OCR and ICR 2. Signatures 3. Read only with OCR or ICR 4. Classification
162

Recognition Profile - MixedPrint
OCR recognition profile full page only ICR recognition profile full page or zonal Mixed OCR and ICR OCR or ICR Threshold for ICR
163

XDoc Representation - Boxes
LongTag = 1 ICR LongTag = 2 Signature Boxes are not created for OCR
164

XDoc XValues
PercentageHP_PageN: X% Can be used for classification
165

Example Project
166
This and That
KTM 5.5 This and That

Drop Down Boxes in Tables Cells Drop Down Boxes Description | Value RecoStar 5 Normalization Format Locator Enhancements Locator Dialogue & Testing Sticky Notes (Annotations) Docking and Zooming Rotation Script
168
KTM 5.5 This and That: Dropdown Boxes in Table Cells

Validation Form Designer Validation Form Same script events as for normal combo boxes
169
KTM 5.5 This and That: Recostar 5

Country and Language
170

Dictionaries
171

Zonal
172
KTM 5.5 This and That: Normalization

Batch and Document structures Memory or on disk?
173
KTM 5.5 This and That: Format Locator Enhancements

Use and Sorting
174
KTM 5.5 This and That: Locator Dialogue & Testing
175
KTM 5.5 This and That: Sticky Notes (Annotations)
176
KTM 5.5 This and That: Sticky Notes (Annotations)

Script events Application_AnnotationCreated Application_AnnotationSaved
177
KTM 5.5 This and That: Docking and Zooming

Allow user to change the view - True/False Docking
178

Allow user to change the view - True/False Docking The zoom value is stored seperately for [Left/Right] and [Top/Bottom]
Top Left Fields Bottom Right
179

Fit to Width
180
181
KTM 5.5 This and That: Rotation

Use case: User rights
182
KTM 5.5 This and That: Rotation

Project script: Document_XDocPageRotated
183
KTM 5.5 This and That: Scripting

Class Script ValidationForm_ButtonDialogClosed ValidationForm_AfterViewerLassoDrawn
184

3.2 - Kofax Partner Connect 2013 - Transformation Modules - Advanced Track and What Is New in KTM

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3.2 - Kofax Partner Connect 2013 - Transformation Modules - Advanced Track and What Is New in KTM

Uploaded by

Copyright:

Available Formats

PartnerConnect Deutschland Frankfurt, 31.

Kofax Transformation Modules Expertenforum & Neuigkeiten in KTM?

How to build a successful KTM project?

New features in KTM

Advanced Track & Whats new in KTM - Overview

Separation Classification Extraction

The Golden Rule of KTM

The Fallacy of OCR Accuracy

User Productivity Example

Groe Supermarktkette in der Trkei

invoices/person/day 800 1200 >2000

Improvement 50% 66% (150%!)

What are the goals of a KTM Project?

Be guided by simplicity, order, speed Do not chase accuracy, chase docs/person/hour.

Anyone can do KTM

Classify Separate Folder Extract Validate Learning

All you need is paper and highlighters

Classify Separate Folder Extract Validate

Train Xdocs (F10) Train Project (

Goals of every KTM Project

1. Human Productivity 2. Eliminate False Positives

How to improve extraction quality beyond just OCR?

information to be extracted in our dictionaries/databases already!

Kofax Clustering Utility

New Utility for Clustering Unknown Documents

What does the Kofax Clustering Utility do?

to provide samples for each class.

What does the Kofax Clustering Utility do?

to provide samples for each class.

What does the Kofax Clustering Utility do?

What does the Kofax Clustering Utility do?

customers monthly mailroom volume

Step by Step KTM OCR Server

Save text files

Step by Step Kofax Clustering Utility

Step by Step Kofax Clustering Utility

Step by Step Kofax Clustering Utility

Step by Step Kofax Clustering Utility

discovered or continue until all documents are discovered

remaining documents are likely in very small clusters

Step by Step Kofax Clustering Utility

Step by Step Kofax Clustering Utility

Step by Step Kofax Clustering Utility

files are used to train KTM content classification later

Importing into KTM

settings to exported directory

Importing into KTM

Importing into KTM

KTM for Forms

Kofax Transformation Modules vs Xtrata

Kofax Transformation Modules vs Xtrata

use the KC validation module) - improves productivity.

Xtrata improves classification accuracy (requires Full Base license)

KTM for Forms

Not supported in KTM for Forms

KTM 5.5 On Ramp

KTM 5.5 On Ramp

Kofax Capture Import Connector

Kofax Transformation Modules Native PDF Support

KTM 5.5 PDF

Advanced Binarization settings at project level

KTM 5.5 PDF

Extracts perfect original PDF text layer. No OCR required!