four-wheel drive: abbyy flexicapture & mobile captureimages.abbyy.com/partner...

Olga Titova, Andrew Zyuzin

Four-Wheel Drive: ABBYY FlexiCapture & Mobile Capture

ABBYY FlexiCapture Enables Complex Automation Environments to Work

2© ABBYY Confidential

Systems and RobotsEnterprise applications and

digital robotic processes

designed to run businesses

DatabasesRobotic Process Automation (RPA)

ECM ERP, CRM, BPM

ABBYY provides the intelligence to turn unstructured content into structured, actionable information, transforming processes for a better customer experience, lower costs, lower risk and competitive advantage.

Customer InteractionsReal-time, Content Centric Processes

Structured & Unstructured Content

FAXMFP

SCANWEB

MOBILE

E-MAIL

Digitization occurs at the Knowledge Worker’s point of need (not IT mandate) – the platform is for making business units more effective in serving their customers, facilitating meaningful change

3

New Opportunities with State-of-the-art AI

© ABBYY Content IQ Conference. Bologna 2019

Intelligent Image Capture

Customer Onboarding

Advanced Classification

Mortgage, Intent Detection

Semi-structured

P2P/O2C, Expenses

NLP

Commercial Real Estate, Healthcare

Intelligent Image Capture


Quality matters!

5© ABBYY Content IQ Conference. Bologna 2019

Automatic image capture

• estimates camera movement, frame sharpness and looks for document boundaries

• captures the best document image from video stream

Crop document and make it look as rectangle

Get the best image possible


Mobile usage scenarios

7

• Customer registration/account opening

On-device, cloud based or on premise server side back end solution• Customers self-service

trailing document

© ABBYY Confidential

No need to download an app, just open a webpage

For user looks like camera working in web-browser

Works on both Android and iOS

Mobile Web Capture: Loan application


LIVE DEMO

Advanced Classification


11

• A major financial sector in US.

• 66% homeowners have mortgage today

• In 2018 $1.75 trillion mortgages originated

• Mortgage refinancing

• 400+ doc types

• Speed of the approval is key for origination

• Depth of analysis – to refinance

Mortgage Challenge

Market Key Players Capture Challenge


• Origination

• 40% banks

• 51% non-depository lending institution

• Refinance - 46% GSEs

FlexiCapture 12How Advanced Classification works


А

Input Classification

text-based

image-based

Ensemble

Result

Confidence89%

IMG

TXT

CNN

RNN & NLP

SVM

?

Class A 90%Class B 50%Class C 10%

Class A 40%Class B 80%Class C 75%

ID

ID

ID

ID


Intent:AccountingPay invoice # <attached>Due date: 2019/07/30

Banks, insurance companies, public sector, service desk of any company receive a huge amount of repetitive messages every day….

When running a video in ADC 3.0 I have recently lost my sound. I have attached screenshots of my MAC sound system settings to get started.Regards,Andy Smith

Intent:Support ticketSound issue

Intent 1:Insurance.Car insurance originationCustomer: Andy Smith

Please calculate the cost of insurance for my car. Documents for the car attached.Andy Smith

Intent Detection with ABBYY FlexiCapture

Hi, see the invoice attached, please pay by the end of the month.

Invoices with Deep Learning


17

ABBYY FlexiCapture for InvoicesPowered by Deep Neural Networks

• Imitates how human reads the invoice

• Quality boost from 80 to 90+%for invoices header fields

• No Master Data

• Extended country support

© ABBYY Confidential

18

P2P/O2C with ABBYY FlexiCapture


Buyer areas

Supplier areas

Purchase order

Order confirmation

Delivery note

Invoice

Remittance advice

Payment

SALESProcess incoming POCreate a sales order

LOGISTIC EXECUTIONDeliver goods or services

ACCOUNTS RECIEVABLECreate outgoing invoice

Process RA and find open itemsClear open items

ACCOUNTS PAYABLEProcess incoming invoicesCreate remittance advice

Pay invoice

LOGISTIC EXECUTIONProcess incoming delivery

notes

PURCHASINGCreate purchase order

Process confirmation

Various departments

Demand, need or requirementCreate a requisition

Procurement to pay and order to cash Buyer's and supplier's

documents:

• Purchase Orders

• Delivery Note

• Sales Orders

• Remittance

Shipment documents:

• Bill of lading

• Waybill


A-Za-z0-9.,

A-Za-z0-9,

A-Za-z0-9.,

A-Za-z.,/

A-Za-z0-9,

A-Za-z0-9,A-Z0-9/.-

• Works perfect for similar layouts, like invoices from 1 supplier, 1 invoice is enough

ABBYY FlexiCapture 12On-premise ML – One-shot Learning

The algorithm automatically splits incoming documents into groups with similar looking document layouts.

Extraction models are trained for each group.

Master Data is not required.

Finally, more accurate learning and less training.

20

ABBYY FlexiCapture 12Unsupervised Clustering Enables On-premise ML

R3

NLP for Unstructured


What can be addressed with NLP

22

bill of lading pay stub noticeappraisal

NLPFlexilayout

Healthcare: research articles analysis

Use Cases for ABBYY FlexiCapture NLP

© ABBYY Confidential 23

Financial services for real estate

Contract approval automation

Financial solutions for Commercial Real Estate (US)


Real estate total return is around 7% in 2018 to 8% in 2019

16M loan agreements in 2018at origination and secondary market

Portfolio Management AnalystCommercial Real EstateSalary: well paidLevel of expertize: high

• Reviewing and analyzing complex loan documents, inter-creditor documents and deal documents

• Knowledge and understanding of credit and risk management principles

Scenarios: Loan scoring, Risk assessment, Loan

underwriting, Compliance regulations

Pain: struggle to extract huge amount of data from

Loans and supporting docs

Service Provider for real estate finance industry

Main Documents in Commercial Real Estate


Leases Loans Loan Applications

Data Extraction from Lease AgreementsNLP assists to portfolio analyst to extract data from hundreds of leases

26

1. Tenant Name: Collis Agency, LLC (“Tenant”)2. Suite Number: Suite No. 9083. Lease Commencement Date: December 1, 20164. Lease Expiration Date: January 31, 20195. Lease Annual Rent:

12/01 /2016-01/31/2017 $846.00/mo02/01 /2017-01/31/2018 $871.00/mo02/01/2018-01/31/2019 $897.00/mo

6. Security Deposit: $798.007. Pro Rata Share: (i) Tenant’s Proportionate Share: .45%,

which is the percentage by dividing (a) the 716 rentable square feet in the Premises by (b) the 158,397 rentable square feet in the Project.

8. Lease Commencement Date definition: If the Commencement Date is not the first day of a calendar month, then the Term shall be extended by the time between the Commencement Date and the first day of the next month. If this Lease is executed before the Premises become vacant or otherwise available and ready for occupancy by Tenant, or if any present occupant of the Premises holds over and Landlord cannot acquire possession of the Premises before the Commencement Date, then (a) Tenant’s obligation to pay Rent

9. Square Footage: 1,35210. Original Occupancy Date: January 12, 201611. Dates of Each Renewal / Extension: Every year12. Insurance: Cost of any insurance or insurance related

expense applicable to the Project and Landlord’s personal property used in connection therewith

…

NLP Model for Leases

• Lease agreement contains 27-35 pages

• 22-37 fields should be extracted

• NLP model was train on 15docs

• 1 min to process 1 lease agreements

NLP Data Extraction from Loan AgreementsLooking for reduction in time, now data are processed manually

27

1. Origination Date: March 29, 20182. Entity Name – role Lender: DEUTSCHE BANK AG, NEW

YORK BRANCH3. Entity Address: 60 Wall Street, 10

thFloor, New York,

New York, 100054. Entity Name - role Borrower: DW PORTLAND LLC5. Entity name with role Co-Borrower: DW OMAHA SUB

LLC6. Entity Address: 640 Fifth Avenue, 17

thFloor, New York,

New York, 100197. Entity Overflow Name: Russell Gimelstob8. Business Day: “Business Day” shall mean any day other

than a Saturday, a Sunday or a legal holiday on which national banks are not open for general business in (i) the State of New York or (ii) the state where the servicing offices of the Servicer are located.

9. Debt Service Coverage Ratio: “Debt Service Coverage Ratio” shall mean, a ratio, as reasonably determined by Lender in which:

(a) the numerator is the Annualized Underwritten Net Cash Flow; and(b) the denominator is the Annualized Debt Service.

10. Debt Yield: “Debt Yield” shall mean, as of any date of determination, a ratio, expressed as a percentage, as reasonably determined by Lender, obtained by dividing:

(a) the Annualized Underwritten Net Cash Flow determined as of such date; and(b) the Aggregate Outstanding Principal Balance.

…

NLP Model for Loans

• Loan agreement contains 200-300 pages

• 50-70 fields should be extracted

• 3 iterations to clarify requirements

• 2 weeks to understand the contact language and to mark-up 23 docs

NLP in ABBYY FlexiCapture

Segment contains terms

Segment contains liability

28

ABBYY FlexiCapture NLP

Start Date: SEPTEMBER 26, 2007

End Date: JUNE 30, 2008

Maximum liability: $6,000

Segmentation enables us:• to improve accuracy of field extraction• to increase speed of processing

29

ABBYY FlexiCapture NLP

NLP understands document as it “knows” a lot about each word in a sentence and relations between them

Patient Leaflet

Pharmacovigilance (PV)


Medical literature: articles in journals related to pharmaceutical products

Pharmacovigilance (PV), also known as drug safety, is the pharmacological science relating to the collection, assessment, monitoring, and prevention of adverse effects with pharmaceutical products.

31

400 doctors and pharmacists Salary: well paidLevel of expertize: high

• 700 drugs are tracked

• 30-200 articles about each drug weekly

• Severe adverse events (death, related to newborns, pregnancy) should be reported less than in 14 days

• Report to a pharmaceutical company and to the local drug regulatory authority

Service Provider for Pharmacovigilance Individual Case Safety Reports (ICSR) detection


Pharmacovigilance (PV)

Age 59-year-old Gender Female Adverse reactions Loss of muscle strengthMedication Simvastatin Dosage 40 mg PO qDay in the eveningMedication Fluconazole Dosage 150 mg PO as a single dose Adverse reactions Rhabdomyolysis

Thank you!
