All articles

How Applica Overcomes NLP and OCR Limitations With a New Breed of AI

While there has been a marked increase in the awareness and usage of text automation tools in recent years, there has been a similar uptick in confusion about what all the acronyms and names actually mean, and furthermore, what the bottom line is for your company’s automation efforts.

Let’s set the scene by digging a little deeper into a few common automation types: 

  • Legacy Natural Language Processing (NLP) tools have been around for quite some time (the idea of The Turing Test dating back as early as 1950) and the two main categories comprising legacy NLP technologies are:
    1. Rule-based systems (based on keywords, handcrafted rules, patterns or templates), which are good for processing homogenous docs, but requires laborious definition and maintenance
    2. Legacy machine learning methods, these need vast amounts of data, while also requiring heavy involvement from Data Scientists for training and maintenance

While there have been incredible advancements made in recent decades, NLP still only deals with plain text – not with tables or forms.

  • Optical Character Recognition (OCR) in its modern form has been around for almost as long as NLP, with major breakthroughs coming in the 1960s and 1970s. Similarly, great advancements were made more recently, and what most organizations use today is so-called Intelligent OCR, based on Computer Vision. The upside to these tools is that organizations can automate templated, semi-structured documents such as tables and forms. But again, this comes with some innate downsides to straight-through processing, because Intelligent OCR relies heavily on templates, while only extracting snippets of text (e.g.: document headers, table cells, etc.) from homogenous, templated documents. Similarly, deployment and maintenance costs are high due to laborious preparation and maintenance of the templates. Intelligent OCR also struggles with untemplated documents and with plain text, therefore organizations have relied on multiple automation tools to match the various document types coming in and out of their business.
  • Applica is the newest member to the automation landscape, as it was specifically developed to bridge the gaps in NLP and OCR. It combines the best of two worlds – deep-learning-driven NLP and Computer Vision – and takes a forward-looking, deep learning-based approach to automation, allowing for comprehension of complex/variable documents no other solution can process. Applica processes all document types – plain text, tables, and forms – all without relying on laborious templates. By combining Computer Vision with layout-aware neural language models, Applica is able to contextually extract information from documents and each data point is given an interpretation in context. Since Applica is managed by typical business users (instead of expensive AI engineers) and self learns with less than 10% of the data required in other AI models, this allows organizations to switch from manual, time-intensive work to highly accurate, automated text comprehension and decision making within seconds. Therefore, employees that often are overqualified for repetitive tasks can be reassigned to higher-value work.

You may still have questions about how Applica actually “works” and could be puzzled as to how it’s able to do so many things that had previously been roadblocks for decades, so let’s break it down further with some additional context.

When organizations began employing automation to their workflows, they quickly realized that only structured data could be processed with Robotic Process Automation (RPA). And since 85% of enterprise data is unstructured text, this led companies to seek out other solutions – such as OCR/Intelligent OCR. But again, these can only automate in very specific, narrow use cases while also relying on templates.

At Applica we saw an opportunity to overcome these long-standing challenges with our own AI-enabled cognitive automation solution and developed Applica. While we use OCR as an entry, it’s important to note that the OCR input does not have to be perfect for Applica to work effectively. This is because we abstract from the “reading order” imposed by OCR software, therefore if the order of words or phrases are broken by the OCR, it will not be an impediment to Applica. Additionally, Applica can handle both born-digital and scanned documents (and also just plain text if needed).

Using our exclusive layout-aware Language Model (LAMBERT) and 2D Contextual Awareness, Applica is unique in that it can handle not just plain text or scanned documents, but also documents in which information is to be found in both purely textual fragments and parts in which the 2D layout plays a greater role. For instance, imagine a form accompanied by instructions or disclaimers, or a report with both tables and long paragraphs – meaning Applica can process all these unstructured and semi-structured documents.

Applica’s layout-aware Language Model (LAMBERT) starts with general knowledge of business-related language and documents. Even before it is given to our customers, it “internalizes” answers to millions of questions like “What to expect in rows starting with ‘total profit’ in columns with the year ‘2020’?” (and LAMBERT learns that an amount of money is expected there rather than an ID number, surname or date, or that the currency is expected along with the sum, e.g.: in a column header), and then uses this powerful knowledge to precisely extract the required information from documents. Additionally, Applica’s 2D Contextual Awareness leverages machine learning in the form of deep neural networks when applying algorithms from computational linguistics and computer vision. This mimics the way a human works with documents containing various layouts (e.g.: forms, tables, reports, etc.) and considers both textual and graphical aspects before finalizing the results. Coverage of both channels of information enables precise semantic analysis and information extraction, reducing the workload for some business processes by 90% and error rates by 85%. As our unique method is not based on handcrafted templates or keywords, Applica can be applied to a wide range of document collections in an organization. For instance, whenever you have a collection of unstructured or semi-structured documents (PDFs, DOCXs, etc.) with business-actionable information locked there, whether it’s large or small, you can apply Applica. Once you define what is relevant and what you are looking for, Applica will seamlessly handle both the classification and information extraction tasks. Once you trained the model and the information is precisely processed into what we refer to as Bots, that information can be easily pushed over to an integrated RPA system to finalize the document processing lifecycle. By adding Applica to an existing RPA or workflow management solution, organizations can achieve 90%+ straight through automation.

Applica is also multilingual – pretrained in most major languages and additional languages can be rapidly added when needed. Therefore, Applica can handle not only multilingual collections of documents, but also a document in which the information given is in two languages (e.g.: an invoice in French with items descriptions in German).

While Applica is industry agnostic, we’ve seen many successful implementations in the finance, insurance, and legal services sectors, including the following use cases:

Want to learn how Applica can help solve automation pain points and improve business continuity at your organization?

Request a demo with an Applica expert.