All articles

Deep learning – the key to next generation document processing

Unfortunately in the world of intelligent automation, there are many half-truths and overpromises being peddled about. Even if you think your automation provider is sufficient, ask yourself one question: is your current platform reliant on machine learning and CPUs? If so, you will never be able to process complex, variable documents – regardless of what you’ve been promised.

Intelligent Document Processing (IDP) – the ability to automate any document type, such as structured, semi-structured, and unstructured – can only happen with deep learning on GPUs or specialized hardware. Conversely, deterministic approaches (aka rules-based approaches) that leverage machine learning, OCR/intelligent OCR, and computer vision cannot handle document variability.

Applica‘s unique deep learning approach is the only platform on the market that can handle document variability because it can learn on its own. Every other platform must be told what to learn by highly skilled humans masquerading as artificial intelligence (AI) by creating endless rules and/or templates on the backend.

Interested in learning why? Let’s dig in further.

The terms machine learning, deep learning, and AI are often used interchangeably, even by category players who should know better, but in fact the differences in meaning are significant and decisive.

Yes, both machine learning and deep learning are subsets of AI. Yes, both can be used to automate the processing of documents, among a great many other things. But machine learning can only do what it’s been told to do, whereas deep learning is designed specifically to figure things out on its own. Sure, both can deliver speed, accuracy, and scalability. But only deep learning has the nimbleness and adaptability that translates into time-proof benefits and real strategic advantage.

Acquiring knowledge, problem solving, communicating, extrapolating, organizing information, translation – these are all-natural modalities of the human brain. Now digital structures are replicating the behaviors of the mind contained within this sophisticated brain. In tech speak, AI is everything that fits this definition. In modern business use, AI is broad term encompassing a vast range of workflow solutions that deploy processes designed to mimic cognitive function. Among them are deep learning and machine learning, which, now you know, are the equivalent of a really fast and precise human knowledge worker who can think for herself and a fast, precise parrot. The latter is good at answering the questions you knew to ask. The former is free to address any question that may wind up relevant in the future. In fact, it might even answer some you didn’t know you had.

What gives deep learning its unique edge is the non-specific and extensive way it is trained – on a data set orders of magnitude larger than that used with machine learning and, notably, without a specific procedural goal. And that’s before the client is even on board. Then, when it is time to give a task to a deep learning network, the specific set of relevant examples and instructions can be small – a few dozen or a couple hundred use cases – and you don’t have to know every possible inquiry in advance. The system has, in a way, “seen it all,” so now it is easy to teach it new tricks. Machine learning, in turn, relies on modeling a hefty stack of training data – often thousands of documents – to a machine that is essentially a blank slate. You have to know what you are going after, and you better get it right the first time.

Eighty percent of the world’s data is contained in complex documents. Historically, leveraging this information has been labor intensive and inefficient. Knowledge workers often spend upwards of 30% of their time reading, comprehending, classifying, and re-keying data. Though tech-based solutions for automating these steps have been around for years, most have worked only with structured or semi-structured documents, such as forms and invoices. Documents with significant variability and dense text-based content, such as contracts, have remained assigned to humans for scrutiny and processing. Somewhat misleadingly, the category featuring these partial solutions is known as IDP, even though the operations that require the highest levels of intelligence remain out of reach of most solutions.

Most current answers to the complex document problem are primarily deterministic approaches that utilize machine learning, which does improve over time as more data is processed, but which forever remains dependent on labels that tell the algorithm what is what. As long as all of the variable conditions can be identified, this approach can work. But unpredictable variability is the weak point – and complex documents still bring unpredictability and variability requiring a different approach.

Because the opportunity is so significant, empty promises flood the market. AI, machine learning, OCR, computer vision – these are a few of the terms second-rate vendors are throwing around carelessly as they claim successful IDP. But no solution can ever exceed the limits described above when its underlying deterministic approaches cannot handle variability. The only way to automate all document types is to create an AI platform that adapts to any structure and actually comprehends the meaning of what is being ingested, classified, and extracted. What is needed is technology thus far associated with different domains, like self-driving cars, but heretofore not unleashed in the IDP sphere. This tech has to be dynamic, mutable, and able to integrate new knowledge even mid-task. It cannot be limited to a finite set of specific decisions, as is the case with machine learning. The system must be smart and agile enough to learn on its own, which can happen when the underlying technology is able to process text-based content in a similar way that humans do. And that is the very definition of deep learning.

Applica’s proprietary technology can be defined as a super-specialized type of IDP – ubiquitous, structure-agnostic, layout aware in a dynamic way, quick to train, and self-correcting. It is of-the-moment deep learning applied to document-based workflows, and it is able to process documents previously impossible to automate. No solution based on machine learning can match Applica’s adaptability, versatility, or strategic edge.

Want to learn more?