Giant language fashions work significantly properly with uncooked textual content. Firms that need to create their very own AI workflow know that it has turn into extraordinarily necessary to retailer and index knowledge in a clear format in order that this knowledge will be reused for AI processing.
That’s why Mistral is launching a brand new API immediately for builders who deal with advanced PDF paperwork. Mistral OCR is an optical character recognition API that may flip any PDF right into a textual content file.
In contrast to most OCR APIs, Mistral OCR is a multimodal API, which means that it will probably detect when there are illustrations and photographs intertwined with blocks of textual content. The OCR API creates bounding bins round these graphical components and consists of them within the output.
Equally, Mistral OCR doesn’t simply output a giant wall of textual content. The output is formatted in Markdown, a formatting syntax that builders use so as to add hyperlinks, headers and different formatting components to a plain textual content file.
Giant language fashions rely closely on Markdown for his or her coaching knowledge units. Equally, if you use an AI assistant, reminiscent of Mistral’s Le Chat or OpenAI’s ChatGPT, they usually generate Markdown to create bullet lists, add hyperlinks or put some components in daring. Assistant apps seamlessly format the Markdown output right into a wealthy textual content output. That’s why uncooked textual content — and Markdown — have turn into extra necessary lately.
“Over the years, organizations have accumulated numerous documents, often in PDF or slide formats, which are inaccessible to LLMs, particularly RAG systems. With Mistral OCR, our customers can now convert rich and complex documents into readable content in all languages,” Mistral co-founder and chief science officer Guillaume Lample mentioned.
“This is a crucial step toward the widespread adoption of AI assistants in companies that need to simplify access to their vast internal documentation,” he added.
Mistral OCR is on the market on Mistral’s personal API platform or by its cloud companions (AWS, Azure, Google Cloud Vertex, and many others.). And for firms working with labeled or delicate knowledge, Mistral additionally gives on-premises deployment.
In accordance with the Paris-based AI firm, Mistral OCR performs higher than APIs from Google, Microsoft and OpenAI. The corporate has examined its OCR mannequin with advanced paperwork that embrace mathematical expressions (LaTeX formatting), superior layouts or tables. It’s also alleged to carry out higher with non-English paperwork.
On condition that Mistral OCR does one factor and one factor solely, the corporate believes additionally it is quicker than what’s on the market. That’s not a shock in case you examine it with a multimodal giant language mannequin like GPT-4o, which additionally has OCR capabilities (amongst many different options).
Mistral can also be utilizing Mistral OCR for its personal AI assistant Le Chat. When a person uploads a PDF file, the corporate makes use of Mistral OCR within the background to grasp what’s within the doc earlier than processing the textual content.
Firms and builders will more than likely use Mistral OCR with a RAG system to make use of multimodal paperwork as enter in an LLM. And there are various potential use circumstances. As an example, I may see regulation corporations utilizing it to assist them swift by big volumes of paperwork.
