Eight document AI systems, reviewed
What each one is good at, where it falls down, what it costs, and what you get for the money. Every review starts from the same benchmark and the same hand-checked answers, then looks at the product around the API, and ends with a scorecard that uses the same weights for everyone.
Top of the board, with no document below two-thirds and the largest leads on long, non-English, and scanned material.
The best direct-model baseline and third overall, strong on long documents, weaker on handwriting and right-to-left layouts.
Closest specialized challenger, best in the field on right-to-left documents, thinner on Chinese and Japanese scans.
Excellent on short, scanned, and handwritten pages; collapses on long, list-heavy documents.
As good as anyone on a typical short form, but returns nothing on the longest documents, where its output budget runs out.
Near the top on single pages and images, near the bottom on anything long, with the widest spread of any system.
Competitive on Chinese and Japanese scans, but unpredictable, with several ordinary short documents returning nothing.
A parsing layer rather than an extractor; strong on clean English pages, weakest on long, tabular, and CJK documents.
What is not on this board, and why
Our benchmark is zero-shot. Every system gets the same schema at request time and none of them get to train on the documents first. That means we can't fairly rank platforms whose product is a model trained on your own documents, or a template you configure per document type, so we don't score them. Nanonets, Rossum, ABBYY, Docparser, and Parseur all fall into that group. It's a boundary of the category, not a verdict on those products.
Vendors that do fit the category get added as we finish their runs. The methodology page explains what a system needs before it can enter the ranking.
Scores are macro-average field accuracy from the DocuBench 0.2.0, 72-document run. Every document, answer key, and raw extraction is open in the explorer; the scoring rules are on the methodology page, and the replication kit re-runs the scorer. A document a system could not process scores zero rather than being dropped.