Yardstick Labrigorous software benchmarking
Document AI benchmark

Document AI rankings

There are two rankings on this page. The first is the benchmark. We gave eight systems the same 72 real documents, asked each for the same fields, checked every answer by hand, and ranked them on accuracy and nothing else. You can rerun it yourself from the published data. The second is our vendor scorecard, where we weigh the five dedicated products on everything else you'd care about if you were buying: whether it just works, what it costs, and what you get with it. The benchmark tells you which system reads documents best. The scorecard tells you which product you'd actually want.

Ranking one

The benchmark: accuracy

How many fields each system got right, averaged over every document. One row per system, at its best setting. Five of them are dedicated extraction platforms; the other three are frontier AI models called directly, which is what you'd get if you built it yourself. There's no judgment in this table.

Accuracy, all documents

Share of fields extracted correctly, averaged per document.

If a system couldn't process a document, that counts as a zero. We don't drop it. Every setting we scored, including each product's cheaper one, is in the table further down.
Ranking two

The vendor scorecard

The five dedicated extraction products, scored on everything a buyer weighs. Accuracy is the biggest single factor, but it's only a third of the weight. The rest is our own scoring of each vendor's published pricing, product, and policies. The weights are fixed, and every number has a reason written next to it on that vendor's review. Sorted by the weighted total.

#SystemAccuracy35%Resilience15%Price15%Getting started10%API and integration10%Data control7.5%Compliance7.5%Weightedout of 10
1DocuPipe9.710.06.010.09.09.010.09.1
2Reducto8.96.07.07.07.07.07.07.5
3Extend8.05.07.09.09.07.05.07.3
4Unstructured6.84.08.06.06.08.08.06.6
5Pulse AI7.13.05.07.08.06.08.06.2

Click a vendor to read the reasoning behind each score. The accuracy column is the benchmark result on a 0 to 10 scale. Every other column is our judgment, based on what the vendor publishes about pricing, product and policies. The three AI models called directly aren't on this card, because they aren't document products and most of these columns don't apply to them. Their reviews still score them the same way: Claude Sonnet 5, GPT-5.5, Gemini 3.5 Flash.

How each area is scored
  • Accuracy (35%). The benchmark result, on a 0 to 10 scale.
  • Resilience (15%). Does it just work? Fewer modes to choose, no tuning to reach the ranked result, no runs that fail outright, and no documents that come back quietly empty. Informed by the benchmark, judged by us.
  • Price (15%). Published cost per page for parse plus extraction, cheapest scoring highest. Unpublished pricing is marked down.
  • Getting started (10%). How far you get without a sales call: a dashboard, a free tier with no card, help authoring schemas, a test environment.
  • API and integration (10%). Webhooks and which tier they are on, SDKs, no-code connectors, whether the API is built around document jobs at all.
  • Data control (7.5%). Deletion controls, scheduled retention, zero-retention availability and on which tier, choice of region.
  • Compliance (7.5%). Certifications stated on public pages: SOC 2, ISO 27001, HIPAA with a BAA, GDPR, private deployment options.
Behind the benchmark

Every configuration we scored

The leaderboard shows one setting per system. Here's the full list, with each product's standard and premium settings next to each other, so you can see what the upgrade actually buys you.

SystemSettingAccuracy
DocuPipeHigh97.02%
DocuPipeStandard96.14%
Claude Sonnet 5Called directly91.73%
ReductoDeep Extract89.38%
ReductoStandard81.11%
ExtendDefault80.28%
GPT-5.5Called directly76.48%
Gemini 3.5 FlashCalled directly72.98%
Pulse AITop tier, extended reasoning70.95%
UnstructuredAuto partitioning67.67%
Slice rankings

Where the benchmark separates

The same results, cut by the document types that usually decide a purchase. Each slice comes straight from the published document metadata, so you can rebuild any of these numbers from the evidence page.