Document AI benchmark

Every document, every score

A ranking you cannot audit is an opinion. This is the full working of the document AI leaderboard: all 50 documents, what each system scored on each one, and a link to the original file so you can open it and judge the document for yourself. The column averages at the bottom of the table are the headline ranking numbers - nothing is held back to make them come out differently.

50documents, all published in full

5system configurations scored on every document

10documents every system extracted perfectly

0documents excluded from the average

How to read this

The scoring, in one paragraph

Each document has a fixed extraction schema and a human-checked answer key. A system's score on a document is the share of fields it got right, so 100 means every field matched the key. The best score on each row is highlighted. Where a system could not process a file at all, that counts as a zero rather than being quietly dropped, which is why the averages here are lower than the numbers vendors publish for themselves.

Document	DocuPipe high	DocuPipe standard	Reducto Deep Extract	Reducto standard	Extend	Source file
Bank Transaction Register (CSV) Englishcsv1 pp	100.0	98.6	100.0	100.0	54.0	Open file
Carson Bank Sample Statement Englishpdf6 pp	100.0	100.0	100.0	100.0	100.0	Open file
Chang Gung Chinese Medical Receipt Chinesejpeg1 pp	100.0	100.0	89.5	89.5	100.0	Open file
Clinical Discharge Summary (HTML) Englishhtml1 pp	100.0	100.0	95.8	95.8	100.0	Open file
CPS Energy Electric Bill Englishpdf2 pp	100.0	100.0	100.0	100.0	100.0	Open file
DC DISB Homeowners Declarations Englishpdf1 pp	100.0	100.0	100.0	96.0	100.0	Open file
Dutch Jaarrekening Balans+Winst-en-verliesrekening (excerpt) Dutchpdf3 pp	100.0	100.0	100.0	95.9	100.0	Open file
EDI 850 Purchase Order Englishtxt1 pp	100.0	100.0	100.0	100.0	88.9	Open file
Ethiopian Air Waybill Englishpdf1 pp	100.0	76.2	95.2	80.0	100.0	Open file
Ethiopian Air Waybill (rotated 90, TIFF) Englishtiff1 pp	100.0	85.7	95.2	95.0	95.2	Open file
FHA 1004 Appraisal (sliced from 21p) Englishpdf7 pp	98.6	96.2	89.9	94.7	94.9	Open file
Fidelity Brokerage Statement (sliced from 28p) Englishpdf18 pp	97.7	94.7	89.7	94.3	87.6	Open file
Financial Sample Spreadsheet (trimmed 30 rows) Englishxlsx1 pp	96.6	96.6	100.0	100.0	94.9	Open file
Five Guys Receipt (rotated 180, PNG) Englishpng1 pp	100.0	100.0	98.9	98.9	100.0	Open file
Five Guys Receipt Photo Englishjpeg1 pp	100.0	100.0	97.7	97.7	97.7	Open file
FL Homeowners Declarations Englishpdf1 pp	100.0	100.0	100.0	100.0	91.2	Open file
FRESA Arabic Tax Invoice Arabicpdf1 pp	98.4	98.4	100.0	98.4	98.4	Open file
German 1976 Wage Statement Germanjpeg1 pp	89.7	100.0	87.3	80.9	78.6	Open file
German DEG Jahresabschluss Bilanz+GuV (excerpt) Germanpdf2 pp	98.3	95.7	92.2	99.1	73.5	Open file
Getbuildingworks BOQ Summary Englishpdf1 pp	100.0	100.0	100.0	100.0	100.0	Open file
Gordmans PO Line Items Englishpdf1 pp	95.2	95.2	92.9	76.2	97.6	Open file
Hebrew Financial Statement (DOCX) Hebrewdocx1 pp	100.0	100.0	100.0	75.9	83.3	Open file
Hebrew Payslip Hebrewjpeg1 pp	100.0	100.0	62.1	13.8	44.8	Open file
Hitachi Japanese Invoice Japanesepdf1 pp	98.4	98.4	97.1	98.1	90.7	Open file
Hitachi Japanese Statement Japanesepdf1 pp	98.9	100.0	90.3	97.6	90.0	Open file
Hyperfine Schedule Sheet (sliced from 16p) Englishpdf1 pp	100.0	100.0	100.0	99.5	100.0	Open file
Italian Busta Paga (payslip) Italianpdf1 pp	97.4	97.4	100.0	100.0	94.3	Open file
Labsmart CBC Lab Report Englishpdf2 pp	100.0	100.0	100.0	100.0	98.1	Open file
Majdal Shams Municipal Financials Hebrewpdf13 pp	95.5	95.1	94.0	95.5	62.8	Open file
Moneck Merchant Statement Englishpdf5 pp	99.8	100.0	99.5	96.2	99.8	Open file
NASA Mechanical Drawing (sliced from 128p) Englishpdf1 pp	94.8	80.4	92.7	93.5	79.5	Open file
NRCS Pedon Handwritten Form Englishpdf2 pp	100.0	97.8	96.6	95.5	97.8	Open file
Optum Discharge Summary Englishpdf2 pp	100.0	94.4	100.0	100.0	100.0	Open file
Orders Export XML Englishxml1 pp	100.0	100.0	88.6	88.6	100.0	Open file
Peppol UBL E-Invoice Englishxml1 pp	100.0	100.0	100.0	95.7	100.0	Open file
PG&E Solar NEM Statement Englishpdf4 pp	100.0	100.0	100.0	100.0	84.8	Open file
Portuguese Balanco e Demonstracao de Resultados (excerpt) Portuguesepdf2 pp	97.6	100.0	97.6	96.4	100.0	Open file
PPL Electric Two Bill Example Englishpdf3 pp	100.0	100.0	100.0	100.0	100.0	Open file
Ramat Gan Arnona Rates 2025 Hebrewpdf5 pp	76.3	78.1	78.3	60.5	73.7	Open file
Sage French Bulletin de Paie (payslip) Frenchpdf1 pp	100.0	95.5	100.0	94.6	81.0	Open file
Samson Oil & Gas Check Stub Englishpdf2 pp	100.0	100.0	94.1	94.1	91.3	Open file
SEC EDGAR FedEx Credit Agreement (excerpt) Englishpdf2 pp	100.0	100.0	100.0	100.0	100.0	Open file
SEC EDGAR Precision Lease 4th Amendment (excerpt) Englishpdf3 pp	100.0	100.0	100.0	100.0	100.0	Open file
Smartsheet Construction Invoice Englishpdf4 pp	100.0	100.0	100.0	100.0	100.0	Open file
Spanish Iberdrola Factura Electricidad Spanishpdf4 pp	100.0	100.0	100.0	100.0	100.0	Open file
Taub Hebrew Tax Invoice Hebrewpdf1 pp	95.5	95.5	100.0	86.4	100.0	Open file
UICCP Packing List Carton Dimensions Englishpdf1 pp	100.0	100.0	100.0	100.0	100.0	Open file
University of Pittsburgh Filled Sample W-2 Englishpdf3 pp	100.0	100.0	100.0	100.0	100.0	Open file
Vince Sea Waybill Englishpdf2 pp	80.0	80.0	98.2	98.2	61.5	Open file
Zionist Strategy NGO Financials Hebrewpdf4 pp	69.4	65.6	75.0	62.5	69.4	Open file
Average across all 50 documents	97.56	96.31	95.77	92.70	91.11	See ranking

The documents, answer keys, and scoring code are published in the open benchmark repository, DocuBench. Anyone can re-run the scorer and get these numbers back.

Corpus

Where these documents come from

Every document is a real file with a traceable public source: SEC filings, utility bills, government forms, published annual reports, and vendor sample documents. Provenance and licensing for each one is recorded in the benchmark repository. The corpus deliberately includes formats that break parsers - spreadsheets, XML, scans, photographs - and documents in a dozen languages, because the interesting part of this category is what happens on the hard tail, not on a clean English invoice.