Document AI benchmark
Every document, every score
A ranking you cannot audit is an opinion. This is the full working of the document AI leaderboard: all 72 documents, what each system scored on each one, and an explorer where you can open any document alongside its answer key and every system's extraction to judge it for yourself. The column averages at the bottom of the table are the headline ranking numbers; no document is removed from them.
How to read this
The scoring, in one paragraph
Each document has a fixed extraction schema and a human-checked answer key. A system's score on a document is the share of fields it got right, so 100 means every field matched the key. The best score on each row is highlighted. Where a system could not process a file at all, that counts as a zero rather than being dropped from the average.
| Document | DocuPipe | Claude Sonnet 5 | Reducto | Extend | GPT-5.5 | Gemini 3.5 Flash | Pulse AI | Unstructured | Explore |
|---|---|---|---|---|---|---|---|---|---|
1950 Census Population Schedule Englishpdf1 pp |
94.4 | 87.2 | 55.2 | 73.8 | 26.0 | 69.3 | 38.1 | 77.1 | Open |
A.L.A. Catalog 1904 — Sociology Englishpdf16 pp |
98.5 | 94.5 | 65.5 | 57.3 | 72.3 | 28.7 | 0.7 | 7.1 | Open |
A.L.A. Catalog 1904 — Useful Arts Englishpdf16 pp |
99.2 | 98.8 | 75.3 | 46.8 | 86.3 | 15.1 | 1.9 | 5.5 | Open |
Alcalá Venceslada Andalusian Dictionary Spanishpdf16 pp |
98.5 | 96.0 | 89.8 | 72.8 | 82.1 | 0.0 | 98.2 | 39.5 | Open |
Bank Transaction Register (CSV) Englishcsv1 pp |
100.0 | 97.1 | 100.0 | 54.0 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
Blanchard Library Auction Catalog 1898 Englishpdf14 pp |
99.2 | 99.5 | 96.7 | 87.8 | 97.0 | 0.0 | 0.0 | 99.8 | Open |
Carson Bank Sample Statement Englishpdf6 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 99.5 | Open |
Chang Gung Chinese Medical Receipt Chinesejpeg1 pp |
100.0 | 100.0 | 89.5 | 100.0 | 94.7 | 100.0 | 100.0 | 100.0 | Open |
Chaturvedi Sanskrit-Hindi Dictionary 1917 Hindipdf28 pp |
95.2 | 91.0 | 82.2 | 32.0 | 0.0 | 3.1 | 31.3 | 25.6 | Open |
Clinical Discharge Summary (HTML) Englishhtml1 pp |
100.0 | 100.0 | 95.8 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
CPS Energy Electric Bill Englishpdf2 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 92.9 | Open |
DC DISB Homeowners Declarations Englishpdf1 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 96.0 | 100.0 | Open |
Des Moines Drug Co. Price List 1906–07 Englishpdf11 pp |
95.6 | 99.4 | 81.8 | 90.3 | 0.0 | 0.0 | 94.7 | 17.9 | Open |
Dutch Jaarrekening Balans+Winst-en-verliesrekening (excerpt) Dutchpdf3 pp |
100.0 | 100.0 | 100.0 | 100.0 | 76.7 | 100.0 | 93.2 | 95.9 | Open |
EDI 850 Purchase Order Englishtxt1 pp |
100.0 | 100.0 | 100.0 | 88.9 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
Ethiopian Air Waybill Englishpdf1 pp |
100.0 | 95.2 | 95.2 | 100.0 | 95.2 | 100.0 | 85.0 | 95.2 | Open |
Ethiopian Air Waybill (rotated 90, TIFF) Englishtiff1 pp |
100.0 | 85.7 | 95.2 | 95.2 | 100.0 | 100.0 | 81.0 | 95.2 | Open |
Everybody's Spanish Dictionary Spanishpdf15 pp |
99.9 | 98.8 | 97.4 | 34.3 | 0.0 | 7.3 | 0.0 | 0.0 | Open |
Fall River City Directory 1905 Englishpdf22 pp |
97.9 | 96.4 | 92.9 | 29.0 | 0.0 | 8.9 | 47.1 | 3.7 | Open |
FHA 1004 Appraisal (sliced from 21p) Englishpdf7 pp |
98.6 | 96.2 | 89.9 | 94.9 | 89.9 | 96.2 | 90.7 | 89.9 | Open |
Fidelity Brokerage Statement (sliced from 28p) Englishpdf18 pp |
97.7 | 86.0 | 89.7 | 87.6 | 97.4 | 90.8 | 72.9 | 81.6 | Open |
Financial Sample Spreadsheet (trimmed 30 rows) Englishxlsx1 pp |
96.6 | 93.2 | 100.0 | 94.9 | 100.0 | 100.0 | 94.9 | 100.0 | Open |
Five Guys Receipt (rotated 180, PNG) Englishpng1 pp |
100.0 | 95.7 | 98.9 | 100.0 | 79.4 | 92.2 | 96.6 | 94.3 | Open |
Five Guys Receipt Photo Englishjpeg1 pp |
100.0 | 100.0 | 97.7 | 97.7 | 98.9 | 97.7 | 88.7 | 100.0 | Open |
FL Homeowners Declarations Englishpdf1 pp |
100.0 | 100.0 | 100.0 | 91.2 | 100.0 | 100.0 | 100.0 | 96.8 | Open |
FRESA Arabic Tax Invoice Arabicpdf1 pp |
98.4 | 100.0 | 100.0 | 98.4 | 100.0 | 100.0 | 100.0 | 98.4 | Open |
German 1976 Wage Statement Germanjpeg1 pp |
89.7 | 100.0 | 87.3 | 78.6 | 96.8 | 100.0 | 3.4 | 96.8 | Open |
German DEG Jahresabschluss Bilanz+GuV (excerpt) Germanpdf2 pp |
98.3 | 92.2 | 92.2 | 73.5 | 95.7 | 99.1 | 95.7 | 72.2 | Open |
Getbuildingworks BOQ Summary Englishpdf1 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
Gordmans PO Line Items Englishpdf1 pp |
95.2 | 100.0 | 92.9 | 97.6 | 66.7 | 100.0 | 0.0 | 83.3 | Open |
Hebrew Financial Statement (DOCX) Hebrewdocx1 pp |
100.0 | 100.0 | 100.0 | 83.3 | 83.3 | 100.0 | 96.3 | 98.1 | Open |
Hebrew Payslip Hebrewjpeg1 pp |
100.0 | 34.5 | 62.1 | 44.8 | 89.7 | 89.7 | 20.7 | 96.6 | Open |
Hitachi Japanese Invoice Japanesepdf1 pp |
98.4 | 100.0 | 100.0 | 90.7 | 99.5 | 100.0 | 90.4 | 80.3 | Open |
Hitachi Japanese Statement Japanesepdf1 pp |
98.9 | 90.6 | 90.6 | 90.0 | 90.6 | 90.6 | 90.6 | 37.2 | Open |
Hyperfine Schedule Sheet (sliced from 16p) Englishpdf1 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 95.4 | 72.4 | Open |
Italian Busta Paga (payslip) Italianpdf1 pp |
97.4 | 100.0 | 100.0 | 94.3 | 96.5 | 100.0 | 97.4 | 97.8 | Open |
Jastrow Talmudic Dictionary Hebrewpdf10 pp |
86.0 | 61.4 | 89.1 | 7.1 | 37.8 | 62.3 | 18.8 | 53.3 | Open |
Kanpō No. 5877 (1946) Japanesepdf1 pp |
77.2 | 63.2 | 13.2 | 38.2 | 2.2 | 14.7 | 40.4 | 3.7 | Open |
Labsmart CBC Lab Report Englishpdf2 pp |
100.0 | 98.1 | 100.0 | 98.1 | 98.1 | 100.0 | 89.0 | 84.6 | Open |
Libbie-Goodell Law Book Auction Catalog 1918 Englishpdf40 pp |
99.6 | 98.9 | 95.8 | 88.1 | 97.1 | 24.6 | 15.0 | 3.0 | Open |
Majdal Shams Municipal Financials Hebrewpdf13 pp |
95.5 | 69.9 | 94.0 | 62.8 | 89.4 | 89.8 | 68.1 | 68.1 | Open |
Merchant Vessels of the United States 1914 Englishpdf10 pp |
99.9 | 89.0 | 95.1 | 98.3 | 0.0 | 0.0 | 0.0 | 26.6 | Open |
Moneck Merchant Statement Englishpdf5 pp |
99.8 | 99.8 | 99.5 | 99.8 | 94.0 | 99.8 | 99.8 | 99.8 | Open |
NASA Mechanical Drawing (sliced from 128p) Englishpdf1 pp |
94.8 | 90.1 | 92.7 | 79.5 | 89.8 | 94.8 | 3.1 | 1.5 | Open |
NOAA Pittsburgh Monthly Weather Summary 1890 Englishjpeg1 pp |
93.5 | 45.2 | 51.6 | 91.9 | 61.3 | 88.7 | 67.7 | 22.6 | Open |
NRCS Pedon Handwritten Form Englishpdf2 pp |
98.9 | 91.1 | 97.8 | 98.9 | 87.6 | 98.9 | 100.0 | 86.1 | Open |
Official Register of the United States 1873 — Treasury Englishpdf21 pp |
99.8 | 97.1 | 96.4 | 97.8 | 0.0 | 0.0 | 93.3 | 0.0 | Open |
Optum Discharge Summary Englishpdf2 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 83.3 | 100.0 | Open |
Orders Export XML Englishxml1 pp |
100.0 | 100.0 | 88.6 | 100.0 | 100.0 | 100.0 | 0.0 | 88.6 | Open |
Peppol UBL E-Invoice Englishxml1 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 94.2 | 95.7 | Open |
PG&E Solar NEM Statement Englishpdf4 pp |
100.0 | 93.3 | 100.0 | 84.8 | 97.5 | 100.0 | 93.3 | 100.0 | Open |
Polk South Bend City Directory 1921 Englishpdf10 pp |
99.1 | 92.8 | 81.5 | 19.5 | 84.6 | 18.0 | 25.3 | 27.9 | Open |
Portuguese Balanco e Demonstracao de Resultados (excerpt) Portuguesepdf2 pp |
97.6 | 96.4 | 97.6 | 100.0 | 97.6 | 96.4 | 96.4 | 74.4 | Open |
Post Office London Directory 1889 Englishpdf3 pp |
99.2 | 91.7 | 56.3 | 24.5 | 0.0 | 0.0 | 57.7 | 88.5 | Open |
PPL Electric Two Bill Example Englishpdf3 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 95.2 | 100.0 | 100.0 | Open |
Ramat Gan Arnona Rates 2025 Hebrewpdf5 pp |
76.3 | 72.8 | 78.3 | 73.7 | 76.1 | 72.1 | 66.4 | 62.6 | Open |
Sage French Bulletin de Paie (payslip) Frenchpdf1 pp |
100.0 | 81.0 | 100.0 | 81.0 | 100.0 | 94.8 | 100.0 | 88.3 | Open |
Samson Oil & Gas Check Stub Englishpdf2 pp |
100.0 | 100.0 | 94.1 | 91.3 | 97.0 | 97.0 | 91.4 | 97.0 | Open |
San Francisco Telephone Directory 1907 Englishpdf16 pp |
96.6 | 99.3 | 94.0 | 99.2 | 0.0 | 15.9 | 92.1 | 0.0 | Open |
SEC EDGAR FedEx Credit Agreement (excerpt) Englishpdf2 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 96.0 | Open |
SEC EDGAR Precision Lease 4th Amendment (excerpt) Englishpdf3 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
Shanghai Desk Hong List 1941 Chinesepdf28 pp |
96.2 | 80.3 | 63.9 | 38.1 | 50.3 | 19.4 | 88.7 | 26.9 | Open |
Smartsheet Construction Invoice Englishpdf4 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 88.3 | Open |
Spanish Iberdrola Factura Electricidad Spanishpdf4 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 70.0 | 85.7 | Open |
Student's Hindi-English Dictionary 1895 Hindipdf14 pp |
98.7 | 80.5 | 32.1 | 25.3 | 17.0 | 13.3 | 74.7 | 6.3 | Open |
Supreme Court Digest 1917 Englishpdf25 pp |
98.5 | 90.7 | 70.8 | 26.6 | 56.7 | 0.0 | 83.1 | 0.8 | Open |
Taub Hebrew Tax Invoice Hebrewpdf1 pp |
95.5 | 90.9 | 100.0 | 100.0 | 90.9 | 90.9 | 0.0 | 86.4 | Open |
Tokyo Telephone Directory 1928 Japanesepdf3 pp |
85.6 | 64.8 | 65.7 | 44.7 | 0.0 | 6.1 | 53.1 | 0.0 | Open |
UICCP Packing List Carton Dimensions Englishpdf1 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 0.0 | Open |
University of Pittsburgh Filled Sample W-2 Englishpdf3 pp |
100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | Open |
Vince Sea Waybill Englishpdf2 pp |
80.0 | 73.3 | 98.2 | 61.5 | 98.2 | 100.0 | 92.7 | 98.2 | Open |
Zionist Strategy NGO Financials Hebrewpdf4 pp |
69.4 | 65.0 | 75.0 | 69.4 | 68.6 | 73.1 | 50.0 | 61.0 | Open |
Average across all 72 documents |
97.02 | 91.73 | 89.38 | 80.28 | 76.48 | 72.98 | 70.95 | 67.67 | See ranking |
Every document, answer key, and system prediction is hosted here — open any of them in the data explorer. To reproduce the scores independently, download the replication kit: the answer keys, extraction schemas, and the scorer, so you can re-run it and get these numbers back.
Corpus
Where these documents come from
Every document is a real file with a traceable public source: SEC filings, utility bills, government forms, published annual reports, and vendor sample documents. Provenance and licensing for each one is recorded in the replication kit. The corpus deliberately includes spreadsheets, XML, scans, photographs, and documents in a dozen languages, because the interesting part of this category is what happens on the hard tail, not on a clean English invoice.