Yardstick Labrigorous software benchmarking
Head to head

Claude Sonnet 5 vs Gemini 3.5 Flash

Two raw models with opposite profiles. Gemini 3.5 Flash is cheaper by a wide margin and slightly better on single-page scans and photos. Claude is nineteen points ahead overall because it doesn't collapse on long documents, where Gemini posted the worst average in the test and had runs fail outright. Neither comes with a document product.

Direct LLM baseline
91.73%
2nd of 8 on accuracy · 7.2/10 weighted
vs
Direct LLM baseline
72.98%
6th of 8 on accuracy · 6.2/10 weighted

Both rows are a general model with a file and a field list and nothing else. On one-page documents Gemini was the better of the two, second only to DocuPipe on forms, receipts, and photographed or rotated scans, and it returned more documents completely correct than any system in the test. If your corpus is single pages, Gemini is the pick and it's the cheaper one.

Everywhere else Claude is ahead, and on long documents the gap is a chasm. Gemini posted the lowest average of any system on registers, catalogs, and directories of six pages or more, and several runs failed outright. Claude worked through the same documents and returned most of the entries. On multi-page statements and on non-English documents Claude was also clearly ahead. Only on Hebrew and Arabic layouts did Gemini beat Claude, and by a fair margin.

That's the resilience story too. Neither has anything to tune. Claude returned nothing empty. Gemini is excellent on one page and empty on twenty, and the only variable is length. A pipeline built on it has to route long documents somewhere else.

On price Gemini wins clearly: $1.50 and $9 per million tokens, well under a cent per page, plus a free tier for development. Claude is $2 and $10, a few cents per page. Both offer batch discounts. Neither has a plan or a production free tier.

On product there's nothing to compare, because neither has one. Both have strong general APIs with native PDF input, Files APIs, and structured outputs. Gemini accepts up to 1,000 pages a request, Claude 600, and Claude adds citations. Neither has a dashboard, webhooks, document jobs, review, or retention controls, and Gemini's data policy differs between its free and paid tiers, which is worth reading before you use the free one.

So Gemini for high-volume single-page work where cost dominates. Claude for anything with length or language variety. If you can't predict your documents, Claude.

Choose Claude Sonnet 5 if

Mixed or multi-page corpora, non-English documents, and any pipeline where a long document must not come back empty.

Choose Gemini 3.5 Flash if

High-volume single-page work such as receipts and photographed forms, where cost per page is the deciding factor.

Scorecard

How it scores, area by area

The accuracy score comes from the benchmark. Every other score is ours, with the reason written next to it. We use the same weights on every review and comparison.

AreaClaude Sonnet 5Gemini 3.5 Flash
Accuracy35% of total
9.2Ranked 2nd of 8 systems on the benchmark.
7.3Ranked 6th of 8 systems on the benchmark.
Resilience15% of total
7.0No settings to tune and nothing returned empty, but a few ordinary forms came back badly misread with no warning, and there is no layout stage to catch them.
3.0Several runs failed outright and the long-document average was the lowest in the test; excellent one day and empty the next depending on page count.
Price15% of total
9.0$2 and $10 per million tokens; a few cents per page at most.
10.0$1.50 and $9 per million tokens plus a free tier; well under a cent per page.
Getting started10% of total
3.0No document product. You build upload, review and results tooling yourself.
4.0No document product, though the free tier makes trying it trivial.
API and integration10% of total
5.0Excellent general SDKs, native PDF input, a Files API, batch mode and structured outputs; no document jobs, webhooks or pipeline.
5.0Strong general SDKs, the largest PDF page limit of the direct models, a Files API and structured output; no document jobs or webhooks.
Data control7.5% of total
5.0Platform policy only; zero retention for eligible organisations. Nothing document-specific.
4.0Platform policy only, and it differs between the free and paid tiers.
Compliance7.5% of total
6.0Platform certifications apply to the API; your pipeline is your responsibility.
6.0Platform certifications apply to the API; your pipeline is your responsibility.
Overallweighted
7.2weighted score out of 10
6.2weighted score out of 10
How each area is scored
  • Accuracy (35%). The benchmark result, on a 0 to 10 scale.
  • Resilience (15%). Does it just work? Fewer modes to choose, no tuning to reach the ranked result, no runs that fail outright, and no documents that come back quietly empty. Informed by the benchmark, judged by us.
  • Price (15%). Published cost per page for parse plus extraction, cheapest scoring highest. Unpublished pricing is marked down.
  • Getting started (10%). How far you get without a sales call: a dashboard, a free tier with no card, help authoring schemas, a test environment.
  • API and integration (10%). Webhooks and which tier they are on, SDKs, no-code connectors, whether the API is built around document jobs at all.
  • Data control (7.5%). Deletion controls, scheduled retention, zero-retention availability and on which tier, choice of region.
  • Compliance (7.5%). Certifications stated on public pages: SOC 2, ISO 27001, HIPAA with a BAA, GDPR, private deployment options.
The detail

By document type

Average accuracy on each group of benchmark documents, both systems on the same files. Every document and score is on the evidence page.

Document typeClaude Sonnet 5Gemini 3.5 Flash
Statements, filings, reports2 to 5 pages · 21 documents92.3% Good87.5% Good
Short forms, invoices, receiptsone page · 30 documents91.7% Good94.1% Good
Long registers, catalogs, directories6 pages and up · 21 documents91.3% Good28.3% Weak
Non-English documents12 languages · 25 documents85.2% Good68.9% Mixed
Scans and photosimages, not PDFs · 7 documents80.1% Good95.5% Strong
The detail

Price, product, and policies

Full reviews: Claude Sonnet 5, Gemini 3.5 Flash.

 Claude Sonnet 5Gemini 3.5 Flash
Pricing modelPer token through the Anthropic API. No document-specific product or plan.Per token through the Gemini API. No document-specific product or plan.
Entry price$2 per million input tokens, $10 per million output tokens.$1.50 per million input tokens, $9 per million output tokens on the paid tier.
Cost per pageDepends on page size and how much you ask it to return; typically a fraction of a cent to a few cents per page.Depends on page size and output length; typically well under a cent to a couple of cents per page, the cheapest of the direct models.
Free tierNo ongoing free tier for API use.Yes. A rate-limited free tier for development.
Pay as you goYes. Pure usage billing.Yes. Pure usage billing.
DashboardNo document product. A console workbench for prompts; upload, review and results tooling is yours to build.No document product. AI Studio for trying prompts; upload, review and results tooling is yours to build.
WebhooksNone for document jobs. You run the pipeline.None for document jobs. You run the pipeline.
API and SDKsOfficial SDKs in Python, TypeScript, Java, Go, Ruby, C# and PHP. PDF input up to 600 pages per request, a Files API, batch processing at reduced cost, structured outputs and citations.Official SDKs in Python, JavaScript, Go and others. PDF input up to 1,000 pages or 50 MB, a Files API with 48-hour storage, structured output.
Data retentionGoverned by Anthropic's API data policies; zero-retention arrangements exist for eligible organizations. Nothing document-specific.Governed by Google's API data policies, which differ between the free and paid tiers. Nothing document-specific.
ComplianceAnthropic's platform certifications apply to the API; your pipeline around it is your responsibility.Google's platform certifications apply to the API; your pipeline around it is your responsibility.

We checked these against the vendors' own pricing and documentation pages on 2026-09-11. Sources: Claude Sonnet 5: platform.claude.com; Gemini 3.5 Flash: ai.google.dev. Prices change, so confirm before you budget.

Questions

Frequently asked

Which is better for document extraction, Claude or Gemini?

Claude overall, 91.73% to 72.98% in our benchmark, because it handles long documents and Gemini doesn't. On single-page documents and scans Gemini was slightly ahead and costs a fraction of the price.

How much cheaper is Gemini than Claude?

Gemini 3.5 Flash is $1.50 per million input tokens and $9 per million output, against Claude Sonnet 5 at $2 and $10, and Gemini has a free tier for development. Both are cents per page or less.

Scores are macro-average field accuracy from the DocuBench 0.2.0, 72-document run. Every document, answer key, and raw extraction is open in the explorer; the scoring rules are on the methodology page, and the replication kit re-runs the scorer. A document a system could not process scores zero rather than being dropped.