AI Document Extraction: How to Verify It's Right

Before you let an AI read invoices, receipts, or permit forms and write the numbers into your system, you need a way to know it got them right without a person checking every document. The answer is not the model's confidence score. It is a deterministic cross-check the document already gives you: line items that sum to the printed total, a date that parses, an invoice number that matches an open purchase order. Run those checks on every extraction and route only the failures to a human. Accuracy is a per-field problem, so you verify it field by field.

Why a single accuracy percentage lies to you

Document AI vendors advertise 95 to 99 percent accuracy. That number is a field-level average across a benchmark of documents. It is not a promise about the invoice sitting in your inbox right now.

Two things go wrong when you treat the average as a guarantee. First, one field being wrong is not the same as the document being wrong. An invoice can have thirty correct fields and one incorrect total, and the total is the only field your accounting system actually cares about. Second, failure is more common than the headline number suggests. On the DocILE invoice benchmark, one research team measured a 26 percent natural failure rate across the 55 fields on a typical invoice. More than a quarter of documents had at least one field the model missed or misread.

So the useful question is never "how accurate is this model." It is "which of the fields on this specific document is wrong, and can I catch it before it hits the ledger." You cannot reconcile a bank statement against an average. You reconcile it against the actual numbers, one at a time.

The check the document hands you for free

Here is the part most extraction guides skip. Many business documents carry their own internal math and structure, and that structure lets the document verify itself. You do not need a second AI to grade the first one. You need to recompute what the document already states and compare.

Invoices are the clearest case. Line items sum to a subtotal. Subtotal plus tax equals the total. If the line amounts you extracted do not add up to the total you extracted, at least one of those fields is wrong, and you know it with certainty, no model involved. Researchers studying extraction from fiscal documents made the same point formally: the structure of fiscal tables, with totals at each level of the hierarchy, allows for internal validation of the extracted data. They turned those hierarchical relationships into multi-level validation checks. You can do the same thing with a few lines of code in your automation.

The pattern generalizes across field types. For each field you extract, write the one check that proves it.

Field type	Free verification check
Money totals	Line items plus tax must equal the stated total
Line amount	Quantity times unit price must equal the line amount
Dates	Must parse to a real date and fall in a plausible range
IDs (invoice, PO, permit number)	Must match a known record or a strict format pattern
Counterparty name	Must match a vendor or customer already in your system
Tax or percentage	Computed rate must match a rate you actually charge

None of these ask the model whether it is sure. They ask the arithmetic and the reference data, which cannot be talked into a wrong answer. This is the same principle behind not trusting a confidence number an AI prints in an automation: route on a signal you can check, not on the model's opinion of itself.

When there is no math to check

Some fields have no arithmetic and no matching record. A free-text description, a project name, a one-off address. For those, you have three tools.

Constrain to known values. If a field can only be one of a fixed set (a status code, an approved vendor, a real product SKU), reject anything outside that set outright. An off-list value is an error you catch without judgment.

Enforce formats. A date field must parse. A phone number must have the right digit count. Structured outputs from the model will guarantee you get a number where a number belongs, but that only fixes the shape. A validly formatted total of zero on a thousand-dollar invoice is still wrong, so formatting is necessary and never sufficient.

Read the document twice, independently. Extract each field on its own in one pass, then do a second pass that scans the whole document holistically, and flag any field where the two disagree. This works because the two reads fail in different ways. A field-by-field read tends to invent a plausible value when a field is missing. A whole-document scan tends to skip fields that are not visually prominent. Their disagreement is a signal in itself. A 2026 confidence-engine study built exactly on this idea, fusing cross-read disagreement with layout and image-quality signals, and reported strong results catching field errors on a 26 percent failure-rate invoice set. You do not need their full model to borrow the core move: two reads, escalate the conflicts.

Worked example: one invoice through the gate

An AI reads a supplier invoice and returns structured JSON: vendor, invoice number, date, an array of line items, subtotal, tax, and total. Before anything posts to your accounting system, it passes through a gate.

1. Structured output is valid, so every field is present and correctly typed.
2. The sum of line item amounts equals the extracted subtotal.
3. Subtotal plus tax equals the total.
4. The date parses and falls within the last 90 days.
5. The vendor matches a name in your approved supplier list.
6. The invoice number does not already exist in your ledger, so you are not paying it twice.

Pass all six and the invoice posts automatically. Fail any one and it goes to a human queue with the specific failed check attached, so the reviewer sees "line items sum to 1,240.60 but total reads 1,240.00" instead of a raw document and a shrug.

That last case is the whole point. Say the model misreads a line item as 1,240.60 when the paper says 1,240.00. No confidence score would have flagged it, because the model was perfectly confident about the wrong number. Check three catches it, because the parts no longer add up to the whole. You caught a field-level error with zero human review and no reliance on the model's self-assessment. The documents that survive the gate are the ones your books can trust.

What to do next

List every field you extract from a document type, and next to each one write the single check that proves it is right. Start with the documents that carry their own totals, invoices, statements, expense reports, and purchase orders, because they verify themselves and are the highest-value to automate first. Anything with no internal check goes through known-value lists, format rules, and a two-read comparison.

This is how we build document pipelines. Field paperwork flowing off a jobsite in field2flow.geninfos.com and permit records in ma-permits.geninfos.com both run verification-first, with humans on the exceptions rather than every record. If you are wiring AI document extraction into your operations and want the gate designed before the data reaches your system of record, our workflow automation work is built around exactly that, and you can tell us what you are extracting.