LLM Document Extraction Failure Modes in Production

LLM document extraction pipelines don't fail the way most engineers expect them to. They don't throw exceptions, they don't return malformed JSON, and they rarely trip an alert. Instead, they keep running, keep validating against the schema, and keep producing answers that are wrong in ways nobody notices until the wrongness has already spread through six weeks of business decisions. That is the actual failure mode to study, because the gap between a prototype that works in a demo and a system that works in production is not a matter of scale. It is a matter of an entirely different taxonomy of failure that most teams never learn to look for.

The canonical case, documented on tianpan.co, involves a document extraction pipeline built on JSON mode that passed QA with almost no parse errors. Six weeks in, every risk assessment in the corpus had quietly been marked "low." The JSON was valid. The field names were correct. The answers were wrong. As the writeup put it, the pipeline had been confidently lying in a schema-compliant format for weeks. Nothing in the monitoring stack caught it, because nothing in the monitoring stack was built to catch it. Parse error rates stayed near zero, schema conformance checks stayed green, uptime stayed at whatever number ops teams like to see. The green readings held steady while the business data they were supposed to reflect was simply false.

Recent academic work backs up the pattern. A 2025 taxonomy from Vaishali Vinay, posted to arXiv (2511.19933), catalogs 15 hidden failure modes in real-world LLM applications and argues that these patterns differ fundamentally from how traditional ML models break. Most of the 15 are invisible to the benchmarks teams already run. That's the throughline for everything that follows: these are system-engineering failures, not model failures, and fixing them takes a system-level response, not a better prompt.

The structural confusion at the root of most silent failures: conformance is not correctness

JSON mode, introduced by OpenAI in November 2023, guarantees exactly one thing. The output is syntactically valid JSON: no unclosed brackets, no trailing commas, no stray prose wrapped around the response. It says nothing about whether the right fields are present, whether the values are the correct type, or whether the extracted data reflects reality. Conformance and correctness are different axes entirely, and a lot of production pipelines are built as though they're the same thing.

Strict mode, which OpenAI shipped in August 2024, tightens conformance further, pushing schema and syntax failures below a 0.1% rate in many cases. That sounds like a solved problem. That sounds like a solved problem, but it isn't. A system with perfect schema enforcement can still be wrong 30% of the time, per tianpan.co, because strict mode enforces shape, not substance.

Some of the resulting failure patterns are almost mechanical. A confidence field that always reads high, regardless of input quality, isn't lying exactly, it's just that nothing in the schema constrains what that field is supposed to measure, so the model defaults to a number that looks reassuring. Field ordering matters more than most schema designers assume: put the answer field before the reasoning fields, and the model has to commit to a conclusion before it's generated any reasoning to support it. That's a structural flaw baked into the prompt, not a model shortcoming. Required fields cause a related problem. When a required field has no good answer given the input, the model doesn't leave it blank or flag uncertainty. It fabricates something plausible, because the schema itself is asking for hallucination whether the designer meant it to or not.

None of that appears in a conformance check. Any monitoring system built only to confirm the JSON parses is blind to the most expensive class of failure a pipeline can produce.

The upstream failure that LLMs inherit silently: what happens before the model sees the document

The widely cited 98 to 99% OCR accuracy figure describes high-quality printed text on clean white paper, scanned under good conditions. Almost no enterprise document corpus looks like that. Real documents are faxed, photocopied, photographed on a phone in bad light, or scanned on machines nobody's calibrated in years. In production, OCR accuracy on real enterprise batches is commonly in the 80s or low 90s, sometimes worse, and that's well below the threshold needed for the kind of touchless, no-human-review processing most pipelines are sold on.

Research documented in arXiv 2606.24420 shows that most extraction errors in document processing are caused by the document, not the model. A frontier LLM handed unreadable source material doesn't hedge. It generates high-confidence tokens about noise, because nothing in its training tells it to recognize garbage input and say so. LLMs don't warn you when they're reading garbage. They just hallucinate from it, confidently, and hand you an answer that looks exactly like a correct one.

Layout compounds the problem. Multi-column documents get read out of order. Table rows lose alignment partway through. Scanned documents with no embedded text layer return empty strings instead of an error, which is worse than an error because nothing downstream knows to stop. Basic extraction tools tend to break on anything with nested tables, embedded figures, or unusual formatting, and the choice of preprocessing tool becomes an architectural decision with real accuracy consequences, not a minor implementation detail. A tool like PyPDF, for instance, works fine on digital PDFs and can pull text out of scanned PDFs that already carry an OCR layer, but it returns nothing useful on an image-only scan with no text layer at all.

Alan, the French insurance company, found something worth noting in production: combining OCR text with the document image outperformed either input alone. Text-only extraction handled most documents fine, but it left a tail of failures that only showed up once the image was fed in alongside the text. That tail is exactly the kind of thing aggregate accuracy numbers hide.

The taxonomy of failures that surface after deployment, not before

The arXiv 2511.19933 taxonomy names 15 hidden failure modes across real-world LLM applications. The ones that matter most for document extraction cluster into a few recognizable shapes.

Multi-step reasoning drift is one. Errors accumulate across pipeline stages, each one slightly off, until the final extraction is badly wrong without any single stage having tripped an alarm. This gets particularly dangerous in pipelines that chain classification, extraction, and validation in sequence without checking against ground truth at any intermediate point.

Context-boundary degradation is another. Accuracy drops off at document boundaries and toward the far end of long contexts, where the model's attention to earlier material gets less reliable. In practice, schema violations tend to cluster at two spots: very long outputs, where the model drifts in the final segments, and deeply nested schemas, where constrained decoders lose track of how deep they actually are.

Version drift deserves its own mention because it's the hardest one to catch by design. Providers update models on their own schedule, with no warning, and that update produces an output distribution shift while the schema keeps validating right through the change. Picture a risk pipeline that used to classify 40% of cases as "moderate." After a silent model update, that number might drop to 25% moderate and 15% high, both of which are perfectly schema-valid outputs. Monitoring shows zero errors. Business metrics drift for weeks before anyone notices. Structure stays intact, semantics change, and conformance monitoring never sees it, which is why it's worth calling this schema-shaped drift specifically.

Cost-driven performance collapse is the failure mode that budget owners cause without meaning to. Downgrading to a cheaper model, or shortening a prompt to save tokens, looks like an operational tweak. Downgrading to a cheaper model, or shortening a prompt to save tokens, looks like an operational tweak, but it isn't, since it's a quality change wearing an operations costume. It's a quality change wearing an operations costume. A pipeline that runs on a premium model in evaluation and a cheaper model under production load isn't the same pipeline, and comparing accuracy between the two versions is a comparison nobody's actually running.

Two more round out the list. Incorrect tool invocation occurs in agentic extraction setups, where a tool call looks syntactically right, correct structure, correct format, but calls the wrong tool or passes the wrong field to the right one, with no exception raised anywhere. And latent inconsistency only becomes visible at the batch level: individual extractions that look plausible on their own turn out to contradict each other across documents in the same corpus, a failure that per-document review will never catch.

Infrastructure failures that compound the model-layer taxonomy

An analysis published through IEEE Computer Society, written by Wrick Talukdar in August 2026, makes a point that cuts against how most teams diagnose IDP problems. Failures at scale rarely trace back to the extraction capability itself. They trace back to systemic production issues that model evaluation never touched in the first place.

Rate limits are the most immediate of these. An IDP system chewing through 10,000 invoices, each needing one to three API calls, can blow past a provider's per-minute rate or token thresholds without much effort, and what follows is 429 errors, retry storms, and backpressure cascading through the rest of the system.

Cost doesn't scale linearly either, which surprises people who budget off average document size. A one-page invoice might run around 1,500 input tokens. A 40-page contract can run 60,000 tokens or more. Without per-document budget controls in place, one shift in the mix of document types coming through the pipeline can inflate monthly spend by an order of magnitude, and nobody notices until the invoice from the provider arrives.

Schema stability across providers is its own quiet hazard. A JSON schema that one model produces reliably can come out subtly malformed when the same prompt gets routed to a different provider, or even a different version of the same model from the same provider. Feed that into a typed database schema, and even a minor structural inconsistency causes a downstream failure that has nothing to do with the extraction logic itself.

And observability gaps tie all of it together. Weak telemetry leaves teams unable to tell whether a latency spike, an accuracy regression, or a cost anomaly is happening until it's already caused an SLA violation. The failure, in that case, isn't really in the extraction. It's in the inability to see the extraction failing while it happens. The gap between prototype and production, at bottom, is an infrastructure orchestration problem. The model works. The system wrapped around it often doesn't.

Why standard benchmarks and QA processes are structurally blind to these failures

Existing benchmarks measure knowledge and reasoning reasonably well. They offer almost nothing on stability, reproducibility, drift over time, or how a model behaves once it's wired into a real workflow, a gap the arXiv 2511.19933 taxonomy calls out directly.

Part of the problem is a metric mix-up that's become so common it barely registers as a problem anymore. "99% accuracy" in document extraction almost always refers to character accuracy, a measure of whether individual characters were transcribed correctly. That number tells you almost nothing about whether an extracted invoice total is correct, and the gap between character accuracy and field accuracy typically runs 15 to 20 percentage points. A tool advertising 99% character accuracy can deliver field accuracy well below that figure by the gap described on real invoices, and both numbers are technically true.

Prompt-only JSON extraction, without constrained decoding, fails on roughly 8 to 15% of calls in production systems processing millions of requests, a rate that stays invisible in a small QA batch and becomes very material at volume. Constrained decoding fixes some of that, but the failures don't vanish, they move. Parse failures turn into refusal responses, where a safety trigger fires and the model returns a refusal that crashes a downstream parser built to expect structured output. Even when constrained decoding works exactly as intended, models reasoning under that constraint show a 10 to 15% performance drop on complex tasks compared to free-form generation, and this is a failure mode that benchmark evaluations typically never reveal.

Confidence calibration rounds out the blind spot. An uncalibrated model can report high confidence on outputs where actual accuracy, at that same confidence threshold, is substantially lower, which makes any automation gate built on a confidence cutoff unreliable by default. A useful gut check: pull a sample of extractions that cleared the stated confidence threshold and manually verify them. If the confidence number and the accuracy number don't line up, the threshold isn't doing what it's supposed to do.

What a production-grade evaluation framework actually measures

Accuracy in document extraction isn't one number, it's a hierarchy: character accuracy, then field accuracy, then document accuracy, then confidence calibration, each one measuring something the others don't. Of these, document accuracy, the percentage of documents that came through with zero extraction errors at all, is the number that should actually govern automation decisions.

Weighted Overall Accuracy (WOA), described in arXiv 2608.01792, is one concrete way to build that up from the field level. It's a weighted average of per-field similarity scores across entity types, using normalized Levenshtein similarity for string fields and a tolerance-based comparator for numeric fields, with each field scored on a continuous 0 to 1 scale rather than a strict pass or fail. F1 works as a stricter complement to that: it applies a binary accept/reject threshold per field, counts only exact or near-exact matches, and gives no partial credit. Running both side by side matters, because WOA's partial-credit scoring can mask a systematic error that F1 would catch outright.

The confidence-coverage trade-off is where the numbers get genuinely useful. Research under the EXTRACTCONF name (arXiv 2606.24420) shows that at 80% coverage, meaning 80% of documents get processed automatically, a well-calibrated confidence system reaches 99.1% accuracy on those automated cases, a 25.8 percentage-point jump over the 73.3% base rate without calibration. The AURC metric, Area Under the Risk-Coverage curve, turns that relationship into a single number that's actually usable in a production dashboard.

Calibration isn't uniform across field types, either. Calibration varies by field type, and an uncalibrated model can report high confidence on outputs where actual accuracy is substantially lower, which argues for routing thresholds that vary by field type rather than a single confidence cutoff applied everywhere.

Aggregate accuracy numbers also hide the tail. Performance should be broken out by document condition: digital PDF versus scanned, skewed and noisy scans, stamps and handwriting, language, templates from a supplier the system has never seen before. That's where production documents actually live, and it's where aggregate numbers go to hide their weak spots. Pair that stratification with business-facing KPIs, straight-through processing rate, manual review rate, false auto-approval rate, and continuous benchmarking against a fixed canary set to catch version drift before it raises costs in the business numbers. The canary set, in fact, is the only layer of monitoring built specifically to catch schema-shaped drift before it costs anything.

For a sense of the ceiling, mid-to-late 2024 commercial LLMs on metadata extraction tasks scored F1 between 0.91 and 0.97 on zero-shot prompts, representing strong performance on those tasks, according to research indexed in NIH PMC 12132202. That's a useful reference point for calibrating expectations about what a production system should be able to achieve.

The validation architecture that catches what schema enforcement misses

A validation architecture that actually holds up in production runs three distinct layers, per tianpan.co, and each one catches something the others miss entirely.

The first layer is generation-time enforcement: native structured outputs or function calling that guarantee schema conformance at the moment the model generates its response. This wipes out the bulk of syntactic failures and is mature enough now that it should be the default anywhere structured output is required. It does not catch semantic correctness, and it was never meant to.

The second layer sits at the application boundary. Every structured output should pass through a validation step, Pydantic in Python, Zod in TypeScript, before any downstream code touches it. This catches things generation-time enforcement misses: outputs truncated because they hit a token limit, or type coercion edge cases that slip through otherwise. Still, this layer doesn't catch semantic correctness either. A value can sail through every type check and still be flat wrong.

The third layer, semantic and business-rule validation, is where most of the real risk actually lives, and it's also the layer most teams underinvest in. This is where cross-field consistency gets checked: do extracted fields agree with one another across the document. It's where cross-field consistency checks live, flagging values that conflict with other fields in the same document. And it's where confidence scores get validated against actual accuracy at each threshold, so the routing boundary reflects real performance rather than an assumed one instead of letting it pass downstream unchecked, which is the practical application of the EXTRACTCONF coverage-accuracy trade-off described earlier. This is the only layer built to catch the silent semantic failures the first two are structurally incapable of seeing.

Alan's production experience is a useful illustration of how the input layer and the validation layer interact. The team initially fed the pipeline only the OCR Markdown transcription. Adding the document image back in, alongside the OCR text, outperformed either input alone and cut down the failure tail that text-only extraction left behind, leaving the validation layer with fewer genuine errors to catch in the first place. On that architecture, Alan reached 70% automation on French insurance documents, a concrete production benchmark for what a properly instrumented extraction pipeline can achieve on a real, messy document corpus rather than a curated benchmark set.

The build-vs-buy calculation once the full failure taxonomy is on the table

Once all of this is laid out, upstream OCR degradation, schema-compliant hallucination, context-boundary drift, version drift, cost-driven collapse, infrastructure failures that have nothing to do with model quality, the build-vs-buy decision stops being about whether a team can write a prompt that extracts fields from a PDF. Almost anyone can do that in an afternoon.

The real question is whether an organization can build and maintain all three validation layers, run continuous canary benchmarking against version drift, calibrate confidence thresholds separately by field type, and instrument the infrastructure layer well enough to catch rate-limit saturation and cost non-linearity before they become incidents. That's a standing engineering commitment, not a one-time build. Teams that treat document extraction as a solved problem because the demo worked are the same teams that discover, six weeks later, that every risk assessment in the corpus has quietly been marked "low."