Part 1 · Topic 1 of 8
Parsing
Parsing is where the quality is lost
2 min read
Start with the unglamorous truth: a large fraction of retrieval failures are parsing failures wearing a disguise.
A PDF is not text. It is a set of drawing instructions — place this glyph at these coordinates — and text order, paragraph boundaries, table structure, headers, footers, and reading order all have to be reconstructed. Naive extraction reads a two-column page left to right across both columns and produces interleaved nonsense. It flattens a table into a run of cell values with no idea which number belongs to which header. It duplicates the running header on every page into the body. It drops the text in a scanned page entirely because there is no text layer, and nobody notices because the pipeline reported success.
Word documents are better but not clean: tracked changes, comments, nested tables, text boxes anchored outside the flow. HTML carries navigation chrome and boilerplate that outweighs the content. Email threads quote themselves recursively. Slide decks have a text layer that bears little relation to what the slide communicates. Transcripts have speaker turns and timestamps that are structure, not noise, and are usually stripped as noise.
The consequence is that the model receives a degraded version of the document and produces a degraded answer, and the failure gets attributed to the model.
I now treat parsing as a stage with its own quality metrics. Parse fidelity — does the extracted structure match what a person sees on the page — is measurable on a sample. Table recovery rate, reading-order accuracy, header/footer suppression, OCR confidence distribution on scanned pages: none of these are hard to measure, and all of them are routinely unmeasured.
On tooling, the landscape has moved fast. The choice is between three families, and the right answer is usually a routing layer over more than one of them.
| Approach | What it does well | Where it fails | Cost profile |
|---|---|---|---|
| Rule-based text extraction (pdf text layer, DOCX XML, HTML readability) | Fast, deterministic, cheap; fine for born-digital single-column prose | Tables, multi-column layouts, scans, figures, anything where layout carries meaning | Negligible |
| Layout-aware document models (open-source layout/structure parsers producing a document tree with tables, headings, reading order) | Recovers structure; handles multi-column, tables, headings; produces a canonical tree | Complex nested tables, handwriting, poor scans; needs OCR fallback; heavier to operate | Moderate compute, no per-token cost, self-hostable |
| Vision-language models reading rendered pages | Highest ceiling on hard layouts, forms, figures, mixed content | Expensive per page; non-deterministic; hard to audit; can hallucinate cell values | High, per-page model cost |
The routing rule I use is simple. Classify the document by format and difficulty first (born-digital vs. scanned, single-column vs. complex layout, table density, page count). Send the easy majority through the cheap deterministic path. Send complex layouts through a layout model. Reserve vision models for the pages where the layout model reports low confidence or where the document class is known to be hard — forms, dense financial tables, diagrams with embedded text. Log which path each page took. The cost difference between "vision model on everything" and "vision model on the 5% that needs it" is an order of magnitude, and the quality difference is often nil.