Retrieval-augmented generation has a part nobody wants to work on. You cut documents into chunks, embed them, get some back, hand them to a model. The retriever and the model get the attention. The cutting gets a chunk_size parameter and a shrug.

That parameter does more damage than it looks like it does. Cut a manual every 1,400 characters and you get passages that start mid-sentence, that no longer sit under any heading, and that cannot tell you which of your documents they came from. The model receives them as equals and reasons over them anyway. Nothing the retriever does well can exceed the quality of what you put in the index.

The prevailing answer has been more room — longer contexts, so you can afford to send more of the wrong thing and let the model sort it out. That does help. It is not the same as fixing it, and it is an expensive way to avoid the question.

pagespeak works at the other end. A retrieved unit should be a section: cut where the document’s own hierarchy says to cut, stamped with the document and chapter it belongs to and a hash of the file it came from, and carrying what the figures said — because a great deal of what a manual actually knows was never text.

What it does

Get the chapters back when the converter flattened them

Marker gives Chapter 1 and its 1.1 subsection the same #### level, so the whole document comes out as a flat pile of peers — no chapter above anything, and nothing for a breadcrumb to name. Docling has the same problem from the other end: it caps at two levels by design. --normalize-headings-mode llm_full sends the headings plus the body text to a model and re-levels them. One call per document, cached, and it will not drop a heading that owns children or owns its own prose.

Know which manual a retrieved chunk came from

Split into one file per section and every file opens with a > ↑ Doc / Chapter / Parent line and carries its own doc_id and source_sha256. So a chunk that comes back from a search can say which manual and which section it is, and you can check it still matches the PDF it was built from. In a 70-manual index, an answer cites the manual behind each step instead of blending three of them together.

Search text that exists only inside a figure

The vision pass writes a description into each image’s alt text, and a Mermaid block when the figure is a flow diagram. In the worked example, “which organ system includes the spleen?” comes back with Figure 1.5 as the top hit, 0.93 similarity — the word appears nowhere in the prose, only in the alt text the vision pass wrote. Descriptions are keyed by image hash, so re-converting the document never pays for them twice.

Find the conversions that came out broken

Marker can hand back a table crushed into a single <br>-joined mega-cell, or a section file that is an empty shell, and nothing about the run will have failed. pagespeak audit conversions/out reads the finished output for ten shapes like those — U+FFFD encoding damage, image links whose files are not there, duplicate headings — and exits 1 if it finds any. Read-only, no model calls, nothing to pay for. --summary-only gives per-check totals across a whole corpus.

Fix one mangled table without re-running the PDF

When Marker jams a whole grid into one cell, pagespeak repair-tables locates that PDF page, re-reads it with Docling, and splices the clean grid in. Prose, headings and the vision cache stay as they were, and it leaves the original alone if Docling’s grid would drop a value. --dry-run prints which tables it would touch.

Re-run one stage, not the whole 500-page PDF

Each of the seven phases writes its own checkpoint, so --from normalize --stop-after normalize re-runs exactly that stage against frozen upstream output. A parallel ingest that dies partway resumes from manifest.json and redoes only the unfinished chunks; convert refuses to build a document while any chunk is still failed.

What that looks like in practice

The clearest case is the one where the answer was never text to begin with. A washing machine puts a code on the display. The people who built it wrote down exactly what to check, in what order, with the voltage to measure and the resistance range that condemns the part — and then printed it as a flowchart, which is to say a picture, which is to say something no index could reach.

what the model already knew what the manual knew
you askThe washer is throwing an error and will not fill. What do I actually check?answered from general knowledgeCheck that the water is turned on and the hoses are not kinked. Clean the inlet screens.Try unplugging it for a minute to reset the control board. If it keeps happening, the inletvalve or the main board may need replacing — a technician can tell you which.answered from the service manual in the corpuscheck the air chamberand its tubeinlet valveerror shownhold WASH/RINSE +DELAY WASH — is waterfrequency under 24.6 kHz?is resistance across thevalve terminals0.8–1.2 kΩ?replace theinlet valve assemblyyesnonoOn the page that tree came from, none of this is text. It is a photograph of a flowchart.The vision pass read the picture and wrote it back out as a diagram a model can walk, one branch at a time.193 diagrams recovered this way from one appliance corpus · 450 from a studio corpuslive retrieval · 2026-09-07 · manufacturer unnamed; three nodes of a longer tree, shown to illustrate the conversion
The first answer is not wrong. It is what anyone can tell you, and it ends by suggesting you call somebody. The second names the buttons that put the machine in test mode, the frequency that separates a supply problem from a sensor problem, and the resistance range that condemns the valve. It is the manufacturer’s own troubleshooting tree — and until something read the picture it was printed in, no search could reach a word of it.

The harder test is a question with no single right source anywhere. I asked one I actually needed answered: how to keep cymbals out of a close tom mic, on the desk while tracking and again in the mix. Answering it properly meant a microphone, a console, a DAW and three plugin makers — sixteen sources, none of which is permitted to mention any of the others.

you askHow do I make the gate open on the tom and not on the cymbals next to it?The technique is one idea: filter what the gate listens to, rather than what it passes. Five boxes in thechain can do it. Every one of them calls it something else.wherefilter the detectorhear what the detector hearsthe consolekey filter — frequency, slopetap the encoder: key source to solo busthe DAW’s own gateFilter, then High/Low CutoffMonitor buttona gate pluginband suppression, draggable crossoversa target profile for tomsa dynamic EQtriggering: Band → Free, custom rangeaudition buttona channel-strip gatekey from one of its own bands—Four of the five will play you the filtered trigger. That is the whole trick, and it is the partnobody writes down, because no manual is allowed to talk about anybody else’s box.It also read my own session notes and told me what I already hadThe gate plugin was already the first insert on both tom channels. Gates were already running.The answer was not “buy this” — it was which control, on which of the things already in front of me.assembled from 16 sources across six makers · live retrieval · 2026-09-08
Six manuals had to agree before this was answerable, and none of them is allowed to mention the others. The console’s manual documents its key filter and never says the phrase “side chain”. The DAW’s gate documents a Monitor button whose whole purpose is auditioning a trigger, and never says the word “key”. A reader who only had one of them would conclude their box was the only one that could do this.

I ran it as a blind comparison: three agents answered from what they already knew, three answered from the corpus, and two judges graded both against 37 facts with sources attached. Unaided, 10 of 33. With the corpus, 25. The unaided answers were fluent and specific and named menu paths, in three different products, that do not exist — which is the failure mode that matters, because nothing in the writing warns you.

The parts of a manual you need when you are actually fixing or connecting something — diagnostic trees, routing, pinouts, front-panel layouts, menu trees — are the parts that were never text. In one corpus 15,539 figures were given a written description, adding 4,976,794 characters that did not exist in the extracted text, and 450 flow diagrams came back out as Mermaid.

Scale is the part that worries people, and it should. Ten questions naming a specific piece of gear, put to an index of 104,922 documents, returned the correct manual first every time — and the correct section within it about two thirds of the time. Landing in the right book on the first try is what makes the second search worth running: the section that comes back says which manual and which chapter it is, so an agent can tell it has the right equipment and the wrong page. A fixed-size chunk tells it neither.

The pass costs money once. Descriptions are cached against the image’s perceptual hash, so the same figure is never analysed twice: not on a re-ingest, not when the model changes, not when it turns up in another document. pagespring keeps the images on disk so the fetch does not repeat either.

None of this is knowledge a model can be trained into having, because it is not general knowledge. It is what your specific machine does, printed once, by the people who built it.