Multiple representations

The first design decision is what to store. The naive answer is chunks and embeddings. The right answer is every representation that a consumer might need, linked together, with the full source at the root.

Raw source bytes, preserved immutably, so that anything derived can be regenerated. The normalized canonical document — the structural tree of sections, blocks, tables — that every downstream process operates on. Sections and chunks at multiple granularities, each pointing at the tree nodes it came from. Summaries at document, section, and topic level. Extracted entities with their normalized identities, and the relationships stated between them. Annotations: classifications, sensitivity labels, quality flags, human review notes. Embeddings for chunks, sections, and summaries. And the metadata that ties them together: source, version, effective dates, owner, provenance.

The reasons to store all of this rather than only the compressed forms:

Different questions need different representations. A "what is our policy on X" question is answered from a summary. A "what exactly does clause 14.3 say" question needs the chunk. A "walk me through the whole agreement" question needs the source. A "which documents mention this counterparty" question needs the entity index. Storing one representation forces every question through it.

Cost follows representation. Serving a summary costs a few hundred tokens; serving the source costs tens of thousands. A knowledge base that can serve the cheapest representation that answers the question is a knowledge base that scales economically. That is the whole basis of progressive retrieval, and it is impossible without multi-representation storage.

Auditability requires the chain. When a consumer asks "where did this claim come from," the answer has to trace from the answer to the chunk to the tree node to the page to the source bytes. Each link in that chain is a representation you chose to keep.

Regeneration requires the root. When the chunking strategy changes — and it will — you regenerate chunks from the canonical tree. When the parser improves, you regenerate the tree from the raw bytes. If the raw bytes are gone because someone thought the embeddings were the asset, you are re-ingesting from source systems that may no longer have the original.

The incremental storage cost of keeping everything is modest. Text is small; embeddings are the largest component and they are needed regardless. The cost of not keeping everything is paid in re-processing and in questions that cannot be answered.