Financial RAG Is Not Just Document Search
Filings, market data, calculations, and provenance each want a different system layer.
“Financial RAG” usually gets built as document search with a finance vocabulary: chunk the filings, embed them, retrieve the top-k, ask the model. It demos well and answers the wrong shape of question, because the thing a financial analyst actually needs is rarely a passage — it’s a number that is correct as of a date and traceable to its source.
Four layers, not one retrieval
The information has at least four distinct textures, and flattening them into one vector store is where accuracy leaks:
- Filings — narrative and tabular text, versioned, where the exact wording and the reporting period matter.
- Market data — dense time series, where the question is almost always “as of when,” not “which paragraph.”
- Calculations — derived quantities that must be reproducible and show their inputs, not retrieved as if they were prose.
- Provenance — the layer that lets any figure above be walked back to the raw fact it came from.
Retrieval is the right tool for exactly one of these. The other three want a database, a calculation ledger, and an evidence trail. A system that respects the difference can say “revenue was X, as of this filing, computed this way” — and let you check every clause. A system that treats all four as text can only say “here’s a passage that mentioned revenue.”
The interesting engineering is in the seams between these layers, which is precisely what a single retrieval step erases.