Financial Balance-Sheet RAG: Building Citation-Grounded Document Intelligence

BLUF: FinBizInfo is a financial document RAG workflow built around traceability. It uses OCR, retrieval, structured extraction, ratio validation, and citations so an analyst can inspect the source behind every material claim instead of trusting a fluent summary.

Key takeaways

• Financial PDFs mix narrative text, tables, footnotes, and scanned pages, so extraction quality must be measured before generation quality.
• Retrieval and structured extraction serve different jobs: one finds evidence, while the other normalizes facts for calculation and comparison.
• Every ratio, risk flag, and auditor observation should retain a source document, page, and supporting passage.
• Evaluation needs factual consistency, citation correctness, retrieval coverage, and robustness to OCR noise.
• Human review remains the final control for ambiguous accounting language and consequential decisions.

The problem is not summarization

Financial document analysis looks like a natural language task until the first real filing arrives. A single package may contain machine-readable text, scanned appendices, wide tables, repeated headers, footnotes, accounting policies, auditor remarks, and values expressed in different units. A general-purpose chat interface can produce a plausible explanation while silently attaching the wrong period, entity, unit, or source page to a number.

That failure mode matters because a financial claim is only useful when it remains connected to evidence. “Current liabilities increased” is not enough. A reviewable system needs to show which filing, reporting period, table, line item, unit, and page support the statement.

FinBizInfo began as a financial balance-sheet RAG prototype for that narrower problem: reduce manual parsing while preserving a path back to the source document. The objective was not autonomous financial judgment. It was a faster, more consistent first-pass analysis that an analyst could verify.

Why a hybrid pipeline is necessary

The original retrieval-augmented generation paper frames RAG as a way to combine model knowledge with retrieved external evidence. Financial documents add a second requirement: important facts often need to be extracted into structured fields before they can be compared or calculated reliably.

The FinBizInfo workflow therefore separates five stages:

1. Ingestion and OCR. Detect whether each page has usable text, then apply OCR only where needed. Preserve page boundaries and document metadata.
2. Layout-aware chunking. Keep table rows, headings, notes, and nearby units together where possible. Naive fixed-length chunks can detach a value from its label or reporting period.
3. Retrieval. Search the document collection for passages relevant to the analyst’s question, with metadata filters for company, document type, and period.
4. Structured extraction and validation. Normalize line items, dates, currencies, units, and ratios. Cross-check generated calculations against extracted values.
5. Citation-grounded response. Return the answer with document, page, and supporting text so a reviewer can inspect the evidence.

This separation is important. Retrieval can find the right passage but still leave a model to misread a table. Structured extraction can normalize a balance sheet but miss the narrative note that explains an exception. The useful system combines both.

What the prototype analyzes

The prototype is designed around questions that recur in financial review:

• What changed in cash, debt, receivables, inventory, and working capital?
• Which ratios moved materially between periods?
• Are there liquidity, leverage, concentration, or covenant signals worth reviewing?
• What qualifications, emphasis-of-matter language, or risk disclosures appear in the auditor or management notes?
• Which extracted facts conflict across tables, narrative sections, or reporting periods?

For tabular facts, the system stores normalized values that can be checked with deterministic calculations. For narrative findings, it keeps the retrieved passage and page reference. The answer layer can then distinguish between observed values, calculated metrics, and model-generated interpretation.

Evaluation before deployment

A RAG system should not be evaluated only by whether its answer sounds correct. The RAGAS evaluation framework is useful because it separates dimensions such as faithfulness, answer relevance, and context relevance. For financial analysis, I extend that idea with checks tailored to documents and calculations:

• OCR accuracy: Are critical labels and values preserved on scanned pages?
• Retrieval coverage: Did the returned context include the table or note needed to answer the question?
• Citation correctness: Does the cited page directly support the generated claim?
• Numerical consistency: Can every ratio be recomputed from the displayed inputs?
• Period and unit integrity: Are year, quarter, currency, and scaling consistent?
• Abstention quality: Does the system say when evidence is missing or contradictory?

The project report records an approximately 60% reduction in manual analysis time during prototype use. I treat that as an internal workflow result, not a universal productivity claim. The more durable measure is whether reviewers can reach the source faster and catch unsupported output before it enters a decision.

Authoritative data and source controls

For public-company work, the SEC’s EDGAR data APIs provide filings and extracted XBRL facts without an API key. Structured XBRL data can serve as a validation source for values extracted from PDFs, while the filing remains the authoritative narrative record.

Source controls should also record document hashes, ingestion time, parser version, and page-level lineage. That creates a reproducible evidence trail when a filing is amended, a parser changes, or a result is challenged.

The governance principle is simple: the system should make unsupported confidence harder. The NIST Generative AI Profile provides a broader risk-management reference for documenting, measuring, and managing generative AI risks across the lifecycle.

Failure modes and guardrails

The highest-risk failure is not a visible crash. It is a polished answer tied to the wrong evidence. FinBizInfo therefore needs controls at each stage:

• reject low-confidence OCR on material tables;
• preserve and display source-page context;
• run deterministic ratio checks outside the language model;
• flag conflicting values rather than choosing one silently;
• separate observations from interpretations;
• require review for consequential recommendations;
• log the model, prompt, retrieval set, and calculation inputs used for each answer.

These controls do not make the system infallible. They make errors easier to detect, explain, and correct.

What I would build next

The next useful iteration is a benchmark set of difficult financial pages: rotated scans, multi-column tables, footnote-heavy disclosures, amended filings, and conflicting units. Each question would have an expected answer, accepted evidence pages, calculation inputs, and an abstention condition.

I would also compare PDF extraction against SEC XBRL facts where coverage overlaps, then report errors by document type rather than one aggregate score. That would reveal whether the system is dependable on common balance-sheet tables while still weak on narrative notes or unusual layouts.

Final thought

Financial RAG is valuable when it shortens the distance between a question and verifiable evidence. It is dangerous when it shortens the distance between a question and an unsupported answer.

FinBizInfo is built around the first goal: faster document intelligence with citations, calculations, and review controls kept visible.

Sources and implementation links

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• RAGAS: Automated Evaluation of Retrieval Augmented Generation
• SEC EDGAR Application Programming Interfaces
• NIST AI 600-1: Generative AI Profile
• FinBizInfo live application
• PolyRAG: related public RAG engineering repository

Related posts

• PolyRAG and governance-layer retrieval
• AI governance and privacy: building the trust layer
• Bayesian optimization on the Higgs graph: an honest evaluation study