ParseMyStatement v1.0 — How We Replaced PaddleOCR with Mistral OCR and Landed 90% F1 on Real Statements

ParseMyStatement v1.0 — How we replaced PaddleOCR with Mistral OCR (mistral-ocr-latest) and scored it against real ground truth

We built ParseMyStatement to answer one question most finance teams ask us daily: "Can you reliably turn this bank statement PDF into clean CSV, Excel, or JSON?" The honest answer at our v1.0 launch (March 12, 2026) was "yes for digital PDFs, and only sometimes for scanned ones." This post is the first in a five-part series where we show, release by release, exactly how we improve OCR accuracy — with the actual model names, the actual code, and the scores our engine achieves against hand-written ground truth.

If you are evaluating a bank statement OCR API, you want three things we now publish openly: the model name behind the extraction, the real benchmark methodology, and results measured on real personal statements — not synthetic test PDFs. That is the standard this series holds itself to.

The problem we started with

Bank statements come in two flavours, and they behave completely differently under the hood:

1. Digital / "native text" PDFs — the text layer is embedded and readable. pdfplumber can pull it out with near-perfect fidelity.
2. Scanned / image PDFs — there is no text layer. The bank printed the statement, scanned it, and shipped a flat image. You must run OCR to recover the characters.

At v1.0 our scanned-page path used PaddleOCR, an open-source recognition model. It worked, but it produced unstructured plain text with no sense of columns. A bank statement is a table: date, description, debit, credit, balance, running balance. PaddleOCR flattened those rows into a soup of tokens, and our normalization model then had to guess where each column began and ended. That guessing is where transactions got misread, dropped, or merged.

The v1.0 decision

We replaced PaddleOCR with Mistral's hosted OCR model, mistral-ocr-latest. Two reasons:

• Table awareness. Mistral OCR returns markdown plus an HTML table per page. For a statement, the table is the whole document — so preserving the column grid makes the next stage dramatically easier.
• Zero on-prem scaling. A hosted model means no GPU fleets, no model weights to maintain — just an API call.

The normalization stage at v1.0 was gpt-4o-mini. Here is the actual shaping code from converter.py:

MISTRAL_OCR_MODEL = "mistral-ocr-latest"

def run_ocr_on_scanned_pages(pdf_path: str, scanned_page_nums: list[int]) -> dict[int, str]:
    """Run Mistral OCR on scanned PDF pages and preserve table structure."""
    api_key = os.environ.get("MISTRAL_API_KEY")
    with Mistral(api_key=api_key) as client:
        response = client.ocr.process(
            model=MISTRAL_OCR_MODEL,
            document={"type": "document_url", "document_url": pdf_url},
        )
        # build page text from markdown + HTML tables, preserving the column grid

The whole pipeline at v1.0 was:

PDF upload → pdfplumber (native text) + Mistral OCR (scanned)
          → gpt-4o-mini normalization (date / debit / credit / balance)
          → bank detection (first page)
          → balance check (opening + credits - debits = closing)
          → light eval (row heuristic + format validation)

How we score against ground truth

We did not trust synthetic test PDFs. Instead we built an eval harness (evals/) that runs three real personal statements we own through the parser and scores every output field against a hand-written ground_truth.json pulled from the banks' own CSV/XLS exports:

Ground truth transaction formats are provided below for two representative rows (account numbers redacted):

{
  "bank": "canara",
  "account_type": "savings",
  "opening_balance": "1944.72",
  "closing_balance": "645.26",
  "transactions": [
    { "date": "2026-01-01", "description": "UPI/DR/600144887709/Indian Ra/SBIN/**iruts@sbi",
      "debit": "90.00", "credit": null, "balance": "1854.72" },
    { "date": "2024-04-02", "description": "REALME MOBILE TELECOMM GURGAON IN",
      "debit": "11758.00", "credit": null, "balance": null }
  ]
}

Each field — date, description, debit, credit, balance, reference, classification — is scored 0..1 and rolled into a mean transaction F1 and an overall score. A score of 1.0 means the parser's output is identical to the hand-checked truth.

v1.0 results on the three real statements

Transaction F1 by statement (v1.0, Mistral OCR):
Canara  ████████████████████████  92.8%
HSBC    ███████████████████████████████  97.6%
ICICI   ████████████████████      81.2%

What those numbers taught us

Three honest takeaways drove every later release:

1. Bank detection at 0%. At v1.0 we had not wired the LLM bank-detection step into the benchmark path — the model names in the pipeline and the model names scored diverged. Lesson: benchmark exactly what you ship.
2. The credit card was the hardest layout. ICICI's credit-card statement has no running-balance column, and its debit accuracy collapsed to 14.8%. When OCR flattens a dense card bill, columns bleed into each other.
3. Descriptions were reliably worse than balances. Human-readable merchant text is the last thing to converge, because the bank's own CSV and a clean parser describe the same merchant differently.

v1.0 was not good enough for production finance workflows — 90.5% mean F1 means roughly one in ten rows still needs a human eye. But it gave us a repeatable measurement loop, which turned out to be the most important artifact of the release.

Try it

Every release since has iterated on this same three-statement harness. You can see the current engine's numbers on our landing page, and the full model-by-model history in Bank Statement OCR Model History & Comparison. To run a statement through the current mistral-ocr-latest + deepseek-ai/DeepSeek-V4-Flash pipeline today, upload a PDF on the home page — no signup required.