Custom OCR engineering
Custom OCR development for documents that off-the-shelf OCR gets wrong
Generic OCR reads characters. It does not read tables, reconcile columns, or tell you when it is guessing. I build production OCR pipelines that combine native text extraction, OCR with block-level confidence, vision-model escalation on low-confidence pages, and arithmetic validation — then prove the accuracy against hand-written ground truth instead of a vendor datasheet.
- Layout-aware table extraction, not just character recognition
- Per-page confidence scoring with automatic vision-model escalation
- Ground-truth benchmark harness so accuracy is measured, not claimed
- Runs on your cloud, your VPC, or fully on-premise
Why standard OCR fails on real documents
Most OCR products are benchmarked on clean, single-column, machine-printed pages. Real production documents are not that. They are faxed invoices, photographed receipts, multi-column bank statements, scanned shipping manifests with stamps across the table, and PDFs where the text layer disagrees with what is visually printed.
The failure mode is rarely a misread character. It is a structural failure: two columns merged into one, a debit silently recorded as a credit, a continuation row attached to the wrong transaction, a total that no longer matches its line items. Those errors pass every spell-check and land straight in your database.
The fix is not a better OCR vendor. It is a pipeline: route each page to the cheapest method that can read it, score how confident that read was, escalate only the pages that need a stronger model, and then validate the structured output against arithmetic the document itself must satisfy.
What I build
Every engagement is scoped to your document types, but these are the components that make up a production OCR system.
Native-text fast path
Digital PDFs never need OCR. A layout-preserving text extractor handles them in milliseconds at zero API cost, with column-header hints so a Debit/Credit/Balance grid survives extraction. Typically this alone routes 60–80% of a document corpus off the expensive path.
OCR with block-level confidence
Scanned and photographed pages go to an OCR model that returns positioned blocks and per-block confidence, not just a wall of text. Tables are preserved as structured HTML rather than flattened into prose, so column relationships survive.
Vision-model escalation
Pages whose confidence falls below a threshold are re-verified by a vision-language model against the rendered page image. Escalation is capped per document and fails safe, so a slow or unavailable model degrades quality gracefully instead of breaking the job.
Structured normalization
An LLM step converts raw text into a strict JSON schema — typed dates, signed amounts, references, per-field page attribution — using constrained JSON output and tolerant parsing so a single malformed response does not fail the document.
Domain validation rules
The step almost everyone skips. Running-balance reconciliation, line-items-sum-to-total checks, date monotonicity, sign inference from balance deltas, duplicate detection. These catch errors no confidence score will.
Evaluation harness
A repeatable benchmark that scores every field against hand-written ground truth and reports precision, recall, F1, latency, and cost per page. Without this you cannot tell an improvement from a regression.
How an engagement runs
Fixed scope, fixed milestones, and a measurable accuracy target agreed before any pipeline work starts.
1. Document audit (week 1)
You send a representative sample — ideally 20–50 documents covering your worst cases, not your best. I categorise them by layout family, measure how many are digital versus scanned, and identify the specific structures that will break a naive extractor.
2. Ground truth and target accuracy (week 1–2)
We hand-write ground truth for a held-out subset and agree what 'good enough' means in numbers: field-level F1, tolerance on amounts, acceptable manual-review rate. This becomes the acceptance criterion for the whole project.
3. Baseline and gap analysis (week 2)
I run your documents through a baseline pipeline and score it. You get a written report of exactly where accuracy is lost, per document family, with the cost and latency profile of each stage. Some clients stop here — the report alone is often enough to redirect an in-house effort.
4. Pipeline build (weeks 3–8)
Iterative build against the benchmark. Every change is scored before and after, so progress is visible rather than asserted. You get working code in your repository from the first week, not a demo at the end.
5. Hardening and handover
Retries, timeouts, concurrency limits, cost caps, structured logging, failure-mode documentation, and a runbook. Plus the benchmark harness itself, so your team can keep scoring changes after I am gone.
Document types this pipeline suits
- Bank statements — retail, business, and credit card, across multiple countries and layout conventions (the type I have taken furthest, in production)
- Invoices and purchase orders with variable line-item tables
- Receipts, including phone-camera captures with skew, glare, and creasing
- Financial reports and statements of account with nested subtotals
- Identity and KYC documents with mixed print and handwriting
- Logistics paperwork — manifests, bills of lading, delivery notes
- Forms with checkbox, signature, and free-text regions
Accuracy on my own production benchmark
Parse My Statement is the reference implementation of this pipeline. These scores come from the same ground-truth harness I build for clients, run against real personal statements (Canara savings, HSBC savings, ICICI credit card) scored field by field against the banks' own CSV exports.
| Engine version | Transaction F1 | Date accuracy | Notes |
|---|---|---|---|
| v2.1 · Aug 2026 | 0.93 | 0.99 | Vision escalation + table-header hints; 100% F1 on two of three statements |
| v2.0 · Jul 2026 | 0.48 | 0.66 | Blocks + confidence, before normalization hardening — a measured regression |
| v1.1 · Jun 2026 | 0.90 | 1.00 | OCR-only path |
| v1.0 · Mar 2026 | 0.86 | 0.99 | First production release |
Engagement options
Indicative ranges. Final scope and price depend on document variety, volume, and how much of the infrastructure already exists.
Extraction audit
from $1,400
1–2 weeks
A written accuracy and feasibility report on your document corpus, with a baseline benchmark you keep.
- Document corpus categorisation
- Ground-truth set for a held-out sample
- Scored baseline with per-family failure analysis
- Cost and latency model per page
- Build-versus-buy recommendation
Pipeline build
from $3,500
4–10 weeks
A production extraction pipeline for one or more document families, delivered into your repository.
- Everything in the audit
- Full pipeline: routing, OCR, escalation, normalization, validation
- Evaluation harness and CI accuracy gate
- Deployment to your cloud, VPC, or on-premise hardware
- Runbook, failure-mode docs, and team handover
Retained engineering
from $2,200 / month
Rolling, 3-month minimum
Ongoing accuracy work as your document mix changes and new sources come online.
- New document families added on a rolling basis
- Regression monitoring against the benchmark
- Model and cost optimisation as providers change
- Priority response on extraction incidents
Frequently asked questions
Can you improve an OCR pipeline we already have?
Yes, and it is often the faster path. The first step is the same either way: build a ground-truth benchmark so we can see where accuracy is actually being lost. In many cases the existing pipeline is sound and the problem is in normalization or missing validation rules, which is a much smaller fix than a rebuild.
Which OCR and vision models do you use?
It depends on the documents and your constraints. Parse My Statement runs Mistral OCR for scanned pages and Qwen3-VL for vision re-verification, with DeepSeek for normalization. For clients who cannot send data to third-party APIs I build the same architecture on self-hosted models — see the self-hosted vision-language model service.
How accurate can you actually get?
That depends entirely on the documents, and any consultant who quotes a number before seeing yours is guessing. What I commit to is a measured baseline in the first two weeks and a target agreed in writing against that baseline. Clean digital documents routinely reach near-perfect field accuracy; degraded phone photos of handwritten forms will not, and the honest answer is to design a human-review queue for them.
Do you hand over the source code?
Yes. Work is delivered into your repository under your ownership as it is written, not at the end. The evaluation harness comes with it so your team can keep measuring after handover.
Can this run without sending data to any external API?
Yes. The same architecture runs on self-hosted OCR and vision models inside your own network or an air-gapped environment. That changes the hardware requirements and usually the accuracy ceiling, and both are quantified during the audit.
Send me your hardest documents
The fastest way to start is a sample of the documents that currently break. I will tell you what is realistically achievable and what it would take — before any engagement is signed.
Related
Document parsing & extraction
Document parsing and data extraction, built as production infrastructure
Turn unstructured PDFs, scans, and emails into validated structured data. Custom document parsing pipelines with schema design, confidence scoring, human review queues, and measurable accuracy.
Self-hosted VLM deployment
Deploy vision-language models on your own hardware
Set up open-weight vision-language and OCR models on your own GPUs or private cloud — Qwen3-VL, InternVL, and similar. Sizing, quantization, vLLM serving, throughput tuning, and accuracy validation.
On-premise & private cloud
On-premise and private-cloud document AI deployment
Deploy the full document extraction stack inside your own infrastructure — VPC, private cloud, or air-gapped — with no document leaving your network. Sizing, deployment, and compliance documentation.
IP & source licensing
License the extraction engine, the source, or the whole product
Source-code licensing, perpetual IP licensing, and acquisition options for a production bank statement extraction engine with a published ground-truth accuracy benchmark.
This page is also available as markdown for AI agents: /services/custom-ocr-development.md · index at /llms.txt. Canonical URL: https://parsemystatement.com/services/custom-ocr-development.