Finetuning olmOCR to be a faithful OCR-Engine

Source: HuggingFace Date: 2025-04-22 URL: https://huggingface.co/blog/tngtech/finetuning-olmocr-to-be-a-faithful-ocr-engine

Summary

Model release and case study: TNG Technology Consulting fine-tunes olmOCR-7B to capture headers and footers the original model intentionally omits. Training: Qwen2.5-VL-72B-Instruct generates 8,000 complete-OCR training documents, 6 hours on 8xH100, 2.5 epochs. Released as tngtech/olmOCR-7B-faithful. No quantitative metrics (no CER/WER) — qualitative examples show improved invoice parsing with headers/footers, simple tables, and multi-column layouts. Based on Allen Institute’s open-source training pipeline.

Implications

Open-weights ecosystem health. Using a 72B VLM to generate training data for a 7B model fine-tune — “teacher distillation via synthetic OCR annotation” — is a practical pattern for filling capability gaps in specialized models without expensive human annotation. The olmOCR-7B-faithful release makes this pattern reproducible and gives the community a production-ready invoice/document OCR model.

Transformers library trajectory. The Allen Institute’s olmOCR open-source training pipeline being reused by TNG for a domain-specific fine-tune demonstrates that the open-weights model ecosystem is building cumulative infrastructure. Each training pipeline open-source release becomes an accelerant for subsequent specialized fine-tunes — the key bottleneck shifts from “how do we train this?” to “what data do we need?”