High-precision conversion of document page images to structured Markdown

中文文档

---

📖 Overview

ABot-OCR is an end-to-end vision-language model that transcribes a document page image directly into clean Markdown in a single forward pass—no brittle modular orchestration. It recognizes and preserves text, mathematical formulas (LaTeX), tables (HTML), and related layout elements.

📰 News

[2026-5-29] 🥳🥳 ABot-OCR's technical report and weights have been released. 🎉🎉

Table of Contents

• 🏆 Benchmark
• 📦 Model Zoo
• 🚀 Quick Start
• 📄 Citation
• 🙏 Acknowledgments

---

🏆 Benchmark

The figure below shows overall evaluation results on OmniDocBench v1.5. See the technical report for full benchmark setup and comparisons (including v1.6).

---

📦 Model Zoo

Model weights are large and are not bundled in this repository. Download them from Hugging Face and place them locally:

Model	Platform	Link
ABot-OCR	🤗 Hugging Face	acvlab/ABot-OCR

repo/
└── abot-ocr/          # Extract / place downloaded weights here
    ├── config.json
    ├── model.safetensors
    └── ...

---

🚀 Quick Start

Inference script: abot-ocr-infer.py

Requirements

We recommend Python 3.11+ and the following dependencies:

pip install vllm==0.18.0 torch==2.10.0

Note: Inference loads the model with vLLM and requires sufficient GPU memory (~4GB weights; actual usage depends on batch_size and image resolution).

Run

1. Download model weights to ./abot-ocr/ (see ModelZoo)
2. Prepare images to recognize (a single file or a directory)
3. Run:

python abot-ocr-infer.py

By default, images are read from images/ and Markdown results are written to ./abot-ocr-infer-output/.

Configuration

Model path — loaded from the repo directory by default; to change:

MODEL_PATH = str(Path(__file__).resolve().parent / "abot-ocr")

Input / output — edit the __main__ block at the bottom of abot-ocr-infer.py:

run_infer(
    input_path="images",                  # Single image or directory (nested subdirs supported)
    llm=llm,
    processor=processor,
    sampling_params=sampling_params,
    batch_size=8,                         # Images per batch; 0 = infer all at once
    output_dir="./abot-ocr-infer-output"  # Omit to write .md next to each image
)

Behavior	Description
Default output	One .md file per image (same basename)
output_dir set	Write to the given directory, preserving relative subpaths
Resume	Images that already have a matching .md are skipped
Failures	Unreadable images are logged to failed_images.log

---

📄 Citation

If you find ABot-OCR useful, please cite our technical report:

@article{jiang2026abotocr,
  title   = {ABot-OCR Technical Report},
  author  = {Jiang, Kaitao and Gong, Ruiyan and Cheng, Xiaolong and Niu, Kangning and Li, Tianlun and Xu, Mu},
  journal = {arXiv preprint arXiv:2605.27978},
  year    = {2026},
  url     = {https://arxiv.org/abs/2605.27978},
}

---

🙏 Acknowledgments

Our work is inspired by many excellent open-source projects. We sincerely thank the developers of Qwen-VL, PaddleOCR-VL, MinerU, and the broader OCR community.