Python PDF Chapter Detection From OCR — Folio (Open Source)

Marker turns a PDF into Markdown/JSON, and on a clean digital PDF that's the whole job — there's a text layer, the structure is mostly intact, done. On a scanned book it isn't, because scanning throws away exactly the metadata chapter detection depends on: there's no PDF outline to read chapters from, running headers and footers repeat on every page and pollute the body text, OCR mangles the contents page itself (which is the one page you most need to read correctly), and the printed page numbers don't line up with PDF page indices — and the offset between them isn't constant, it drifts across the front matter.

Folio is the layer that sits on top of Marker's output and reconstructs a scanned book's structure anyway. Pure Python, standard library plus pypdfium2, Apache-2.0. It's not affiliated with Datalab or the Marker project — it reads Marker's JSON, it contains none of Marker's code or model weights.

Install

pip install -e .

The import name is folio; the distribution name on PyPI is folio-book, because the bare name folio is held by an unrelated, abandoned 2013 package.

Usage

from folio import convert
        result = convert(marker_json, embedded_page_texts=embedded)
        result["chapters"]                      # reconstructed chapters
        result["metadata"]["chapterDetection"]   # which tier produced them
        result["metadata"]["totalChapters"]

Each chapter comes back with title, startPage, wordCount, plainText and separated footnotes.

How it decides — a cascade, not a model

Detection runs as a tiered cascade, strongest signal first, degrading to a safe fallback instead of guessing:

Around that cascade: clean_toc reconstructs the printed-page → PDF-page offset (and that drift across front matter mentioned above); header_footer strips running headers using position and repetition, not a model, so it works on any book; chapter_titles recovers real titles from opener pages and rejects OCR garbage rather than surfacing it; and two-up scans — one landscape image holding two actual book pages — get split into portrait pages before any of the above runs, so reading order comes out right instead of interleaved.

What I actually verified

I ran the repo's own test suite before writing this:

python -m pytest tests -q
        85 passed in 0.08s

85 tests, all green. More interesting than the count is what the project argues you should actually gate on: making one book work without breaking the last one. corpus_check.py turns that from a feeling into a number — it scores every fixture book against its expected outcome and enforces a safety property beyond the happy path: a book may legitimately degrade to full-text, but the harness fails it if it ships a mega-chapter spanning 40%+ of the book, a large uncovered leading gap, or mostly-generic titles. Degrading honestly is allowed; degrading and pretending it didn't is not.

Where it's actually used

Folio was extracted from the PDF pipeline of Velo, the RSVP and audiobook-style reading app, where this exact chapter-reconstruction logic runs in production over user-uploaded scanned books — the corpus this repo's regression harness protects is drawn from the same kind of real, messy scans Velo's users actually import.

Known limitation

The regression corpus itself is gitignored — corpus/cache/ holds OCR of real, copyrighted books, so you can't clone the repo and immediately run the full fixture set against those books; corpus/fixtures.json (the expectations) and the harness ship, but populating the cache with your own books is a manual step (see corpus/README.md). And by design, a sufficiently hard scan is allowed to fall all the way back to full-text — one undivided chapter with no fabricated table of contents — rather than guess. That's the documented trade-off, not a bug.

Apache-2.0 licensed. Full detection logic and the corpus harness are on GitHub.

← All guides