Skip to content

[Exporters] Add FigureEncoder to handle detected pictures by the layout model - #2137

Draft
felixdittrich92 wants to merge 3 commits into
mindee:mainfrom
felixdittrich92:figure-addition
Draft

[Exporters] Add FigureEncoder to handle detected pictures by the layout model#2137
felixdittrich92 wants to merge 3 commits into
mindee:mainfrom
felixdittrich92:figure-addition

Conversation

@felixdittrich92

Copy link
Copy Markdown
Collaborator

This pull request introduces support for exporting detected figures (such as images or diagrams) in reading order when exporting documents to Markdown, AsciiDoc, and HTML formats. Figures detected by the layout model are now included in the reading order and can be materialized in various ways (inlined, referenced, or as placeholders) during export. The documentation and codebase have been updated to reflect and support this new functionality.

Exporters: Figure Handling and Reading Order

  • Figures detected by the layout model are now included in the reading order alongside blocks and tables, and are handled as special floating elements. This affects how content is linearized and exported, with figures optionally absorbing adjacent captions. ([[1]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL115-R118), [[2]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL125-R131), [[3]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL150-R156), [[4]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL165-R174), [[5]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL187-R199), [[6]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL282-R303), [[7]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR317-R329), [[8]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR339-R383), [[9]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR393-R444))

  • The FigureEncoder class and related functions are now imported and used to control how figures are materialized during export (as embedded images, referenced files, placeholders, or omitted). Exporters for Markdown, AsciiDoc, and HTML now support figures, with format-specific rendering and placeholders. ([[1]](https://github.com/mindee/doctr/pull/2137/files#diff-7d12e5d8d91d065449b99788d9539d946aaa2ebc4393ae9260e7f54b115ca59fR3), [[2]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR15-R19), [[3]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR533-R534), [[4]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR559-R566), [[5]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR581-R582), [[6]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR607-R615), [[7]](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dR639-R640))

Documentation Updates

  • The documentation has been expanded to explain how figures are detected, how they participate in reading order, and how their export can be configured via the images argument or a FigureEncoder. Detailed usage examples and explanations are provided. ([[1]](https://github.com/mindee/doctr/pull/2137/files#diff-4ec60e4ff05a071e3e4c9e7964f251f5d1c3ba7ce8a5e384343266a6babca7d4R180-R223), [[2]](https://github.com/mindee/doctr/pull/2137/files#diff-bee53c8935e4aa8f6ad459b019fedd88844d2a783bda7f7298f47d8022e6b50aR477-R491))

Internal Improvements

  • The reading order signature and memoization now account for layout regions (figures), ensuring cache invalidation when figures are added or changed. ([doctr/io/exporters.pyL95-R105](https://github.com/mindee/doctr/pull/2137/files#diff-4c8a0beab5e4c284ede1afc05149f48ef5ab259e3da92c3b528c14671908176dL95-R105))

These changes collectively make figure handling in document exports more robust, flexible, and user-configurable, improving the quality and fidelity of exported documents.

@felixdittrich92 felixdittrich92 added this to the 1.2.0 milestone Sep 1, 2026
@felixdittrich92 felixdittrich92 self-assigned this Sep 1, 2026
@felixdittrich92 felixdittrich92 added topic: documentation Improvements or additions to documentation type: enhancement Improvement module: io Related to doctr.io ext: docs Related to docs folder topic: layout detection Related to document layout detection labels Sep 1, 2026
@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.59615% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.19%. Comparing base (b50ddd1) to head (8b7b1a4).

Files with missing lines Patch % Lines
doctr/io/exporters.py 96.39% 4 Missing ⚠️
doctr/io/figures.py 98.95% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2137      +/-   ##
==========================================
- Coverage   97.24%   97.19%   -0.05%     
==========================================
  Files         169      170       +1     
  Lines       10040    10225     +185     
==========================================
+ Hits         9763     9938     +175     
- Misses        277      287      +10     
Flag Coverage Δ
unittests 97.19% <97.59%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext: docs Related to docs folder module: io Related to doctr.io topic: documentation Improvements or additions to documentation topic: layout detection Related to document layout detection type: enhancement Improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant