Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions docs/source/modules/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,47 @@ page, so exporting one page to several formats orders it only once.
.. autofunction:: doctr.io.exporters.predictions_in_reading_order

.. autofunction:: doctr.io.exporters.to_json_safe

Figures
-------

When the predictor runs with ``detect_layout=True``, the figures found by the layout model take part in the
reading order and are materialized by the Markdown, AsciiDoc and HTML exports. How they are materialized is
controlled by the ``images`` argument, which accepts either an image mode or a configured
:class:`FigureEncoder`:

* ``'placeholder'`` (the default): a comment marks where a figure was detected, without touching the pixels
* ``'none'``: the figures are dropped entirely
* ``'embedded'``: each figure is cropped out of the page and inlined as a base64 data URI
* ``'referenced'``: each crop is written next to the export and referenced by a relative path

.. code:: python

from doctr.io import DocumentFile, FigureEncoder
from doctr.models import ocr_predictor

predictor = ocr_predictor(pretrained=True, detect_layout=True)
doc = predictor(DocumentFile.from_pdf("report.pdf"))

# A self-contained Markdown file
markdown = doc.export_as_markdown(images="embedded")
# ... or one that points at the crops on disk
markdown = doc.export_as_markdown(images=FigureEncoder("referenced", image_dir="assets", path_prefix="assets/"))

A caption detected next to a figure becomes its alternative text (and its ``<figcaption>`` in HTML) as soon as
the export carries the pixels. Plain text and the hOCR export never inline an image: hOCR positions each figure
as an ``ocr_photo`` area instead. Pages restored from a JSON export carry no pixels, so their figures fall back
to a placeholder.

.. autoclass:: FigureEncoder
:members: resolve, source, enabled, materializes, materializes_on

.. autofunction:: crop_layout_region

.. autofunction:: encode_crop

.. autofunction:: picture_regions

.. autofunction:: is_picture_region

.. autofunction:: is_picture_label
15 changes: 15 additions & 0 deletions docs/source/using_doctr/using_models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,21 @@ In addition to running the :py:meth:`layout_predictor <doctr.models.layout.layou

The same ``detect_layout`` / ``layout_arch`` arguments are available for the :py:meth:`kie_predictor <doctr.models.kie_predictor>`.

The figures found by the layout model also take part in the reading order, and the Markdown / AsciiDoc / HTML exports can materialize them, either inlined as base64 data URIs or written next to the export (see :ref:`Figures` for the details):

.. code:: python3

# A self-contained Markdown file, figures included
markdown = result.export_as_markdown(images="embedded")

# ... or one referencing the crops written to an `assets` directory
from doctr.io import FigureEncoder

encoder = FigureEncoder("referenced", image_dir="assets", path_prefix="assets/")
markdown = result.export_as_markdown(images=encoder)

By default (``images="placeholder"``) a comment marks the position of every detected figure, and ``images="none"`` drops them entirely.


Running the predictors on GPU
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
1 change: 1 addition & 0 deletions doctr/io/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from .elements import *
from .exporters import *
from .figures import *

Check warning on line 3 in doctr/io/__init__.py

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

doctr/io/__init__.py#L3

'.figures.*' imported but unused (F401)
from .html import *
from .image import *
from .pdf import *
Expand Down
Loading
Loading