fix: convert images off the event loop in LLMDocumentContentExtractor.run_async - #12359
Conversation
….run_async `run_async` calls `DocumentToImageContent.run` directly. That component reads each file from disk, renders the requested page of every PDF and base64-encodes the result, and it implements no `run_async` - so the whole batch is converted on the event loop before the first LLM call is even scheduled. Everything else sharing that loop waits. Route it through `_execute_component_async`, which this module already imports and uses for the chat generator: it prefers `run_async` when a component has one and otherwise runs `run` in a thread. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Someone is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @pcbeingused333, thanks for your interest in contributing to Haystack! 🙏 This is an automated message to help us keep the review queue healthy. |
|
Hi @pcbeingused333, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the |
|
recheck |
1 similar comment
|
recheck |
|
Thanks for signing the CLA, @pcbeingused333! 🎉 This PR is now ready for review again and the reviewer has been re-assigned. |
Related Issues
Proposed Changes:
LLMDocumentContentExtractor.run_asyncconverts the documents to images with a direct call:DocumentToImageContentreads every file from disk, renders the requested page of each PDF and base64-encodes the result, and it implements norun_async. So the whole batch is converted on the event loop before the first LLM call is even scheduled, and anything else sharing that loop waits for it.The module already imports
_execute_component_asyncand uses it for the chat generator, which is exactly the helper for this: it prefersrun_asyncwhen a component has one and otherwise runsrunin a thread. This change routes the conversion through it. IfDocumentToImageContentgrows arun_asynclater, this call site picks it up for free.runis untouched.How did you test it?
test_run_async_converts_images_off_the_event_loop: patchesDocumentToImageContent.runto record the thread it executes on and asserts it is not the thread running the event loop. It fails onmain, where the conversion runs on the main thread.hatch run test:unit— 6116 passed, 10 skipped.hatch run test:typesandhatch run fmtclean.Notes for the reviewer
I looked for the same pattern across the components with a
run_asyncand this was the one remaining call that does blocking I/O; the other directruncalls in async methods are prompt builders, which are pure CPU and cheap.I used an AI assistant while writing this change. I have reviewed it, reproduced the behaviour, and run the tests.
Checklist