Running a graph from code that already has an event loop still fails on 2.1.6 with the error reported in #122, #179, #186, #694, #708 and #771:
RuntimeError: asyncio.run() cannot be called from a running event loop
Minimal repro, no LLM or network needed, the error fires before any fetch:
import asyncio
from scrapegraphai.docloaders.chromium import ChromiumLoader
async def main():
loader = ChromiumLoader(["https://example.com"], headless=True)
return loader.load()
asyncio.run(main())
The same call works at the top level of a plain script. The crash comes from ChromiumLoader.lazy_load, which calls asyncio.run() per URL (scrapegraphai/docloaders/chromium.py line 463 on current main). FetchNode uses loader.load(), so any graph that fetches a URL with the default loader hits this in notebooks, FastAPI handlers, and any other code running under a loop.
Two nodes try to handle the running-loop case but the guard is inverted. graph_iterator_node.py lines 74-75 and generate_answer_from_image_node.py lines 121-123 call eventloop.run_until_complete() exactly when eventloop.is_running() is true, the one situation where that call always raises. Those branches can only ever have worked with nest_asyncio applied. And nest_asyncio is a patch on the loop rather than a fix in the library: in #179 a user who applied it just hit a different error.
The repo already has the right idea in AbstractGraph.run_safe_async, which offloads the blocking call to an executor. Applying the same idea at the four call sites that use asyncio.run or run_until_complete (chromium.py, browser_base.py and the two nodes above) fixes these environments without changing anything for plain scripts. Tested on 2.1.6 (current main), Python 3.12, Windows. I have the fix ready and will open a PR in a few minutes.
Running a graph from code that already has an event loop still fails on 2.1.6 with the error reported in #122, #179, #186, #694, #708 and #771:
Minimal repro, no LLM or network needed, the error fires before any fetch:
The same call works at the top level of a plain script. The crash comes from
ChromiumLoader.lazy_load, which callsasyncio.run()per URL (scrapegraphai/docloaders/chromium.pyline 463 on current main). FetchNode usesloader.load(), so any graph that fetches a URL with the default loader hits this in notebooks, FastAPI handlers, and any other code running under a loop.Two nodes try to handle the running-loop case but the guard is inverted.
graph_iterator_node.pylines 74-75 andgenerate_answer_from_image_node.pylines 121-123 calleventloop.run_until_complete()exactly wheneventloop.is_running()is true, the one situation where that call always raises. Those branches can only ever have worked with nest_asyncio applied. And nest_asyncio is a patch on the loop rather than a fix in the library: in #179 a user who applied it just hit a different error.The repo already has the right idea in
AbstractGraph.run_safe_async, which offloads the blocking call to an executor. Applying the same idea at the four call sites that useasyncio.runorrun_until_complete(chromium.py, browser_base.py and the two nodes above) fixes these environments without changing anything for plain scripts. Tested on 2.1.6 (current main), Python 3.12, Windows. I have the fix ready and will open a PR in a few minutes.