api:
url: http://0.0.0.0:8000
env:
metadata:
instrument: i06
sources:
- kind: deviceManager
module: dodal.beamlines.i06_1
mock: True
- kind: planFunctions
module: dodal.plans
- kind: planFunctions
module: dodal.plan_stubs.wrapped
stomp:
enabled: true
url: tcp://localhost:61613/
numtracker:
url: http://localhost:8406/graphql
And then running this with blueapi python client, when trying to run a plan such as moving a device it runs okay. However, when trying to run a scan, an error occurs that is not obvious what is causing it.
2026-08-04 15:43:43,360 INFO blueapi.worker.task_worker Submitting: task_id='ef0c21cc-bdb1-49e4-8851-caad0170ffeb' task=Task(name='step_grid_scan', params={'detectors': [], 'params': [['scmc.cart.x', [0, 1, 0.1]]]}, metadata={'user': 'Unknown', 'instrument_session': 'cm44150-1'}) request_id=None is_complete=False is_pending=True errors=[] outcome=None
2026-08-04 15:43:43,360 INFO blueapi.worker.task_worker Got new task: task_id='ef0c21cc-bdb1-49e4-8851-caad0170ffeb' task=Task(name='step_grid_scan', params={'detectors': [], 'params': [['scmc.cart.x', [0, 1, 0.1]]]}, metadata={'user': 'Unknown', 'instrument_session': 'cm44150-1'}) request_id=None is_complete=False is_pending=True errors=[] outcome=None
2026-08-04 15:43:43,360 INFO blueapi.worker.task Asked to run plan step_grid_scan with {'detectors': [], 'params': [['scmc.cart.x', [0, 1, 0.1]]]} and metadata {'user': 'Unknown', 'instrument_session': 'cm44150-1'} for all runs
2026-08-04 15:43:43,360 INFO bluesky Executing plan <generator object step_grid_scan at 0x7fb259eecf20>
2026-08-04 15:43:43,360 INFO bluesky.RE.state Change state on <bluesky.run_engine.RunEngine object at 0x7fb29bd2fc90> from 'idle' -> 'running'
2026-08-04 15:43:43,362 INFO blueapi.service.main 127.0.0.1:35674 PUT /worker/task 200
2026-08-04 15:43:43,368 ERROR bluesky Run aborted
Traceback (most recent call last):
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
raise exc from None
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
raise exc
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request
stream = await self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_async/connection.py", line 124, in _connect
stream = await self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp
return await self._backend.connect_tcp(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
with map_exceptions(exc_map):
File "/usr/lib64/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: All connection attempts failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/run_engine.py", line 1616, in _run
msg = self._plan_stack[-1].send(resp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/dodal/plans/wrapped.py", line 509, in step_grid_scan
yield from bp.list_grid_scan(
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/plans.py", line 356, in list_grid_scan
return (yield from scan_nd(detectors, full_cycler, per_step=per_step, md=_md))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/plans.py", line 1168, in scan_nd
return (yield from inner_scan_nd())
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/utils/__init__.py", line 1351, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/preprocessors.py", line 1013, in stage_wrapper
return (yield from finalize_wrapper(inner(), unstage_devices()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/preprocessors.py", line 548, in finalize_wrapper
ret = yield from plan
^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/preprocessors.py", line 1011, in inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/utils/__init__.py", line 1351, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/preprocessors.py", line 362, in run_wrapper
rs_uid = yield from open_run(md)
^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/utils/__init__.py", line 2047, in __iter__
return (yield from self._iter)
^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/plan_stubs.py", line 1366, in open_run
return (yield Msg("open_run", **(md or {})))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/run_engine.py", line 1683, in _run
new_response = await coro(msg)
^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/run_engine.py", line 1855, in _open_run
self.md["scan_id"] = await maybe_await(self.scan_id_source(self.md))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/bluesky/utils/__init__.py", line 2029, in maybe_await
return await ret
^^^^^^^^^
File "/scratch/bluesky_development/blueapi/src/blueapi/core/context.py", line 159, in _update_scan_num
scan = await numtracker.create_scan(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/src/blueapi/utils/numtracker.py", line 98, in create_scan
response = await client.post(
^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1859, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1540, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
File "/usr/lib64/python3.11/contextlib.py", line 158, in __exit__
self.gen.throw(typ, value, traceback)
File "/scratch/bluesky_development/blueapi/.venv/lib64/python3.11/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed
When trying to run blueapi locally using the following configuration
uv run blueapi -c /path/to/blueapi_source.yaml serve
And then running this with blueapi python client, when trying to run a plan such as moving a device it runs okay. However, when trying to run a scan, an error occurs that is not obvious what is causing it.
Client side
Server side
@tpoliaw managed to help me fix it by commenting out numtracker configuration and then the scan would work again, but this traceback is not obvious at all what is causing the issue. Is it possible we could make the traceback user friendly mentioning num tracker?