Wait before rebuilding a computer that just lost a race - #78
Open
rhlsthrm wants to merge 1 commit into
Open
Conversation
Creating a computer and starting it are two calls to Docker. A name the daemon has not published yet answers the second with a 404, which the supervisor already reads correctly as a lost race and answers by rebuilding. It went straight back round, though, and the retry is the whole budget: it arrived a millisecond later, saw the same unpublished name, and spent the second attempt on the state that failed the first. The request then failed as Docker being unreachable, so the first browser action a Bot was ever asked for was refused with "no such container", and the next message seconds later worked. One poll interval first, which is what waitUntilAnswering already uses for the same question. The two sleeps share a helper.
rhlsthrm
requested review from
MikeRyanDev,
davidmckayv,
guidovizoso and
tylerslaton
as code owners
August 21, 2026 08:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Creating a computer and starting it are two calls to Docker. A container name the daemon has not published yet answers the second with a 404, and
ensurealready reads that correctly as a lost race whose answer is to rebuild.It retried immediately, though, and with
ATTEMPTS = 2the retry is the entire budget: it arrived within a millisecond,inspectOwnedstill saw nothing, the create came back 409 because the name was in fact taken, andstartreturned 404 again with no attempt left. The whole request then failed asDockerUnavailableError, which is surfaced to the person as the computer not being startable and recorded ascomputer.action_failed.The symptom is that the first browser action a Bot is ever asked for fails, and the same request seconds later works, so it reads as a flaky Bot rather than as a race. On this machine (OrbStack) it hit two of three Bots on their first ever action.
One poll interval before rebuilding fixes it — 250 ms, which is what
waitUntilAnsweringalready uses for the same question about the same container. Both sleeps now share apausehelper.Where it runs
ensurecall.ensurecalls are still resolved by Docker refusing a duplicate name with 409, which this path already treats as success.Boundary and audit
computer.action_failedwith the Docker message, which is how it was diagnosed.Changelog
Unreleased→Fixed.Proof
Before, on a first-ever action for a Bot, from
audit_events:Then the same request a minute later succeeded, with the container present. The same pair appeared for
risk-analyston its first action.After, on this commit: rebuilt the supervisor, deleted
openbot-computer-risk-analystoutright, and asked that Bot to open a page. The container was created and started on the first attempt, the page rendered in the live screen, andaudit_eventshascomputer.action_allowedwith no failure row beside it. A second Bot (knowledge) also provisioned first time.