fix(execution): fail an attachment job when the disk does, instead of escaping - #259
Merged
Merged
Conversation
… escaping
A full disk while staging an attachment left the job non-terminal for good and
answered the route with HTTP 500.
`_publish_bytes` caught only `ExecutionRepositoryError`:
except ExecutionRepositoryError:
raise ArtifactBoundaryError("artifact_publish_failed") from None
But `publish_artifact` re-raises: its handler removes the partial files and
then `raise`s the original, so a full disk, a permission error or an antivirus
lock arrives unwrapped as `OSError`. That escaped the boundary, so
`AttachmentStagingService`'s `except ArtifactBoundaryError` did not match,
`_fail` never ran, and the job it had already created stayed at `queued`
forever while the exception surfaced as an unhandled 500.
Reproduced by making `publish_artifact` raise `OSError(28)`:
before: OSError escaped the boundary job status left as: queued
after: AttachmentStagingError raised job status left as: failed
`publish_outputs`, a few lines down in the same class, already catches
`ExecutionRepositoryError` and `OSError` and rolls back for both -- so this
was an inconsistency between two sibling paths rather than a deliberate
distinction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A full disk while staging a chat attachment leaves the job non-terminal for the rest of the installation's life and answers the route with an unhandled HTTP 500 instead of a stable error code.
Reproduced by making
publish_artifactraiseOSError(28, "No space left on device"):OSErrorescaped the boundaryAttachmentStagingErrorqueuedfailedRoot cause
_publish_bytescaught only one of the two exception typespublish_artifactcan produce:publish_artifactre-raises. Its own handler cleans up the partial files and then propagates the original:So a full disk, a permission error, or an antivirus lock on
temporary.open("xb")/os.replacearrives unwrapped asOSError— and escapes the boundary entirely.The caller is written to rely on that boundary:
create_jobhas already committed a row by this point, so when_failis skipped the job simply staysqueued. Nothing later reaps it — the retention pass only removes terminal jobs.Not a deliberate distinction
publish_outputs, in the same class a few lines down, already treats both identically — catchingExecutionRepositoryErrorandOSError, each with its own rollback:So the author already knew
publish_artifactcan raiseOSError._publish_byteswas simply out of step with its sibling.The fix
No rollback is needed here that is not already done:
publish_artifactremoves both the temporary and target files on its way out, and_publish_byteshas created nothing else.Verification
The new test injects
OSError(28)and asserts both halves of the contract — that a typed error reaches the caller, and that the job row is left terminal:The job-status assertion is the one that matters: an exception-type-only test would pass on a fix that translated the error but still stranded the job.
Against the unfixed code it fails.
python -m pytest -qpython -m mypypython -m ruff check backend tests tools main.py app_factory.pypython tools/artifact_boundary_review.py --json --strictRun on Python 3.14, one of the versions in the compatibility matrix.
Compatibility and rollback
Widens one
exceptclause. Every success path and every already-handled failure behaves exactly as before; the only change is for an exception that previously escaped unhandled. No API contract, stored data, or migration. Reverting the commit restores the previous behaviour exactly.Limits
Jobs already stranded by this before the fix are not retroactively failed — they are non-terminal rows that the retention pass will not collect. Sweeping them needs a migration that can distinguish a genuinely-stuck job from a running one, which is a separate change.
The error code is the existing generic
artifact_publish_failed. A disk-full condition is arguably worth its own code so the UI can say "not enough disk space", but that is a contract addition rather than a correctness fix.🤖 Generated with Claude Code