cuda.core: define the error handling policy and report failures that cannot be raised

cuda_core/AGENTS.md

@@ -101,6 +101,62 @@ and agents should flag violations.
   (kernel arguments, memcpy/memset operands, `dst_owner`/`src_owner`, and
   host-callback closures) inherit this contract.
 
+## Failure handling
+
+The user-facing contract lives in `docs/source/error_handling.rst`; the rules
+below are for contributors. Reviewers and agents should flag violations.
+
+- **Raise by default**: any failure on a path where an exception can propagate
+  raises. Driver statuses go through `HANDLE_RETURN` (Cython) or are returned as
+  `CUresult` from the C++ handle layer and then `HANDLE_RETURN`ed; never
+  replace a `CUresult` with a generic `RuntimeError`, and drain
+  `get_last_error()` immediately after a handle constructor returns empty so a
+  stale status cannot be misattributed later.
+- **Guarantees**: a call that creates a resource must create nothing when it
+  raises (undo the creation if a later step fails). Every call except
+  `Device.set_current` must leave the calling thread's current context as it
+  found it. Do not hand-roll `cuCtxPush/Pop/SetCurrent` sequences in Cython; use
+  the handle layer's scoped-context helpers (`invoke_in_context`,
+  `invoke_in_context_or_undo`, `cleanup_in_context`, `context_get_device`,
+  `graph_node_set_params`) so the failure handling exists in one place.
+- **Publish before you raise**: when a driver mutation has succeeded and a later
+  step can still fail, commit whatever keeps that mutation memory-safe (for
+  example the graph attachment that retains a node's new owners) before raising
+  the later error. Rolling back the retention of a live mutation creates a
+  dangling reference. When ownership cannot be established, retain the
+  resources anyway (leak) rather than release them; a leak is always preferred
+  to a use-after-free.
+- **Non-propagating paths never raise and never discard a status**: shared_ptr
+  deleters, `__dealloc__`, CUDA callbacks and cleanup after a failure report
+  through one channel, `report_cuda_error()` / `report_message()` in C++ (the
+  `pw_*` wrappers) or `warnings.warn(..., CUDAWarning)` in Cython and Python,
+  which emits `cuda.core.CUDAWarning`. No `print(file=sys.stderr)` and no
+  `fprintf` outside that helper. `CUDA_ERROR_DEINITIALIZED` is filtered by the
+  helper because it means the driver is shutting down.
+- **Rollback failure**: the original exception propagates; the failed rollback
+  is reported out of band (or chained with `raise ... from` when a second
+  exception must be raised). Bare `except:` is acceptable only for
+  rollback-then-`raise` blocks.
+- **Finalization**: once `py_is_finalizing()` is true, do no Python work from
+  destructors or callbacks and accept the leak (see
+  `_cpp/resource_handles.hpp` and `_cpp/GRAPH_ATTACHMENTS.md`).
+- **Aborting**: `std::abort` (or any process termination) is reserved for an
+  internal invariant violation where continuing could corrupt memory or produce
+  silently wrong results *and* no leak-based fallback exists. A failed CUDA
+  call, including a failed context restoration, never qualifies: raise or
+  report instead. There is currently no such path; if one is ever needed it
+  must go through a single helper that writes a diagnostic (call, CUDA error,
+  invariant, "please report") to stderr before aborting, must never trigger
+  during interpreter finalization or for driver-shutdown errors, and must be
+  called out in the docs and release notes. An *implicit* abort (an exception
+  escaping a `noexcept` function or a deleter, including `std::bad_alloc` from
+  an allocation inside `noexcept` code) is a bug (#1489, #2417), not a policy
+  choice: `noexcept` helpers must not allocate, or must catch what they call.
+- **Testing**: inject restoration failures with
+  `cuda.core._resource_handles._set_context_restore_fault_for_testing`; assert
+  reports with `pytest.warns(CUDAWarning)` or `warnings.catch_warnings`, never
+  by matching stderr text.
+
 ## API design guidelines
 
 These are some API design guidelines we try to follow when adding new APIs to

cuda_core/cuda/core/__init__.py

@@ -102,8 +102,10 @@ class _PatchedProperty(metaclass=_PatchedPropMeta):
 from cuda.core._stream import __all__ as _stream_all
 from cuda.core._tensor_map import *
 from cuda.core._tensor_map import __all__ as _tensor_map_all
+from cuda.core._utils.cuda_utils import CUDAWarning
 
 __all__ = [
+    "CUDAWarning",
     *_context_all,
     *_device_all,
     *_device_resources_all,

cuda_core/cuda/core/_cpp/DESIGN.md

@@ -275,6 +275,37 @@ Related functions:
 - `peek_last_error()`: Returns the error without clearing it
 - `clear_last_error()`: Clears the error state
 
+Some functions return a `CUresult` directly instead of a handle (for example
+`context_synchronize`, `context_get_device`, `graph_node_set_params`). Their
+callers `HANDLE_RETURN` the value.
+
+### Context-scoped operations
+
+Operations that must run in a specific context use `invoke_in_context` /
+`invoke_in_context_or_undo` (propagating paths) and `cleanup_in_context`
+(deleters). They switch the current context, run the operation, and restore the
+caller's context. When restoration fails after the operation succeeded, the
+creation is undone and the restoration status is returned; the helper also
+records a thread-local detail (`take_last_error_detail()`) that the Cython error
+path appends to the raised `CUDAError`, so the user learns that the caller's
+context was not restored and which context is current. When both the operation
+and the restoration fail, the operation status is returned and the restoration
+failure is reported out of band. Tests inject restoration failures with
+`set_context_restore_fault_for_testing()`.
+
+### Reporting from non-propagating paths
+
+Deleters, CUDA callbacks and cleanup-after-failure cannot raise. They report
+through `report_cuda_error()` / `report_message()` (the `pw_*` wrappers
+decorate destroy calls with it), which emit a `cuda.core.CUDAWarning` through
+the Python warnings machinery when the interpreter is usable, deliver an
+escalated warning as an unraisable exception, and fall back to stderr when the
+GIL cannot be taken (for example during finalization). `CUDA_ERROR_DEINITIALIZED`
+is never reported because it means the driver is shutting down. No status is
+discarded silently anywhere in this layer, and nothing in this layer terminates
+the process; see `docs/source/error_handling.rst` and the "Failure handling"
+section of `AGENTS.md` for the policy.
+
 ## Usage from Cython
 
 ```cython