Fix #1641: Handle cleanup when exceptions are thrown

[mdboom]

In driver, runtime and nvrtc, the generated code follows the following pattern:

def func(args):
    $init
    $precall
    $call
    $postcall
    $return

For certain argument types, $precall and $postcall are required to be pairs, such as malloc/free. If an exception is thrown when parsing one of the other arguments in $precall, $postcall will not get called, leaking memory or other resources.

This reorganizes the code (whenever $postcall is non-empty) to:

def func(args):
    $init
    try:
        $precall
        $call
    finally:
        $postcall
    $return

This diff is larger than it otherwise might need to be since Cython only allows cdef declarations at the top level. So all the cdef's that used to be part of $precall need to be moved to $init.

Performance impact

try/finally is implemented by Cython with gotos. It generates two copies of the finally clause: one for success and one for failure, so there is a cost in overall code size. (This is similar to CPython's bytecode implementation of try/finally). However, the runtime performance penalty is below the noise threshold in the #659 benchmark. There is a small performance penalty of around 200ns for the error case, but that's to be expected -- the old implementation leaked memory.

Alternatives considered

We could use C++ RAII to automatically free memory. In fact, some of our code in these files already does that in the use of std::vector. However, this is not generally applicable as a solution, since it's not possible to implement a custom RAII struct Cython that is not also a PyObject. This would make the HelperVoidPtrStruct optimization impossible.

We could require that $init does all validation (but not any resource allocation), and the $precall would never raise, so $postcall would always be guaranteed to run. It seems like this may have been part of the original design, but then has not been strictly enforced everywhere over time. Even if we could get to that, it has two problems. (1) Some exceptions in $precall are unavoidable, even if the inputs are valid, such as malloc failing. (2) Doing a separate validation and conversion pass on all the arguments is never going to be as performant as a single pass.

[copy-pr-bot]

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[mdboom]

/ok to test

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-1669/
https://nvidia.github.io/cuda-python/pr-preview/pr-1669/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-1669/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-1669/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[mdboom]

/ok to test

[leofang]

Q: Maybe we should encapsulate _helper_input_void_ptr and _helper_input_void_ptr_free as a C++ class (not Cython cdef class, which was the previous approach IIRC)?