Batch arena records for reverse-mode matrix arithmetic

[sims1253]

I've been working on the allocation and registration cost of scalar autodiff records produced by Matrix<var> arithmetic. The idea is to allocate output records together and register one span, while keeping the existing expression interface.

The PR on my Math fork covers addition, subtraction, division, elementwise multiplication, and matrix multiplication. It also updates adjoint reset, nested recovery, and profiling to account for the spans.

An earlier hier_2pl benchmark showed 5–7% less wall time and 17.8% fewer whole-run instructions. The current version passes 39 focused tests, including nested memory lifetimes, repeated gradients, empty matrices, and mixed storage orders.

Would this approach be worth bringing upstream? I could start with the allocation/lifecycle support and one operation if you'd prefer a smaller first PR.

[SteveBronder]

Reading that PR it is not clear to me what the benefit of that is. Can you explain? It is adding a lot of machinery that seems to be duplicating work

[sims1253]

When an operator produces a Matrix<var> it creates an autodiff record for each output element. The change allocates these records together and registers the group once. That should reduce allocation and bookkeeping overhead but the underlying calculations remain the same.
I guess the argument for this version is that it speeds up existing code without need for change.

Putting the allocation choice in a helper and keeping one callback per operation would keep most of the benefit and reduce the diff a lot.
It'd also be possible to just batch the allocation and register them with the current.

[SteveBronder]

Yes I see make_nochain_vari_array says it is doing that, but reading the code I do not see how it would be more useful than what we are currently doing. The current code uses an arena allocator already and allocated the memory for a matrix all at once

[sims1253]

Yes, the matrix storage already uses the arena. This targets the scalar vari records behind the Matrix<var> handles.: Those still make individual arena allocation requests and register individually.

[SteveBronder]

If you just want to make all of the vari at once you can do that on our standard arena allocator and remove vari_no_stack etc. You could test writing a specialization of var for arena_matrix that does a bulk allocation of vari. Though idt it will give much of a speed improvement. What are you using for benchmarking?