Allow scheduling of CF completion in parallel

[fxbonnet]

When a batch loading function returns, we currently complete all CompletableFutures sequentially on the calling thread. Since CompletableFuture.complete(...) also executes any continuations attached via thenApply(...) or thenCompose(...), large batches can significantly delay the return of dispatch().

This PR introduces an option to execute these continuations in parallel instead.

[bbakerman]

This misses the Try path of completion

[andimarek]

@fxbonnet I am not sure about this PR: with the new hook introduced by @bbakerman in #275 you can already offload every completion onto a new thread and hence it will run in parallel if you choose todo so. I have problems seeing how this approach is different/better.

[fxbonnet]

@andimarek I created this PR after my comment on @bbakerman 's PR #275 (comment)
The PR allowed to offload the completions to a separate thread, but in case the batch load contains multiple keys, the completions for each key would still happen sequentially.
For clarity I just rebased this PR onto @bbakerman 's branch. The main change is that instead of passing a single Runnable to scheduleCompletion, we would pass a List<Runnable> (one for each key), giving the option to execute them in parallel.

[fxbonnet]

clearCacheKeys may now be accessed concurrently by multiple thread if we execute the Runnables in parallel.

[bbakerman]

good call - price of side effects outside a thread

[andimarek]

Requesting changes for a semantic regression around ValueCache hits and context-aware batch loaders.

I opened #278 as a small draft PR showing one possible fix: #278

The PR now constructs a BatchLoaderEnvironment before value-cache filtering:

BatchLoaderEnvironment environment = mkBatchLoaderEnv(keys, keyContexts);
CompletableFuture<List<V>> batchLoad = invokeLoader(environment, keys, keyContexts, queuedFutures, loaderOptions.cachingEnabled());

But invokeLoader(..., cachingEnabled=true) can then remove value-cache hits and invoke the actual batch loader with only missedKeys / missedKeyContexts while still passing the original environment:

CompletableFuture<List<V>> batchLoad = invokeLoader(environment, missedKeys, missedKeyContexts, missedQueuedFutures);

That breaks the existing semantics of BatchLoaderEnvironment: environment.getKeyContextsList() is expected to be aligned with the keys argument passed to the loader. A concrete failure case:

1. The queued dispatch contains keys=[A, B] and keyContexts=[ctxA, ctxB].
2. ValueCache returns a hit for A and a miss for B.
3. The actual batch loader is invoked with keys=[B].
4. The reused environment was built from [A, B], so environment.getKeyContextsList().get(0) is ctxA, not ctxB.

That means a BatchLoaderWithContext implementation that indexes environment.getKeyContextsList() alongside its keys argument can load B using A's context. This is observable behavior and can be a security/data-isolation issue if key contexts carry tenant, auth, locale, or request-scoped metadata.

Please build the BatchLoaderEnvironment from the actual key/context list passed to the concrete loader call after value-cache filtering, or otherwise preserve the invariant that environment.getKeyContextsList() is ordered and sized to match the loader's keys argument. This should also get a regression test with a partial ValueCache hit and a context-aware batch loader asserting that the missed key receives its own context.

PR #278 does exactly that by rebuilding the environment for missedKeys / missedKeyContexts before the concrete loader invocation and adding the regression test.

[andimarek]

I opened a small draft PR with one possible fix for the ValueCache / BatchLoaderEnvironment context alignment issue: #278

[fxbonnet]

Thanks @andimarek I have added your commit to the PR