feat(parakeet-cpp): dynamic batching for concurrent transcription requests

[localai-bot]

Summary

Adds dynamic batching to the parakeet-cpp backend so concurrent
/v1/audio/transcriptions requests are coalesced into one batched call through
parakeet.cpp's batched encoder. This is a GPU throughput feature: under
concurrent load the batched-GEMM path raises utilization. On CPU it does not
help (the GEMMs already saturate the threads and padding adds work), so it is
disableable.

What changed (all under backend/go/parakeet-cpp/):

• New in-process batcher (batcher.go): handler goroutines submit requests; one
  dispatcher goroutine accumulates them until batch_max_size or
  batch_max_wait_ms, then makes a single batched engine call. The dispatcher is
  the sole caller of the C engine, so engine access stays single-threaded.
• The backend drops base.SingleThread (which serialized every call) for
  base.Base, so concurrent AudioTranscription handlers actually run and reach
  the batcher. An engineMu keeps the streaming path and batched-unary mutually
  exclusive on the one shared engine context.
• AudioTranscription decodes the file, submits to the batcher, and shapes the
  per-item JSON exactly as before (text, word/segment timestamps, tokens).
• Two model options: batch_max_size (default 8) and batch_max_wait_ms
  (default 15). batch_max_size: 1 disables batching (recommended on CPU).
• Docs: docs/content/features/audio-to-text.md.

Dependency

Requires the parakeet.cpp side that adds the
parakeet_capi_transcribe_pcm_batch_json C-API (batched transcription with
timestamps) and bumps the ABI to 2. The backend binds that symbol via purego at
runtime, so this Go code builds without it, but the backend image must ship a
libparakeet.so built from that parakeet.cpp branch for the feature to work.

Test plan

• Pure-Go batcher unit tests pass under -race: go test ./backend/go/parakeet-cpp/ -run TestBatcher -race (coalescing, size trigger, window trigger, size-1 bypass).
• go build / go vet clean (one pre-existing unrelated unsafe.Pointer warning).
• End-to-end (reviewer / GPU host): build the backend image with the updated libparakeet.so, then make test-extra-backend-parakeet-cpp-transcription; fire N concurrent transcription requests and confirm correct per-request transcripts, and that throughput improves at batch_max_size > 1 on GPU.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]