[2026.4] Gemma4 native tool calling: parser, guided generation and template hardening

[DassaultFalconKing]

Gemma4 emits native tool calls such as <|tool_call>call:tool_name{...}<tool_call|>, with its own quoted-string, reasoning and tool-response protocol. Treating those arguments as flat text can lose nested values or mistake markers inside strings for call boundaries. A tool response can also leave the Google-rendered prompt inside an already-open thought channel, which must agree with the generation grammar.

This draft brings the downstream Gemma4 tool-calling stack to releases/2026/4. The design follows the native OVMS parser/input-processing interfaces and OpenVINO GenAI structural tags. The architectural explanation follows the earlier Gemma4 PR #4525; the source here comes from the later frozen RC 170644006a5334cb971b05824e4a8c95b495c4e2, with newer framing, prompt-state and bounded-whitespace repairs.

What changes

Custom tool and reasoning parsers. Recursive native/JSON argument parsing supports nested objects/arrays, native quoted strings and parenthesized argument forms. It distinguishes call delimiters from quoted payload and preserves numeric lexical values rather than unnecessarily converting them through floating point. Registry-aware detection restricts executable names to request tools; bare call: recovery is limited to a logical line/phase boundary. Malformed candidates are bounded so a later valid call can still be parsed. Parser-owned boundaries and reasoning-to-tool routing keep framing consistent across streaming chunks.

Explicit generation policy. With no tools or tool_choice=none, no tool grammar is imposed. auto uses lazy TriggeredTags: ordinary text is legal until the native tool marker appears, then request names/schemas constrain the call. Required/named choices impose at least one call, restrict a named choice to that tool and retain hard constraints after validation failure. A Union accepts tools directly or a complete thought channel followed by tools. Both modes honor parallel_tool_calls through stop_after_first.

Multi-turn template compatibility. Capability-driven history adaptation converts tool-response JSON strings to mappings only for mapping-safe templates. Templates that iterate content parts via part.get(...) retain string semantics, avoiding the 'str object' has no attribute 'get' failure. An already-open thought channel in the rendered Google prompt is reconciled with the hard grammar instead of requiring a second opener.

Whitespace and termination repair. Every tool JSON schema receives max_whitespace_cnt=2 through the typed GenAI API, bounding otherwise legal tab/newline runs that could consume the output limit without a complete call. Terminal streamer flushes pass the actual LENGTH/STOP/TOOL_CALL reason and record incomplete-frame diagnostics.

Opt-in session state. The downstream disk journal and deterministic per-turn seed contract are included, activated through OVMS_SESSION_STORE_DIR and X-OVMS-Session-ID, with bounded storage/cache limits. This API/storage part needs separate upstream design and security review and can be split out.

The earlier PR documents Google as the protocol authority, with vLLM and llama.cpp as independent behavioral references for parsing and lazy tool activation. This implementation uses native OVMS/GenAI machinery. Windows deployment/build-policy changes and later cache diagnostics/cache-off experiments are excluded. Existing upstream logprob fixes, utility APIs and all Gemma4 regression cases are retained.

Six focused contract targets cover parsing/recovery/reasoning, generation, rendered-prompt state, endpoint parallel policy and template capabilities.

Validation and provenance

The frozen downstream RC, rather than this new upstream PR head, was exercised on Windows with Intel Arc 140V GPU, VLM_CB and Gemma4 26B A4B INT4:

• 14/14 benchmark requests returned HTTP 200. Weighted long-response throughput: 24.624 tok/s (curl), 25.231 tok/s (Invoke-WebRequest), actual completion usage / total client wall time.
• Exact single, two same-name parallel, and SSE echo semantics: 3/3 PASS, including count, arguments and tool_calls finish.
• Raw synthetic requests/responses, summaries and the dependency patch are under docs/gemma4.
• Packaged RC and provenance.

Why draft

Dependency blocker: target GenAI lacks JSONSchema(schema, optional whitespace_bound). Companion draft GenAI #4477 now carries the C++ API, frozen XGrammar revision and four serialization/equality tests. The paired changes still require review, merge and a coordinated dependency update; this OVMS PR cannot compile with the unmodified target dependency.

Standalone checks against the real companion GenAI header: original API fails compilation (RED); transferred API compiles and passes legacy/bounded/zero/equality assertions (GREEN). Full GenAI/native GTests are not claimed as passed.

Upstream license scanner preflight with Linux paths/UTF-8: PASS after adding headers to the three new BUILD files and packaging raw SSE/CSV traces without altering their contents. The old #4525 missing-ownsToolCallBoundaries wiring is present here.

Jenkins job 1 ran on initial OVMS head dd7ac8d and reported ERROR. Its Intel-hosted console hostname could not be resolved from the preparation host, so the exact failure is not classified; an accessible console log is needed. The complete Linux/Windows pipeline remains an open gate on the updated head. Local product build/executable contracts/retained upstream regressions on this assembled OVMS head: NOT RUN. Full multi-turn/real-tool/session acceptance is open. Journal persistence and endpoint-wide hard-choice validation may need separate PRs. See docs/gemma4/UPSTREAM-HANDOFF.md for scope, provenance, targets and merge gates.