Skip to content

fix(observability): tool span arguments sentinel + missing reasoning parts - #218

Merged
luk384090-cloud merged 4 commits into
dashscope:mainfrom
XudongZhang-1224:fix/observability-tool-args-and-llm-reasoning
Oct 27, 2026
Merged

luk384090-cloud merged 4 commits into
dashscope:mainfrom
XudongZhang-1224:fix/observability-tool-args-and-llm-reasoning

Conversation

@XudongZhang-1224

@XudongZhang-1224 XudongZhang-1224 commented Oct 26, 2026 •

Copy link
Copy Markdown

Description

Fix two silent-degradation bugs in the GenAI observability mapping layer
(dashscope/finetune/reinforcement/component/observability/genai/), both
observed in production RFT agentic-rollout traces and still present on
main @ v2.0.1:

  1. _bind_tool_arguments calls typing.Dict(bound.arguments); typing.Dict
    cannot be instantiated, so every @observe_tool call raises TypeError,
    which the broad except Exception swallows and replaces with
    {"_error": "could_not_bind_arguments"} as the span's tool-call arguments.
    sig.bind_partial itself succeeds; only the dict conversion crashes.
    Fix: use builtin dict().
  2. openai_completion_to_output_messages and
    dashscope_response_to_output_messages only map content and
    tool_calls; reasoning_content is never read, although the schema
    (opentelemetry.util.genai.types.Reasoning, semconv ReasoningPart)
    supports it and the trace UI renders it. Fix: export Reasoning from the
    _core facade and insert a Reasoning part first in each output message.

Related Issue: N/A — no upstream issue; reported from production traces
(dashscope 1.26.7, and live-confirmed on 1.27.6 in a running FC function).

Security Considerations: This change adds no logging of secrets and
exposes no credentials. (An unrelated pre-existing issue is noted under
Additional Notes.)

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactoring

Component(s) Affected

  • Model
  • Application
  • Common
  • Documentation
  • Tests
  • CI/CD

Checklist

  • Pre-commit hooks pass
  • Tests pass locally
  • Documentation updated (if needed)
  • Ready for review

Testing

  • Unit: against this source tree, _bind_tool_arguments returns the real
    kwargs (no sentinel); the OpenAI converter emits parts
    [reasoning, tool_call] for a completion carrying reasoning_content;
    the DashScope converter emits [reasoning]; a message without reasoning
    stays [text] (no regression).
  • Integration: with an in-memory SpanExporter and tracing enabled, the
    decorated tool path records gen_ai.tool.call.arguments as the real
    command JSON (no sentinel) plus gen_ai.tool.call.result; the decorated
    LLM path (demo-style result_mapper) records gen_ai.output.messages
    containing reasoning and tool_call parts.
  • Platform: an agentic-rl job running a wheel built from this branch
    (FC startup log: Successfully installed dashscope-2.0.1) shows real
    shell commands in bash tool span inputs and reasoning parts in chat span
    outputs in the trace UI.
  • The repository pytest suite has no coverage for this module; maintainer-side
    CI expected to run as usual.

Additional Notes

  • Compatibility: when loongsuite-util-genai is absent, _core.Reasoning
    is None and the new helper no-ops, preserving the module's
    silent-degradation contract; messages without reasoning_content are
    byte-identical to before.
  • Separate pre-existing issue observed while validating (not addressed here):
    the AgenticRL submit-path log prints the full job request payload at INFO,
    including component env vars that carry API keys. Suggest masking env
    values in that log in a follow-up.

yulan.zxd added 3 commits October 25, 2026 18:48
typing.Dict cannot be instantiated; the TypeError was swallowed by the broad except in _bind_tool_arguments, so every @observe_tool span recorded {"_error": "could_not_bind_arguments"} instead of the real arguments.
The OpenAI and DashScope output-message converters ignored reasoning_content, so chat spans never echoed model thinking although the schema (Reasoning part) supports it. Export Reasoning from the _core facade and insert the part first.
W0611 unused-import: Reasoning is re-exported for the converters in messages.py, not referenced inside _core.
@XudongZhang-1224
XudongZhang-1224 force-pushed the fix/observability-tool-args-and-llm-reasoning branch from 15293e3 to 5248631 Compare October 26, 2026 11:41
Import Reasoning in a nested try so environments with loongsuite-util-genai < 0.4.0 degrade to missing reasoning parts instead of GENAI_AVAILABLE=False.
@XudongZhang-1224
XudongZhang-1224 force-pushed the fix/observability-tool-args-and-llm-reasoning branch from 5248631 to be55a1b Compare October 26, 2026 11:45
@luk384090-cloud
luk384090-cloud merged commit f2a915e into dashscope:main Oct 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants