Skip to content

fix: guard oversized JSON responses in code-generation prompt - #76

Open
tkatta-stack wants to merge 1 commit into
Integuru-AI:mainfrom
tkatta-stack:fix/json-response-context-length-30
Open

tkatta-stack wants to merge 1 commit into
Integuru-AI:mainfrom
tkatta-stack:fix/json-response-context-length-30

Conversation

@tkatta-stack

@tkatta-stack tkatta-stack commented Sep 29, 2026 •

Copy link
Copy Markdown

What

generate_code() (in integuru/util/print.py) builds a single LLM prompt per DAG node to write that node's request-parsing function. For a text/html/application/javascript response it already guards against oversized bodies (>100000 chars) by substituting short context snippets instead of the full response. The application/json branch had no such guard — it always embedded the complete raw response text verbatim, on top of the key_paths already computed that pinpoint exactly where each extracted variable lives in the parsed JSON.

Why this causes #30

A single large JSON API response captured in the HAR file (a bulk data dump, a paginated listing, etc.) gets embedded in full. By itself this can push one prompt well past the model's context window — matching the reported "messages exceeded available capacity by roughly 3x" after a normal run, and the second reporter's identical experience.

Fix

Mirrors the existing 100000-character threshold from the HTML/JS branch onto the JSON branch. Past that threshold, the prompt drops the raw response text and relies solely on the already-computed key_paths, which are sufficient for the model to navigate the parsed JSON and write correct extraction code without seeing the raw payload. Small/typical JSON responses are unaffected — same prompt as before.

Testing

Added tests/test_generate_code.py:

  • a response with a 200,000-character value confirms the raw text is excluded from the prompt while the resolved key path is still present
  • a small response confirms the original (full-text) prompt is unchanged

Both pass with the fix; I also confirmed the large-response test fails against the pre-fix code (i.e. it genuinely catches the regression).

Note: tests/test_integration_agent.py currently fails locally/in CI independent of this change — it needs a test.har fixture file that isn't committed to the repo. Not touched here since it's unrelated to this fix; flagging for visibility.

Fixes #30

generate_code() (util/print.py) embeds the full HTTP response body in
the prompt it sends to the LLM when writing the parsing code for each
node. The text/html and application/javascript branch already guards
against oversized responses (>100000 chars) by substituting short
context snippets instead of the full body. The application/json
branch had no such guard: it always embedded the complete raw
response text verbatim, in addition to the already-resolved key_paths
that pinpoint where each extracted variable lives in the parsed JSON.

A single large JSON API response (a bulk data dump, a paginated
listing, etc.) captured in the HAR file is therefore embedded in full,
and can by itself push a single prompt well past the model's context
window -- this matches the reported symptom of "messages exceeded
available capacity by roughly 3x" after a normal integuru run.

This mirrors the existing 100000-character threshold from the
HTML/JS branch onto the JSON branch: past that threshold, the prompt
drops the raw response text and relies solely on the key_paths already
computed, which are sufficient for the model to navigate the parsed
JSON and write correct extraction code without needing to see the raw
payload.

Added tests/test_generate_code.py covering both the oversized-response
path (raw text excluded, key path included) and the normal small-
response path (behavior unchanged).

Fixes Integuru-AI#30

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WV3dWyboqjHodswTzHTBKr
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Maximum context length errors with OpenAI

3 participants