Skip to content

Improve error handling in prompt rendering - #2403

Closed
Mengxi-Xu wants to merge 1 commit into
abetlen:mainfrom
Mengxi-Xu:main
Closed

Mengxi-Xu wants to merge 1 commit into
abetlen:mainfrom
Mengxi-Xu:main

Conversation

@Mengxi-Xu

Copy link
Copy Markdown

Jinja2ChatFormatter renders chat templates from model metadata using jinja2’s sandboxed environment. When a template calls range() with more than MAX_RANGE (100000) items, the sandbox raises OverflowError — which is not a TemplateError subclass. Since call did not catch it, the exception escaped and crashed the caller (e.g. every POST /v1/chat/completions request for a model whose GGUF embeds such a template returned a raw traceback instead of a clean error).

Fix: wrap the render(...) call and translate OverflowError into ValueError, matching the error contract already used by the raise_exception helper in the same method. The sandbox behavior itself is unchanged — only the error surface is normalized

Jinja2ChatFormatter renders chat templates from model metadata using jinja2’s sandboxed environment. When a template calls range() with more than MAX_RANGE (100000) items, the sandbox raises OverflowError — which is not a TemplateError subclass. Since __call__ did not catch it, the exception escaped and crashed the caller (e.g. every POST /v1/chat/completions request for a model whose GGUF embeds such a template returned a raw traceback instead of a clean error).

Fix: wrap the render(...) call and translate OverflowError into ValueError, matching the error contract already used by the raise_exception helper in the same method. The sandbox behavior itself is unchanged — only the error surface is normalized
@Mengxi-Xu Mengxi-Xu closed this by deleting the head repository Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant