Add configurable reasoning preservation for openai completions #268
+111
−38
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zikajk
force-pushed
the
configurable-reasoning-history
branch
from
a94e839 to
1be0ff3
Compare
ericdallo
reviewed
Jan 15, 2026
CHANGELOG.md
Outdated
|
|
||
| ## Unreleased | ||
|
|
||
| - (OpenAI Chat) - Configurable reasoning history via `keepHistoryReasoning` (model-level, default: prune) |
Member
There was a problem hiding this comment.
I wonder if this default for all openai-chat is a good idea or if we should have it enabled only for google provider in config.clj, also shouldn't this be a flag by provider and not model?
Member
Author
There was a problem hiding this comment.
Based on what I've found in Opencode and OpenAI SDK it is default for every model.
- OpenCode transform
<think>block to message of type reasoning and this type is ignored by OpenAI SDK and never reach the LLM. - Delta reasoning messages are usually persisted in one turn and then thrown away (as you can see in GLM docs bellow, first vs second diagram)
It shouldn't be by provider because you might want to configure it only for e.g. GLM-4.7 (second diagram here ->
https://docs.z.ai/guides/capabilities/thinking-mode).
Member
There was a problem hiding this comment.
ok, let's keep an eye on that if affects any openai-chat model, I still think it's a little bit dangerous move, but if openai sdk is really doing that it may be ok
Member
Author
|
@ericdallo If you approve I will merge it tommorow after fixing integration tests. |
ericdallo
reviewed
Jan 16, 2026
zikajk
changed the title
zikajk
changed the title
zikajk
force-pushed
the
configurable-reasoning-history
branch
2 times, most recently
from
8e3e3f2 to
776c92f
Compare
zikajk
changed the title
zikajk
force-pushed
the
configurable-reasoning-history
branch
2 times, most recently
from
c2d8af8 to
446bbb1
Compare
Standard behavior: discard reasoning before last user message (based on OpenAI SDK). Model-level config enables preservation (e.g. GLM-4.7 with preserved thinking). Changes: - prune-history: add keep-history-reasoning parameter - Tests: cover both pruning and preservation - Docs: add keepHistoryReasoning to model schema
Introduce a more granular control for reasoning retention in requests: "all" (default, send everything) "turn" (current turn only) "off (discard all) Both delta-reasoning (reasoning_content) and think-tag reasoning are handled uniformly. DB storage is unaffected. Reasoning is always persisted for UI display. This setting only controls what gets sent back to the model.
zikajk
force-pushed
the
configurable-reasoning-history
branch
from
446bbb1 to
4a4d990
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Discard reasoning before last user message (based on OpenAI SDK).
Model-level config enables preservation (e.g. GLM-4.7 with preserved thinking).
Changes:
prune-history: add keep-history-reasoning parameter
Tests: cover both pruning and preservation
Docs: add keepHistoryReasoning to model schema
I added a entry in changelog under unreleased section.
@ericdallo Subjectively, the behavior seems better now. Since it's difficult to evaluate, I went through the OpenAI SDK with Opencode via Opus and it found that they discard it as well. And I couldn't find any documentation that would suggest we should do otherwise by default.