feat: add Granite 4.2 model defaults - #1587
Draft
planetf1 wants to merge 12 commits into
Draft
Conversation
8 tasks
…sting DO NOT PUSH — ibm-research/ namespace is internal/unreleased. Local evaluation on eval/granite-4.2-prerelease branch only. Signed-off-by: Nigel Jones <nigel_jones@uk.ibm.com> Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Switch every test fixture and direct OllamaModelBackend/LocalHFBackend
instantiation from granite-4.1 to granite-4.2 (ibm-research/ namespace,
ollama granite-4.2-{3b,8b,30b}:latest tags).
Key changes:
- Add ModelOption.THINKING: False to all direct OllamaModelBackend
instantiations (tracing, metrics, genstub, component_typing tests)
to prevent thinking-token slowdowns
- Add default_extra_body={chat_template_kwargs:{enable_thinking:False}}
to OpenAI backend fixtures (test_openai_ollama, test_metrics_backend)
- Add model_options={THINKING:False} to LiteLLM backend in test_metrics_backend
- HF tests (test_huggingface, test_spans, test_rag, test_core) updated to
ibm-research/granite-4.2-3b — accessible with HF_TOKEN
- Update granite4.1 string references in test_base_context context-length
prefix tests to use granite-4.2 Ollama tag equivalents
Eval-branch only — not for main until 4.2 is publicly released.
Assisted-by: IBM Bob
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
…runs
- test_huggingface.py: add _hf_model_for_eval() — reads GRANITE42_MODEL to
select IBM_GRANITE_4_2_{3B,8B,30B}; defaults to 3B
- test_ollama.py: add _ollama_model_for_eval() — same logic for Ollama tags;
start_session() and _ensure_model_warm now use the selected tag
LSF tasks updated in .local/ to pass GRANITE42_MODEL inside bsub bash -c body.
Assisted-by: IBM Bob
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
planetf1
force-pushed
the
feat/granite-4-2-defaults
branch
from
August 26, 2026 12:00
c798894 to
2eabb25
Compare
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Assisted-by: Codex Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Contributor
Tool PR ChecklistUse this checklist when adding or modifying tools in Protocol Compliance
Integration
|
Quality-matrix runs on PR generative-computing#1587 hit two Ollama stall failure shapes the existing retry net could not handle: - test_litellm_token_metrics_integration[non-streaming] died to the 900 s pytest-timeout watchdog: LiteLLMBackend passes no request deadline, so litellm falls back to a 600 s per-attempt timeout and the OpenAI SDK silently retries a stalled attempt (verified: 3 attempts on a stalled socket), exhausting the budget with no retryable error. - The flaky marker from generative-computing#1369 matched only the native httpx.ReadTimeout shape; the litellm path raises litellm.exceptions.Timeout whose APITimeoutError message never matched, so its retry was dead code. Changes: - Bound the litellm live test to the same 300 s the native OllamaModelBackend uses, with num_retries: 0 so one pytest attempt can never exceed the watchdog budget (verified end-to-end through LiteLLMBackend against a stalled socket: 1 request, 4.1 s at timeout=4). - Extend the conftest flaky marker to a shared OLLAMA_TIMEOUT_RERUN_PATTERNS constant covering both timeout shapes, and pin the match behaviour in test/test_flaky_ollama_rerun.py (including that the watchdog kill itself is NOT retried). - Capture the Ollama server log in the quality workflow and dump it on failure: server-side stalls were previously undiagnosable from the job log (zero server lines across a 90-minute test window). Local verification: new regression tests + litellm, native-Ollama metrics and tracing live tests pass against Ollama 0.33.0 / granite4.2:3b; ruff, mypy clean. Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Issue
Fixes #1583
Description
Make Granite 4.2 3B the default local text model, and add the public 3B, 8B,
and 30B identifiers. The PR also aligns CI, the GPU runner, and existing
Ollama/vLLM tests with the published model tags so the same model is selected
locally and in automation.
What changed
start_session,OllamaModelBackend, andLiteLLMBackend.the same Granite 4.2 text model.
catalogues do not provide Granite 4.2 weights.
dense text-model update; this PR does not introduce a Granite 4.2 vision
model.
than require a running Ollama service.
Why the test-runner changes are included
The runner provisions the models that live tests use. Changing the default
model therefore also requires updating the published Ollama tags, constraining
the test context window, and keeping the Ollama and vLLM test lanes isolated.
Without those changes, a clean test environment can select an unavailable model
or start incompatible inference servers together.
Things to check
ModelOption.THINKING=False; enhancement: huggingface should handle ModelOption.THINKING #1212 supplies the Hugging Face backend mappingfrom that public option to the loaded model's chat-template variable.
adapter tests. It should be exercised only on suitably provisioned GPU
hardware.
GGUF model, with a constrained context window.
work is completed.
Current status
128 deselected, 1 xpassed.
run (32972890653)
is now root-caused as two Ollama-stall failure shapes, not a model-defaults
defect: the non-streaming LiteLLM test hit the 900 s watchdog because that
path had no bounded request deadline and no retryable error, and the native
live tests exhausted the
ReadTimeoutretry. Commitc0278730bounds theLiteLLM test to the same 300 s as the native backend, extends the retry net
to the LiteLLM timeout shape (regression-pinned), and captures the Ollama
server log so future stalls are diagnosable. The remaining pre-existing
issues (server-side stall cause; an empty-response shape seen on
main)are tracked in #1589.
Neither result is treated as waived: the matrix must pass before this draft
is ready for review.
after enhancement: huggingface should handle ModelOption.THINKING #1212 merges.
Testing
uv run ruff format --check .uv run ruff check .uv run mypy .uv run pytest test/ -m "not qualitative"on the final head.Attribution
Adding a new component, requirement, sampling strategy, or tool?