[pos-ei-don]

Description

Every chat request with stop fails on the vLLM backend with HTTP 500 (EngineGenerateError), streaming or not. Reproduce with any model on the vllm backend:

{"model": "<vllm model>", "messages": [{"role": "user", "content": "Count from 1 to 10"}], "stop": ["4"]}

Cause: _build_sampling_params constructs SamplingParams and then assigns the request values with setattr. The protobuf repeated field StopPrompts arrives as RepeatedScalarContainer, not list, and __post_init__ never runs. On current master with StopPrompts=["4"]:

• sp._verify_args() / sp.clone() → AssertionError (assert isinstance(self.stop, list))
• MsgpackEncoder().encode(sp) (what ships the params to the engine core) → TypeError: Object of type RepeatedScalarContainer is not serializable

Skipping __post_init__ also means the stop-string hold-back buffer (output_text_buffer_length) is never set, so the start of a multi-character stop string can be streamed, and seed=-1 is not mapped to None.

This PR collects the values, turns repeated fields into lists and constructs SamplingParams once, as vLLM's OpenAI server does.

Behaviour changes worth noting:

• Values vLLM rejects (e.g. presence_penalty outside [-2, 2], an empty stop string) now fail with vLLM's validation error instead of being set unchecked.
• With temperature=0, vLLM's __post_init__ now normalises to greedy (top_p=1.0, top_k=0). Sampling was already greedy; the existing test test_explicit_zero_temperature_and_seed_are_preserved checked top_p == 0.9 on a temperature-0 request, so that assertion now uses a sampled request.

Notes for Reviewers

New server-less tests in backend/python/vllm/test.py (TestSamplingParamsConstruction, no GPU/model): stop is a list and survives _verify_args/clone/msgpack, hold-back buffer, request values applied, seed -1, invalid values rejected. Run in the backend's venv against vLLM 0.31.0 and 0.32.0: all pass, together with the existing server-less classes. On unpatched master, 3 of the new tests fail with exactly the errors above.

Signed commits

• Yes, I signed my commits.
• Documentation updated (docs/content/) for user-facing changes, or not applicable

[localai-org-maint-bot]

@mudler source review complete against master 9c4a8b2a8: no blocking code findings. Constructing SamplingParams once lets vLLM normalize and validate the request values, and converting repeated fields to lists addresses the stop-parameter serialization failure.

I pushed the missing sampling documentation as 41ce77925 on this branch.

Validation: clean integration with current master and whitespace checks pass. I could not run the backend tests: the available Python executables require a loader absent from this environment. The contributor's reported test results have not been independently reproduced here.

Source-review sign-off, conditional on backend test validation and human DCO certification of the docs commit. The added commit has Assisted-by attribution and no AI Signed-off-by.