feat: support Intern-S2-Preview TS forecaster

[CUHKSZzxy]

Summary

• Add Intern-S2-Preview time-series encoder and forecast support in the PyTorch backend.
• Route supported time-series requests to structured forecast outputs, including point forecasts, quantile forecasts, and predicted horizon metadata.
• Document the time-series understanding and forecasting request/response format for English and Chinese docs.

Validation

• Focused pre-commit checks for the touched Python and docs files passed.
• Focused Python syntax checks for the time-series forecaster passed.
• Real API smoke testing verified forecast responses with and without an explicit forecast horizon.

Assistance

Assisted with Codex + GPT-5.6-Sol xHigh, reviewed manually

[Copilot]

Pull request overview

This PR adds Intern-S2-Preview time series understanding + forecast support end-to-end in LMDeploy’s PyTorch backend, including routing forecast-capable requests to structured forecast payloads through the engine and OpenAI-compatible API responses.

Changes:

• Introduces InternS2Preview model implementations (TS encoder + optional TS forecaster) and forecast postprocessing that suppresses text tokens for forecast-only responses.
• Plumbs per-request forecast metadata through ModelInputs/StepContext, async engine outputs, and OpenAI protocol streaming/non-streaming responses (ts_forecast field).
• Updates docs and tests for time-series inputs and Intern-S2-Preview forecast examples; adds TS channel metadata propagation.

Reviewed changes

Copilot reviewed 34 out of 35 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/test_lmdeploy/test_vl/test_preprocess_utils.py	Adds TIME_SERIES modality token id and a test ensuring ts_channels is preserved.
tests/test_lmdeploy/test_ts_forecast_postprocessor.py	New tests for forecast-only suppression, chunk-state carry, and model-input meta merge/index-select behavior.
lmdeploy/vl/model/qwen3_5.py	Removes InternS2Preview handling from Qwen3.5 VL model (moved to dedicated class).
lmdeploy/vl/model/preprocess_utils.py	Includes ts_channels when expanding TIME_SERIES multimodal items.
lmdeploy/vl/model/interns2_preview.py	New VL wrapper for Intern-S2-Preview with TS preprocessing (normalization/truncation + token expansion).
lmdeploy/vl/model/builder.py	Registers InternS2PreviewVisionModel in the VL model builder imports.
lmdeploy/vl/model/base.py	Adds ts_channels mapping and enforces single TS input per request in preprocess.
lmdeploy/serve/openai/protocol.py	Extends OpenAI request schema with enable_forecasting/forecast_horizon and response message schema with ts_forecast.
lmdeploy/serve/openai/api_server.py	Plumbs forecast params into generation config and attaches ts_forecast to the final streamed/non-streamed assistant message.
lmdeploy/serve/core/async_engine.py	Adds multimodal_outputs to GenOut/Response and adjusts finish-reason handling for forecast-only outputs.
lmdeploy/pytorch/strategies/ar/sequence.py	Filters suppressed placeholder token ids (-1) from generated history for non-input updates.
lmdeploy/pytorch/strategies/ar/model_inputs.py	Preserves multimodal_output_metas across decoding, merges, and index-select operations.
lmdeploy/pytorch/models/whisper.py	Adds helper to synthesize missing Whisper K-proj bias for packed QKV loading.
lmdeploy/pytorch/models/qwen3_vl.py	Removes TIME_SERIES multimodal handling from Qwen3-VL preprocessing (now errors on unsupported modality).
lmdeploy/pytorch/models/qwen3_5.py	Removes time-series input plumbing from Qwen3.5 model path and ensures multimodal input list ignores None.
lmdeploy/pytorch/models/qwen3_5_moe.py	Removes legacy TS model construction from Qwen3.5 MoE base (InternS2Preview has dedicated implementation).
lmdeploy/pytorch/models/module_map.py	Maps InternS2Preview architectures to the new PyTorch InternS2Preview implementation.
lmdeploy/pytorch/models/interns2_preview.py	New InternS2Preview PyTorch model: splits vision vs TS multimodals, pads TS tensors, and runs optional forecaster + payload conversion.
lmdeploy/pytorch/models/interns2_preview_ts_forecaster.py	New TimeOmni_v2 forecaster implementation (aligner + backbone + quantile outputs + horizon head).
lmdeploy/pytorch/models/interns2_preview_ts_encoder.py	New InternS2Preview TS encoder using Whisper-style layers + chunking + channel handling.
lmdeploy/pytorch/models/interns1_pro.py	Switches InternS1Pro TS encoder import path and applies Whisper fake-bias helper during weight loading.
lmdeploy/pytorch/models/interns1_pro_ts_encoder.py	New/renamed InternS1Pro TS encoder module (previously interns1_pro_time_series).
lmdeploy/pytorch/models/bert.py	New minimal BERT implementation used by InternS2Preview TS encoder components.
lmdeploy/pytorch/model_inputs.py	Adds multimodal_output_metas to ModelInputs and StepContext and propagates it in StepContext.new().
lmdeploy/pytorch/messages.py	Adds forecast_horizon + ts_forecast to SamplingParam and maps from GenerationConfig.
lmdeploy/pytorch/engine/model_agent/ts_forecast.py	New TSForecastPostprocessor for chunk-state carry and forecast-only output mutation.
lmdeploy/pytorch/engine/model_agent/agent.py	Integrates TSForecastPostprocessor into the sampling/postprocess pipeline and carries multimodal_outputs.
lmdeploy/pytorch/engine/inputs_maker.py	Adds per-request multimodal_output_metas creation gated to AR + non-spec decoding.
lmdeploy/pytorch/engine/engine.py	Adds multimodal_outputs to InferOutput for engine response transport.
lmdeploy/pytorch/engine/engine_loop.py	Carries multimodal_outputs to EngineOutput; suppresses logprobs for forecast-only steps.
lmdeploy/pytorch/engine/engine_instance.py	Parses and forwards multimodal_outputs from async stream responses into outputs.
lmdeploy/metrics/stats.py	Adjusts request stats handling for completions that can finish without emitting text tokens.
lmdeploy/messages.py	Adds GenerationConfig fields for forecasting and plumbs multimodal_outputs through Response/EngineOutput.
docs/zh_cn/multi_modal/multimodal_inputs.md	Updates time-series support note and adds Intern-S2-Preview forecast + streaming examples.
docs/en/multi_modal/multimodal_inputs.md	Updates time-series support note and adds Intern-S2-Preview forecast + streaming examples.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.