diff --git a/demos/README.md b/demos/README.md index e380580705..ffe5cc0508 100644 --- a/demos/README.md +++ b/demos/README.md @@ -87,4 +87,5 @@ Integrations | Demo | Description | |---|---| |[Integration with Open WebUI](integration_with_OpenWebUI/README.md)|Using Open WebUI with OVMS as inference provider. Shows text and image generation as well as usage with RAG and tools.| -|[Visual Studio Code assistant](./code_local_assistant/README.md)|Use Continue or Cline extension to Visual Studio Code with local OVMS serving.| +|[Visual Studio Code assistant](./code_local_assistant/README.md)|Use Continue extension in Visual Studio Code with local OVMS serving.| +|[Cline coding agent integration](./cline_integration/README.md)|Use the Cline coding agent in Visual Studio Code with local OVMS serving, including MCP, image input and reasoning_effort usage.| diff --git a/demos/cline_integration/README.md b/demos/cline_integration/README.md new file mode 100644 index 0000000000..531fe23ace --- /dev/null +++ b/demos/cline_integration/README.md @@ -0,0 +1,123 @@ +# Cline Coding Agent with OpenVINO Model Server {#ovms_demos_cline_integration} + +## Intro +[Cline](https://cline.bot/) is an autonomous coding agent extension for Visual Studio Code. It can read and edit +files, run terminal commands, browse the web and call tools exposed by MCP servers, all driven by an LLM reachable +through an OpenAI-compatible API. This demo shows how to serve that LLM locally with OpenVINO Model Server (OVMS) +so that Cline runs entirely on your own machine, without sending code or prompts to an external service. + + +## Requirements +- Windows (standalone package) or Linux (Docker) +- Visual Studio Code with the [Cline extension](https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev) installed +- Make sure that Cline works with `next` version (to change it go to Visual Studio Code settings and search `@ext:saoudrizwan.claude-dev`, check `next` option from dropdown) +- Hardware: Tested on Intel Core Ultra iGPU with 32GB RAM and a discrete Intel Arc B70 GPU (dedicated VRAM). `Qwen3-Coder-Next` can be deployed on iGPU with 64GB RAM on board. +- Memory requirements depend on the chosen model (see table below) + +![extension_webpage](./extension_webpage.png) + +## Suggested models + +| Model | HF Link | Notes | +|---|---|---| +| `OpenVINO/Qwen3.8-27B-int8-ov` | [link](https://huggingface.co/OpenVINO/Qwen3.8-27B-int8-ov) | Vision-capable (VLM); general purpose chat/agent model | +| `OpenVINO/Qwen3.6-35B-A3B-int4-ov` | [link](https://huggingface.co/OpenVINO/Qwen3.6-35B-A3B-int4-ov) | Vision-capable (VLM); general purpose chat/agent model | +| `OpenVINO/Muse-Glimmer-30B-int4-ov` | [link](https://huggingface.co/OpenVINO/Muse-Glimmer-30B-int4-ov) | Vision-capable (VLM); general purpose chat/agent model | +| `OpenVINO/gpt-oss-20b-int4-ov` | [link](https://huggingface.co/OpenVINO/gpt-oss-20b-int4-ov) | General purpose chat/agent model | +| `OpenVINO/Qwen3-Coder-Next` | **To be published soon** | Big coding model; available only on iGPU with minimum 64GB of RAM | +| `OpenVINO/Qwen3.5-9B-int4-ov` | [link](https://huggingface.co/OpenVINO/Qwen3.5-9B-int4-ov) | Smaller model, use when RAM/VRAM is limited or for quick, low-latency edits; not recommended for harder coding tasks | + +## Deploy OVMS + +### Windows: deploying on bare metal + +All models can be deployed with the same command, only `--source_model` parameter should be changed to desired model. +```bat +mkdir c:\models +ovms --model_repository_path c:\models --source_model OpenVINO/Qwen3.8-27B-int8-ov --rest_port 8000 +``` + + +### Linux: via Docker + +All models can be deployed with the same command, only `--source_model` parameter should be changed to desired model. +```bash +mkdir -p ${HOME}/models +export GPU_ARGS=$(if ls /dev/dri/render* >/dev/null 2>&1; then echo "--device /dev/dri --group-add $(stat -c '%g' /dev/dri/render* | head -n1)"; fi) +docker run -d -p 8000:8000 --rm --user $(id -u):$(id -g) -v ${HOME}/models:/models/:rw ${GPU_ARGS} \ + openvino/model_server:latest-gpu \ + --model_repository_path /models --source_model OpenVINO/Qwen3.8-27B-int8-ov --rest_port 8000 +``` + +## Set Up Visual Studio Code + +### Install the [Cline extension](https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev) + +### Point Cline at your OVMS instance + +Open Cline's settings and add a new API provider configuration: +- **API Provider:** `OpenAI Compatible` +- **Base URL:** `http://localhost:8000/v1` +- **API Key:** any placeholder value, e.g. `unused` (unless configured on OVMS server side) +- **Model ID:** value provided as `--source_model`, e.g. `OpenVINO/Qwen3.8-27B-int8-ov` or as the `--model_name` e.g. `ovms-model` + +![examplary_configuration](./examplary_configuration.png) + +Cline lets Plan mode and Act mode use different models/providers, so you can, for example, use +`Qwen3.6-35B-A3B-int4-ov` for Plan-mode reasoning and switch to `Qwen3.8-27B` (on B70) for Act-mode code changes. +You can set that checking **Use different models for Plan and Act modes**. + +Running two large models like these at the same time can exceed available RAM/VRAM, so configure OVMS's +[idle servable management](../../docs/idle_servable_management.md) (preview feature) to keep only the model used by +the active mode loaded; switching modes in Cline then wakes up the other model on demand instead of requiring both +to be resident at once. + +## Usage examples + +### Chat (Plan mode) +Ask a question about the codebase without letting Cline make changes: +```text +Explain how request validation works in this repository and list the files involved. +``` + +### Coding / agentic (Act mode) +Switch to Act mode and give Cline a task that requires editing files and running commands: +```text +Add a unit test for the `parseConfig` function and run the test suite to confirm it passes. +``` +Cline will use its built-in tools (file read/write, terminal, browser) to make the change and verify it, relying on +OVMS's auto-detected tool-call format (derived from the model's chat template) to turn the model's tool calls into +these actions. + +### Image input +With `Qwen3.8-27B-int8-ov`, `Qwen3.6-35B-A3B-int4-ov` or `Muse-Glimmer-30B-int4-ov` deployed as a VLM, attach an +image to a Cline chat message, e.g.: +```text +[attach a screenshot of a UI bug] +What is wrong with this layout and which CSS file should I fix? +``` +Image attachments are only available when the configured model declares `supportsImages` in Cline's model +configuration; enable it for the custom OpenAI-compatible model entry when using a vision-capable model like +`Qwen3.8-27B-int8-ov`. + +## `reasoning_effort` usage + +Cline exposes a reasoning-effort control that is sent as part of the chat +completion request. OVMS supports the OpenAI API `reasoning_effort` field natively, so this setting is honored +across all of the suggested models above, regardless of the underlying chat template. +It may be changed in Cline's **Settings** under **Reasoning Effort** section. + +## Coding task + +For coding task example check [CodingAgenticWorflow Demo](https://github.com/intel-samples/agentic-demos/tree/main/CodingAgenticWorkflow#setup-ovms-server). + +## Agentic and coding capabilities summary + +- **Agentic**: Act mode combines the model's tool-calling ability (auto-detected by OVMS from the model's chat + template) with Cline's built-in tools (terminal, file read/write, browser) and any MCP servers you register, + letting Cline plan and execute multi-step tasks autonomously. +- **Coding**: Use `Qwen3.8-27B` on B70 or `Qwen3-Coder-Next` on iGPU for the strongest code generation/editing quality among the suggested models; + fall back to `Qwen3.5-9B` for lightweight or latency-sensitive edits. +- **MCP**: any MCP server (weather, filesystem, browser, etc.) becomes available to Cline the same way it is + available to the OpenAI Agents SDK example in the agentic AI demo. +- **Image input**: use `Qwen3.8-27B`, `Qwen3.6-35B-A3B-int4-ov` or `Muse-Glimmer-30B-int4-ov` for prompts that include screenshots or diagrams. diff --git a/demos/cline_integration/examplary_configuration.png b/demos/cline_integration/examplary_configuration.png new file mode 100644 index 0000000000..b569ee8a47 Binary files /dev/null and b/demos/cline_integration/examplary_configuration.png differ diff --git a/demos/cline_integration/extension_webpage.png b/demos/cline_integration/extension_webpage.png new file mode 100644 index 0000000000..4dfd667533 Binary files /dev/null and b/demos/cline_integration/extension_webpage.png differ diff --git a/demos/integrations.md b/demos/integrations.md index 54cd10985b..a886d1ab2b 100644 --- a/demos/integrations.md +++ b/demos/integrations.md @@ -7,6 +7,7 @@ maxdepth: 1 OpenWebUI VSC Continue dev +Cline demo ```