Skip to content

--no-cnv and --simple-io flags are ignored, forcing conversation mode and garbled output #9

@Buildit1964

Description

@Buildit1964

Name and Version

Build: 7134 (Commit: 96ac5a2)
Built with cc (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0 for x86_64-linux-gnu

Operating systems

Linux

GGML backends

HIP

Hardware

AMD Radeon PRO W6800 (gfx1030) + AMD Radeon Graphics (gfx1036)

Models

1.deepseek-coder-33b-base.Q4_K_M.gguf
2.deepseek-llm-7b-chat.Q5_K_M.gguf
3.dolphin-2.9.4-llama3.1-8b-Q4_K_M.gguf
4.MS3.2-PaintedFantasy-24B.Q4_K_M.gguf
5.Qwen2.5-VL-32B-instruct-Q4_K_M.gguf
6.WizardLM-13B-Uncensored-Q4_K_M.gguf

Problem description & steps to reproduce

Summary: The --no-cnv and --simple-io flags are ignored by the binary, forcing the model into conversation mode and causing garbled output. The only workaround is to manually format prompts in ChatML.

Steps to reproduce:

  1. Build from the current ROCm main branch (commit 96ac5a2) with: cmake .. -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1030 -DCMAKE_BUILD_TYPE=Release
  2. Run a model with a chat template (like Dolphin) using the flag meant to disable conversation mode: ./bin/llama-cli -m dolphin-2.9.4-llama3.1-8b-Q4_K_M.gguf -p "Hello" -n 50 --ctx-size 4096 -ngl 25 --temp 0.7 --no-cnv
  3. Observe the logs: main: chat template is available, enabling conversation mode (disable it with -no-cnv) followed by main: interactive mode on. and garbled output (e.g., `B!5F%+@F.C)Is there a requirements.txt ? ggml-org/llama.cpp#8...").
  4. The --simple-io flag exhibits the same behavior.
  5. Workaround: Manually formatting the prompt in ChatML (<|im_start|>system...) produces correct text, proving the model and GPU work fine.

Expected behavior: The --no-cnv flag should disable conversation mode and allow simple prompt completion without requiring manual ChatML formatting.

First Bad Commit

No response

Relevant log output

Command: ./bin/llama-cli -m /home/twm/ai_models/dolphin-2.9.4-llama3.1-8b-Q4_K_M.gguf -p "Hello" -n 50 --ctx-size 4096 -ngl 25 --temp 0.7 --no-cnv

Key log lines:
...
main: chat template is available, enabling conversation mode (disable it with -no-cnv)
*** User-specified prompt will pre-start conversation, did you mean to set --system-prompt (-sys) instead?
main: interactive mode on.
...
user
Hello
assistant
B!5F%+@F.C)#8-")420D'0<<,C8<8<'GB(HB'&098&!,G+7#)2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions