Skip to content

command_reference.md

Latest commit

 

History

History
151 lines (125 loc) · 3.82 KB

command_reference.md

File metadata and controls

151 lines (125 loc) · 3.82 KB
Raw

CLI Command Reference (cli/)

InferenceOS provides a 21-command terminal interface accessible via inferenceos <command>.

Command Matrix

1. chat

Launch an interactive terminal UI chat session with live streaming, modal model pickers, floating slash autocompletion, and Kahneman fast/slow hybrid symbiosis.

# Standard generative chat
inferenceos chat models/llama-3-8b.gguf --theme nord

# Hybrid Symbiosis (System 1 Reflex + System 2 Deep Reasoning with auto-escalation)
inferenceos chat --mode hybrid --system1-model laya/agent --system2-model models/qwen3-4b.gguf --tau 0.85

Interactive TUI Controls:

  • Ctrl+O or /model: Open split-view interactive model picker modal.
  • Ctrl+T or /mode: Cycle cognitive mode (system2, system1_reflex, symbiosis).
  • Tab: Autocomplete slash commands and model paths.
  • Ctrl+C: Clear prompt buffer without exiting.
  • Slash commands: /model, /mode, /tau <float>, /dagger export <path>, /stats, /clear, /theme, /help, /exit.

2. run

Execute a single prompt against a model with live token output and metrics.

inferenceos run models/llama-3-8b.gguf --prompt "Explain quantum computing" -b vulkan -t 8

3. serve

Launch local OpenAI and Ollama REST API server.

inferenceos serve models/llama-3-8b.gguf --host 0.0.0.0 --port 11434

4. benchmark

Run end-to-end performance benchmark matrix across batch sizes and thread counts.

inferenceos benchmark models/llama-3-8b.gguf -g 28 -t 8

5. optimize

Automated 5-step hardware detection, placement generation, matrix benchmark, and caching.

inferenceos optimize models/llama-3-8b.gguf

6. profile

Profile execution and generate interactive flamegraph HTML reports.

inferenceos profile models/llama-3-8b.gguf

7. hardware

Inspect CPU, GPU, iGPU, VRAM, RAM, and interconnect bandwidth.

inferenceos hardware

8. doctor

Run diagnostic health checks across drivers, SDKs, and executable binaries.

inferenceos doctor

9. inspect

Inspect GGUF binary metadata, tensor shapes, and layer counts.

inferenceos inspect models/llama-3-8b.gguf

10. models

Register, list, remove, search, or tag models in the local library.

inferenceos models list
inferenceos models add --nickname llama3 --path /path/to/llama.gguf

11. config

View, edit, or interactively configure runtime settings.

inferenceos config runtime.threads 8
inferenceos config --interactive

12. plugins

List and manage loaded extension plugins.

inferenceos plugins

13. cache

View or clear placement and benchmark optimization caches.

inferenceos cache clear

14. logs

View and tail recent runtime execution logs.

inferenceos logs -n 100

15. telemetry

Launch historical metrics & telemetry dashboard UI.

inferenceos telemetry

16. monitor

Launch HTOP-style live process & hardware resource monitor.

inferenceos monitor

17. stats

Display aggregated performance summary statistics.

inferenceos stats

18. placement

Compute and visualize layer placement across CPU/dGPU/iGPU.

inferenceos placement models/llama-3-8b.gguf

19. update

Check engine binaries and C++ build updates.

inferenceos update

20. version

Display InferenceOS release version and system build information.

inferenceos version

21. reset

Reset settings, profiles, and caches back to default state.

inferenceos reset

22. decide

Execute fast non-autoregressive decision & reflex inference on System 1 models (e.g. Laya, Kev) with sub-150ms latency.

inferenceos decide --model laya/agent --input "Route query to financial compliance department?" --confidence-threshold 0.85