From 176a79c15e671a116f60e486d32677251fba3e0e Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 19:54:16 +0530 Subject: [PATCH 01/11] docs: describe the VS Code extension 3.0.0 as API and DB mocks The docs still sold the Keploy VS Code extension as a one-click unit test generator: running-keploy/utg-vscode-extension promised a "Generate Unit Tests" button, and integrations-vscode said the extension generates, views and manages AI unit tests. Extension 3.0.0 is a different product. It records the calls a test command makes to APIs and databases and replays them with those dependencies switched off, through `keploy mock`. It installs the Keploy CLI, works without an account, can set up AI coding agents, and writes CI jobs. It no longer generates unit tests, and no docs page described what it does. Rewrite the extension page from the extension's own behaviour: what it is for, requirements (VS Code 1.104+, Apple Silicon only on macOS, Linux kernel 5.10+ and glibc 2.34+ for the released CLI, Windows x64), the first-run CLI install, record and replay per platform, optional sign-in and what it adds, agent skill consent and where files land, CI setup, settings, the telemetry opt-out and what it does not cover yet, and the failure messages the panel shows. The page keeps its id, so the live URL keeps working; the S3 deploy has no working redirects. Point integrations-vscode at the new page, and stop the unit test generator pages (introduction, PR agent, FAQ, best practices, glossary) from sending readers to the extension for unit tests. The PR Agent is where unit test generation lives now. Signed-off-by: slayerjain --- .../glossary/unit-test-automation.md | 5 +- .../keploy-explained/integrations-vscode.md | 34 +-- .../keploy-explained/unit-testing-faq.md | 3 +- .../keploy-explained/utg-best-practices.md | 45 +-- .../running-keploy/unit-test-generator.md | 9 +- .../running-keploy/utg-pr-agent.md | 1 - .../running-keploy/utg-vsc-extension.md | 274 ++++++++++++------ 7 files changed, 210 insertions(+), 161 deletions(-) diff --git a/versioned_docs/version-4.0.0/concepts/reference/glossary/unit-test-automation.md b/versioned_docs/version-4.0.0/concepts/reference/glossary/unit-test-automation.md index 78674519b..bd9e1c932 100644 --- a/versioned_docs/version-4.0.0/concepts/reference/glossary/unit-test-automation.md +++ b/versioned_docs/version-4.0.0/concepts/reference/glossary/unit-test-automation.md @@ -100,16 +100,15 @@ There are many unit testing tools available for a wide range of languages and en ### Keploy -Keploy offers a one-click unit test generation tool that streamlines the testing process for developers. Instead of spending time writing test cases from scratch, you can generate them instantly, allowing you to focus on coding and delivering new features. +Keploy's PR Agent generates unit tests for the code you change in a pull request. Instead of spending time writing test cases from scratch, you get them on the pull request itself, allowing you to focus on coding and delivering new features. **Benefits of Using Keploy:** - **Efficiency**: Quickly create unit tests to save time and effort. - **Enhanced Coverage**: Automatically generate tests for various scenarios, improving code reliability. - **Focus on Quality**: Spend more time writing high-quality code while ensuring thorough validation. - Keploy -Get started by downloading the tool from the [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=Keploy.keployio) and boost your development workflow today! +Get started with the [PR Agent](/docs/running-keploy/utg-pr-agent/) and boost your development workflow today! ### pytest (Python) diff --git a/versioned_docs/version-4.0.0/keploy-explained/integrations-vscode.md b/versioned_docs/version-4.0.0/keploy-explained/integrations-vscode.md index ae5cba93e..e2d79b4c3 100644 --- a/versioned_docs/version-4.0.0/keploy-explained/integrations-vscode.md +++ b/versioned_docs/version-4.0.0/keploy-explained/integrations-vscode.md @@ -1,35 +1,35 @@ --- id: integrations-vscode -title: IDE Integrations β€” VS Code -sidebar_label: Integrations & Connected Tools -description: Learn about IDE integrations for Keploy’s AI-powered unit testing. +title: IDE integrations β€” VS Code +sidebar_label: VS Code +description: Keploy's VS Code extension records the API and database calls your tests make and replays them with those dependencies switched off, from inside the editor. tags: - integrations - vscode - - unit testing + - mocks +keywords: + - keploy vs code extension + - vscode mocks + - api mocks + - database mocks --- -# πŸ”Œ IDE Integrations β€” VS Code +# IDE integrations β€” VS Code -Keploy aims to make AI-powered unit testing as seamless as possibleβ€”right inside your favorite tools. +Keploy's VS Code extension, **Keploy: API & DB Mocks**, brings the [`keploy mock`](/docs/running-keploy/mock-your-tests/) loop into the editor. It records the real calls your tests make to APIs and databases, then replays your tests with those dependencies switched off. It installs the Keploy CLI for you, can set up your AI coding agent to record and replay, and can add a CI job that replays the recording on every pull request. -## βœ… Currently Supported +Install it from the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=keploy.keployio), then follow the [VS Code extension guide](/docs/running-keploy/utg-vscode-extension/). -**Visual Studio Code (VS Code):** -We offer smooth integration with VS Code, so you can generate, view, and manage AI-generated unit tests without leaving your IDE. +The extension no longer generates unit tests. To generate unit tests, use Keploy's [PR Agent](/docs/running-keploy/utg-pr-agent/). -## πŸ’‘ Want Support for Another IDE? +## Request another editor -Need Keploy in JetBrains, Neovim, or another editor? - -[Reach out to our team!](mailto:support@keploy.io) - -Your feedback shapes our roadmap! - -More IDE integrations are coming soonβ€”stay tuned. +If you need Keploy in JetBrains, Neovim, or another editor, [tell our team](mailto:support@keploy.io). ## Related +- [Keploy VS Code extension](/docs/running-keploy/utg-vscode-extension/) β€” install, record, replay, and troubleshoot. +- [Mock your tests](/docs/running-keploy/mock-your-tests/) β€” the `keploy mock` commands the extension runs. - [SCM Integrations β€” PR Agent](/docs/keploy-explained/integrations-pr-agent/) β€” the pull-request integration. - [Keploy Troubleshooting Guide](/docs/keploy-explained/common-errors/) β€” fix common setup issues. - [What is Keploy?](/docs/keploy-explained/introduction/) β€” project overview. diff --git a/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md b/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md index 103dc4400..f286ad347 100644 --- a/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md +++ b/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md @@ -72,12 +72,11 @@ Yes! Keploy is built to handle large, complex projects. Processing time may vary ### 10. Which method should I use to generate tests? - **PR Agent**: Best for automated test generation on GitHub pull requests. -- **VS Code Extension**: Ideal for developers who prefer working in their IDE. - **CLI Tool**: For those who want more control or integrate into scripts. ### 11. Do I need an API key? -Only for the CLI method. Other tools like PR Agent and VS Code Extension use their own secure auth flows. +Only for the CLI method. The PR Agent uses its own secure auth flow. ### 12. How does Keploy ensure the quality of generated tests? diff --git a/versioned_docs/version-4.0.0/keploy-explained/utg-best-practices.md b/versioned_docs/version-4.0.0/keploy-explained/utg-best-practices.md index b41b67c7e..c70c42636 100644 --- a/versioned_docs/version-4.0.0/keploy-explained/utg-best-practices.md +++ b/versioned_docs/version-4.0.0/keploy-explained/utg-best-practices.md @@ -14,7 +14,7 @@ tags: > **Master the art of automated testing with proven strategies, expert insights, and advanced techniques for maximizing your testing ROI.** -This comprehensive guide outlines battle-tested best practices for writing exceptional unit tests, leveraging automation effectively, and extracting maximum value from Keploy's Unit Test Generation ecosystem β€” including the PR Agent and VS Code extension. +This comprehensive guide outlines battle-tested best practices for writing exceptional unit tests, leveraging automation effectively, and extracting maximum value from Keploy's Unit Test Generation ecosystem β€” including the PR Agent. ## **Foundational Unit Testing Principles** @@ -210,49 +210,6 @@ Combine automated generation with manual expertise: - **Domain Expert Review**: Have domain experts validate test logic - **Continuous Refinement**: Iteratively improve generated test quality -## **VS Code Extension Power User Techniques** - -### **1. Real-Time Development Integration** ⚑ - -Maximize productivity by generating tests during active development: - -- **Function-Level Testing**: Generate tests immediately after writing functions -- **Refactoring Safety**: Create tests before refactoring existing code -- **Bug Reproduction**: Generate tests to reproduce and fix reported bugs -- **API Exploration**: Use generated tests to understand third-party APIs - -### **2. Advanced Configuration Management** - -Customize the VS Code extension for optimal team workflows: - -```json -{ - "keploy.utg.outputDirectory": "./tests", - "keploy.utg.testFramework": "jest", - "keploy.utg.mockingStrategy": "auto", - "keploy.utg.coverageTarget": "branches", - "keploy.utg.namingConvention": "descriptive" -} -``` - -### **3. Incremental Test Development** - -Build comprehensive test suites incrementally: - -- **Start Small**: Begin with core utility functions -- **Expand Gradually**: Add tests for more complex business logic -- **Maintain Quality**: Regularly review and refactor generated tests -- **Document Patterns**: Create team guidelines for test generation - -### **4. Integration with Development Workflow** - -Seamlessly blend test generation with existing development practices: - -- **Code Review Integration**: Generate tests before requesting code reviews -- **Pair Programming**: Use generated tests as conversation starters -- **Knowledge Sharing**: Use tests to document expected behavior -- **Onboarding Tool**: Help new team members understand codebase behavior - ## **Advanced Metrics & Monitoring** ### **Test Quality Indicators** diff --git a/versioned_docs/version-4.0.0/running-keploy/unit-test-generator.md b/versioned_docs/version-4.0.0/running-keploy/unit-test-generator.md index 97558d591..72973e605 100644 --- a/versioned_docs/version-4.0.0/running-keploy/unit-test-generator.md +++ b/versioned_docs/version-4.0.0/running-keploy/unit-test-generator.md @@ -21,7 +21,7 @@ keywords: Keploy's Unit Test Generator (UTG) simplifies testing by generating high-quality, maintainable unit tests with minimal manual effort. Inspired by Meta’s LLM research, it uses code semantics and Large Language Models (LLMs) to produce meaningful test cases. -To fit into modern development workflows, Keploy now supports multiple methods for unit test generation - ranging from automated pull request comments to a one-click VS Code extension +To fit into modern development workflows, Keploy generates unit tests on your pull requests. Keploy's UTG aims to: @@ -29,14 +29,11 @@ Keploy's UTG aims to: - **Improve edge case coverage:** Catch complex scenarios often missed manually. - **Boost test coverage:** Make comprehensive coverage achievable as your codebase grows. -### Try the Unit Test Generator in two ways: +### Try the Unit Test Generator -1. **PR Agent**: Install a GitHub App on your repo to automatically generate tests when you create a pull request. - -2. **VS Code Extension**: Generate test files with a single click directly from your IDE. +Install the [PR Agent](/docs/running-keploy/utg-pr-agent/), a GitHub App, on your repository to automatically generate tests when you create a pull request. ## Related - [Unit Test Architecture](/docs/running-keploy/unit-test-architecture/) β€” how the generator is structured. - [Keploy's PR Agent](/docs/running-keploy/utg-pr-agent/) β€” generate unit tests on pull requests. -- [Keploy's VS Code Extension](/docs/running-keploy/utg-vscode-extension/) β€” generate unit tests in your editor. diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-pr-agent.md b/versioned_docs/version-4.0.0/running-keploy/utg-pr-agent.md index 906c2740c..271884266 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-pr-agent.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-pr-agent.md @@ -93,5 +93,4 @@ _Elevate your development workflow. Make every pull request a quality checkpoint ## Related - [Keploy Unit Test Generator](/docs/running-keploy/unit-test-generator/) β€” the engine behind the PR agent. -- [Keploy's VS Code Extension](/docs/running-keploy/utg-vscode-extension/) β€” the same generator in your IDE. - [Unit Test Architecture](/docs/running-keploy/unit-test-architecture/) β€” how generated unit tests are built. diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index bbdcf2a4e..3211753d2 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -1,133 +1,231 @@ --- id: utg-vscode-extension -title: Keploy's VS Code Extension +title: Keploy VS Code extension sidebar_label: VS Code Extension -description: This section documents usecase of Keploy's AI powered unit test vs code extension +description: Record the API and database calls your tests make from VS Code, then replay them with those dependencies switched off. Install, first run, sign-in, AI agent setup, CI, settings, and troubleshooting. tags: - - utg - - unit test generator - - unit test generator pull request agent - - unit test generator pr agent - - generate unit test - - unit test + - vscode + - vs code extension + - mocks + - mocking + - record and replay + - ai agents keywords: - - unit test generator - - unit testing - - unit tests - - documentation - - testcases - - AI testing - - Gemini - - OpenAI + - keploy vs code extension + - vscode mocks + - api mocks + - database mocks + - dependency mocking + - record and replay + - ai coding agents + - keploy.keployio --- -> **Transform your IDE into a testing powerhouse. Generate comprehensive, intelligent unit tests with a single click - trusted by 500K+ developers worldwide.** +The **Keploy VS Code extension** (listed as **Keploy: API & DB Mocks**, ID `keploy.keployio`) records the real calls your tests make to their dependencies, such as HTTP and gRPC APIs, databases, caches, and queues, and saves them next to your code. It then replays your tests with those dependencies switched off, answering every call from the recording. -The **Keploy Unit Test Generator VS Code Extension** revolutionizes how developers approach testing by bringing seamless, AI-powered test generation directly into your favorite IDE. Whether you're building new features, debugging complex issues, or refactoring legacy code, generate production-ready unit tests instantly without leaving your development environment. +The extension runs the Keploy CLI's [`keploy mock`](/docs/running-keploy/mock-your-tests/) commands for you from a panel in VS Code. Keploy works at the network layer, so you don't need an SDK or any change to your test code. Which languages and protocols it supports depends on where your tests run; see [Choose where your tests run](#choose-where-your-tests-run). -## **Why 500K+ Developers Choose Keploy** +:::note The extension no longer generates unit tests -### **Instant Productivity Boost** +Version 3.0.0 of the extension replaces the earlier unit test generator. To generate unit tests, use Keploy's [PR Agent](/docs/running-keploy/utg-pr-agent/), which writes them on your GitHub pull requests. -- **One-Click Generation**: Transform any function into a comprehensive test suite in seconds -- **Zero Context Switching**: Stay in your IDE flow without opening external tools -- **Intelligent Analysis**: AI understands your code's business logic and generates relevant test scenarios +::: -### **Enterprise-Grade Quality** +## What you can do with it -- **LLM-Powered Intelligence**: Leverages GPT-4o and advanced language models for superior test quality -- **Framework Agnostic**: Works seamlessly with Jest, Mocha, JUnit, pytest, and more -- **Production Ready**: Generated tests follow industry best practices and your project's conventions +- **Record** the dependency calls your test command makes while the real dependencies are running. +- **Replay** the same command with the dependencies switched off. Every call is answered from the recording, and a call that was never recorded fails the run. +- **Hand it to your coding agent.** The extension can install Keploy's skill and rules for Claude Code, Cursor, GitHub Copilot, Codex, and Antigravity, so your agent records and replays instead of inventing stubs. +- **Check it in CI.** One click adds a GitHub Actions job that replays the recording on every pull request. -## **Generate Tests in Simple Steps** +## Requirements -### 1. Install the Extension +| Requirement | Details | +| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Editor | VS Code 1.104 or later. | +| macOS | Apple Silicon only. The Keploy CLI's macOS build doesn't run on Intel Macs, so the extension doesn't install it there. | +| Linux | x86-64 or arm64, with Linux kernel 5.10 or later **and** glibc 2.34 or later, such as Ubuntu 22.04, Debian 12, or RHEL 9 and later. The released CLI doesn't start on older glibc (Ubuntu 20.04, Debian 11, RHEL 8, Amazon Linux 2) or on musl-based distributions such as Alpine. | +| Windows | x64. Windows on Arm isn't supported. | +| Disk | A few hundred MB in your home folder for the CLI. | +| Docker (optional) | Only if your tests run in containers. | -You can install the Keploy extension using either of the following methods: +## Install the extension -**Method 1: From VS Code Marketplace** +1. In VS Code, open the **Extensions** view, search for **Keploy**, and select **Install** on **Keploy: API & DB Mocks**. You can also install it from the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=keploy.keployio), or from a terminal: -- Open the **Extensions** tab in VS Code. -- Search for **"Keploy"**. -- Click **Install**. + ```bash + code --install-extension keploy.keployio + ``` - -Searching for the Keploy extension in the VS Code Extensions Marketplace +2. Select the **Keploy** icon in the Activity Bar to open the panel. -**Method 2: Direct Link / VS Code Marketplace** +For a guided tour, run **Welcome: Open Walkthrough** from the Command Palette and choose **Get started with Keploy**. -- Visit [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=Keploy.keployio). -- Click **Install** and follow the prompts to open it in VS Code. +## Install the Keploy CLI on first run - -Keploy extension page on the Visual Studio Code Marketplace +The extension runs your tests through the Keploy CLI, which it downloads and manages for you: -### 2. Generate Unit Tests in a Single Click +- **New install.** The download starts on its own the first time VS Code starts with the extension. A notification shows its progress; select **Cancel** to stop it. Nothing is installed, and the panel offers **Install Keploy** for when you're ready. +- **Update from an earlier version of the extension.** If you don't have the Keploy CLI yet, the extension asks before it downloads it. +- **Size and location.** The CLI is a single file of a few hundred MB. It goes to `~/.keploy/bin/keploy` on macOS and Linux, and to `%APPDATA%\.keploy\bin\keploy.exe` on Windows. The extension checks the download against the SHA-256 checksum Keploy publishes and discards it if they don't match. Installing the CLI changes nothing else on your machine. +- **Updates.** When the CLI in that folder is older than the minimum version the extension needs, the extension replaces it automatically. It never touches a `keploy` binary you installed somewhere else, such as `/usr/local/bin`. +- **A fixed version.** To hold one build, set `keploy.cli.version` to an exact release number. Leave it empty to follow the current release. -Once the extension is installed: +## Record and replay your tests -1. **Look for the Keploy icon** in the **Activity Bar** (left-hand sidebar) of VS Code. -2. Click the **Keploy icon** to open the extension UI. -3. You’ll see a simple interface with a button labeled **β€œGenerate Unit Tests.”** -4. Click the button - that’s it! - - Keploy extension UI in VS Code with the Generate Unit Tests button +### Record with the dependencies running -Keploy will analyze your codebase and automatically generate all relevant unit test files, covering: +1. Start the real dependencies your tests use: a database, a cache, a local service, or a staging API. +2. In the Keploy panel, enter the command that runs your tests, such as `npm test`, `pytest`, or `go test ./...`. The panel suggests one when it recognizes your project. +3. Select **Start recording**. -- Core logic -- Edge cases -- Boundary conditions -- And more - all with meaningful assertions and clean test structure. +Keploy runs your command and saves every outgoing call it captures into a mock set, in `keploy//mocks.yaml` in your workspace. While the run is in progress, the panel counts the captured calls; select **Stop** to end it early. -## What Happens Next? +### Replay with the dependencies switched off -After clicking **Generate Unit Tests**, Keploy will: +1. Stop the dependencies. +2. Select **Replay offline**. -- Parse and understand your source code using code semantics. -- Create relevant test files alongside your existing code (in your test directory or alongside source files, depending on your config). -- Provide complete test coverage with little to no boilerplate code. +Keploy runs the same command and answers every call from the recording. The panel replays with `--on-miss fail`, so a call that was never recorded fails the run instead of reaching a real service. When the tests pass, the panel shows **Dependencies off. Tests still passed.** - -Unit test files generated by Keploy alongside the source code in VS Code +Commit the `keploy/` folder with your code so that teammates and CI replay the same recording. -All tests are: +### Start a run from elsewhere -- Readable -- Maintainable -- Ready to run using your preferred test runner (e.g., Jest, Mocha, etc.) +- **Command Palette:** run **Keploy: Record Mocks** or **Keploy: Replay Mocks**, and confirm the test command to run. +- **Test files:** at the top of recognized test files, such as `*.test.ts`, `*_test.go`, `test_*.py`, or `*Test.java`, the extension adds **Record with Keploy** and **Replay with Keploy mocks**. Each asks for the test command to run, filled in with your project's command, and uses a mock set named after the file. To hide them, set `keploy.codeLens.enabled` to `false`. -## **Smart Output Management** +### What the panel runs -### **Intelligent Directory Structure** +The panel runs the CLI in a VS Code terminal with the same commands you would run yourself: -Generated tests are organized following best practices: - -``` -your-project/ -β”œβ”€β”€ src/ -β”‚ └── utils/ -β”‚ └── calculator.js -└── tests/ - └── utils/ - └── calculator.test.js +```bash +keploy mock record -c "" +keploy mock replay -c "" --on-miss fail ``` -## **Get Started Today** +It adds `--name ` for a named set, `--container-name` for a container command, and `--local` when you aren't signed in. See [Mock your tests](/docs/running-keploy/mock-your-tests/) for the other flags and for per-test scoping. + +### Choose where your tests run + +| Where your tests run | What to enter | Notes | +| --------------------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Directly on Linux | Your test command | Keploy uses eBPF, which needs root. Unless VS Code runs as root, the CLI asks for your `sudo` password in the Keploy terminal. To run the whole command with `sudo -E` instead, set `keploy.cli.elevation` to `sudo`. | +| Directly on macOS (Apple Silicon) | Your test command | No `sudo` needed. | +| Directly on Windows (x64) | Your test command | No Administrator rights needed. | +| In a Docker container | A `docker` or `docker compose` command, such as `docker compose run --rm --name my-app-tests tests` | The panel also asks for the name of the container your tests run in. | + +Directly on macOS and Windows, Keploy supports tests written in Go, Node.js, Python, and Java, and understands their HTTP and HTTPS, MySQL, and MongoDB calls. It captures other protocols, such as PostgreSQL, Redis, Kafka, and gRPC, only as raw bytes, and those usually don't replay. For other languages or those dependencies, run your tests in a Docker container. The [macOS](/docs/installation/macos-installation/) and [Windows](/docs/installation/windows-installation/) guides have the details. + +## Sign in (optional) + +You don't need an account to record and replay on your machine. Without one, the panel runs with `--local`, and your recordings stay on your disk. The Command Palette and the test-file actions ask whether to sign in or to run without an account. + +When you sign in, runs go through your Keploy account instead. Depending on your plan, the CLI then also keeps each mock set in Keploy's [Mock Registry](/docs/keploy-cloud/mock-registry/): it uploads the set after a successful recording, and downloads it before a replay. Signed-in runs check your account with Keploy's servers first, so they need a network connection. To record and replay with no network at all, sign out. + +- **Sign in:** select **Sign in** in the panel, or run **Keploy: Sign In**. The extension runs `keploy login`, which opens your browser. +- **One session:** the extension uses the CLI's own session, so a `keploy login` in any terminal signs the panel in too. If `KEPLOY_API_KEY` is set, or `~/.keploy/cred.yaml` holds an API key, the CLI uses that key first. +- **Sign out:** run **Keploy: Sign Out**. + +## Set up your AI coding agent + +The extension can teach your coding agent to record and replay your tests, so the agent works against recorded calls instead of stubs it guesses. + +After your first recording that captures calls, the extension asks once whether to install the Keploy skill for the coding agents it finds. The question says where the files go, and the **Just this project** button lists the repository folders it writes to. You can also start this at any time with **Keploy: Add Mocking Rules for AI Agents**. Nothing is written before you answer, and the skill acts only in repositories that use Keploy. + +| Your answer | Where the files go | +| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **All my projects** | Your agents' home folders: `~/.claude/skills` for Claude Code, `~/.agents/skills` for Codex, and `~/.gemini/config/skills` for Antigravity. | +| **Just this project** | This repository: `.claude/skills`, `.cursor/rules`, `.agents/skills`, or `.github/instructions`, for the agents it found. Offered only when a folder is open. | +| **Add it** | This repository. Offered instead of the two preceding answers when only Cursor or GitHub Copilot is found. | +| **No thanks**, or closing the question | Nowhere. The extension doesn't ask again on its own. | + +Cursor and GitHub Copilot have no home-folder location, so only a repository install, **Just this project** or **Add it**, sets them up. + +If the extension finds no coding agent, **Keploy: Add Mocking Rules for AI Agents** writes rules into the repository instead: `.claude/skills/keploy-mocking/SKILL.md`, `.cursor/rules/keploy-mocking.mdc`, `CLAUDE.md`, `AGENTS.md`, and `.github/copilot-instructions.md`. In files that can hold your own content, it edits only the block between `` and ``, and leaves the rest alone. A notification lists every file it wrote. + +To keep the skill out of home folders on a machine, set `keploy.agentSkill.install` to `false`. The extension then never asks, and the command writes into the current repository only. + +In VS Code's chat, the extension also provides tools that an agent can call, or that you can reference with `#keployStatus`, `#keployMocks`, `#keployRecord`, `#keployReplay`, and `#keployVerify`. The record, replay, and verify tools ask you to confirm before they run your test command. + +## Check your recordings in CI + +After a replay passes, the panel offers to add a CI job that replays the recording on every pull request: + +- **GitHub Actions:** **Check this on every pull request** writes `.github/workflows/keploy-offline-tests.yml`, or `keploy-offline-.yml` for a named set. The job installs the same CLI version, fails if the recording is missing, and runs `keploy mock replay --local` with `--on-miss fail`. The replay needs no database and no API keys. In a repository with no CI yet, the button reads **Add a GitHub Actions replay job**. +- **GitLab, Woodpecker, CircleCI, and others:** **Copy the replay steps for** your CI copies the steps to your clipboard, to paste into the job that runs your tests. The extension doesn't edit those pipeline files. + +To keep recordings from drifting away from the real services, run **Keploy: Set Up Mock Auto-Refresh (CI)**. It writes `.github/workflows/keploy-refresh-mocks.yml`, a GitHub Actions workflow that runs every Monday at 06:00 UTC and on demand. It re-records against your real dependencies, fails if nothing was recorded, and commits the refreshed set. Before you rely on it, replace its **Bring up real dependencies** step, which is a placeholder. This workflow is for GitHub Actions only; in a repository that uses another CI, the extension doesn't write it. + +## Keep credentials out of your repository + +A recording holds real traffic, so it can hold real credentials. The panel checks the start of each recording for common credential patterns, such as `Authorization` headers, cookies, bearer tokens, JWTs, and API key headers, and warns you when it finds one. The check doesn't read a whole large recording, and it doesn't recognize database passwords, so a clean result isn't a guarantee: review a recording before you commit it. The extension never rewrites recorded traffic. + +When the panel finds credentials and `keploy/` isn't ignored, it offers **Add keploy/ to .gitignore**. If `keploy/` is already committed, it offers **Stop tracking keploy/** instead, and if it can't tell, **Keep keploy/ out of git**. An ignored recording stays on your machine, so teammates and CI can't replay it, and the CI job fails for lack of a recording. **Stop tracking keploy/** doesn't remove credentials that were already pushed, so rotate them. + +Once `keploy/` is ignored, the panel offers **Record again with fresh credentials**: record against test credentials to get a recording that's safe to commit. + +## Settings + +| Setting | Default | What it does | +| --------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `keploy.cli.version` | empty | Pins the CLI to an exact version. Empty follows the current release, never below the minimum version the extension needs. | +| `keploy.cli.elevation` | `auto` | Linux only. `auto` and `none` both run the command as is, and the CLI asks for `sudo` itself when it needs it. `sudo` runs the whole command with `sudo -E`. | +| `keploy.agentSkill.install` | `true` | Lets the extension offer the Keploy skill for your agents' home folders. `false` keeps it out of them, and agent rules go into the repository only. | +| `keploy.codeLens.enabled` | `true` | Shows the record and replay actions at the top of test files. | +| `keploy.telemetry.enabled` | `true` | Sends the extension's usage events. See [Telemetry](#telemetry). | + +## Telemetry + +The extension sends usage events to Keploy: which features you use, and whether recordings and replays succeed or fail. The events never include source code, file contents, paths, test commands, or request and response data. They carry VS Code's anonymous machine ID, and your Keploy account only when you're signed in. + +To turn them off, set `keploy.telemetry.enabled` to `false`, or set VS Code's `telemetry.telemetryLevel` to `off`. Either one stops the extension's events. Neither setting reaches the Keploy CLI yet, so the runs that the extension starts still send the CLI's own telemetry. + +## Troubleshoot + +When a run fails, the panel names the problem and offers the action that fixes it. After a failed replay, the panel shows **The offline run did not pass.** or **The replay did not finish.**, unless Keploy refused your sign-in or API key, and the notification that appears when the run ends gives the specific reason. The messages you're most likely to see: + +| Message | What it means | What to do | +| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **The install did not finish.** | The CLI download failed. The panel names the cause, such as a full disk, no write access to the install folder, a proxy asking for credentials (HTTP 407), or a refused download (HTTP 403). | Fix the cause, then select **Try the install again**. Downloads start at `keploy.io` and are served from the CDN it redirects to, so a proxy or firewall must allow both. | +| **Keploy's macOS build is Apple Silicon only.** | You're on an Intel Mac, where the extension can't run Keploy. | To use Keploy on this Mac, run the CLI inside a Lima VM. See the [macOS guide](/docs/installation/macos-installation/). | +| **The pinned Keploy version was never published.** or **This Keploy version has no build for this Mac.** | `keploy.cli.version` names a release that doesn't exist, or one older than the Apple Silicon build. | Clear the setting, or pin a published release. | +| **That run captured nothing.** | The recording finished without capturing a single call. | Check that your dependencies were running and reachable from the test command. For a container command, check that the tests ran in the container you named. | +| **That recording was cut short.** | The run stopped before your test command finished, so the recording is incomplete. | Record again before you rely on it. | +| **Keploy needs permissions to attach.** | On Linux, the CLI couldn't get the root privileges that eBPF needs. | Enter your `sudo` password when the CLI asks, set `keploy.cli.elevation` to `sudo`, or run your tests in a container. | +| **Keploy stopped that run.** | Keploy itself failed before your tests ran, for example a signed-in run that couldn't reach Keploy's servers. The line under it gives Keploy's reason. | Follow that line. With no network, sign out, then record or replay without an account. | +| **That run did not pass.** | The test command failed. Keploy passes your command's exit code through, so this is usually your tests. | Run the command on its own. If it passes there, the failure came from running it under Keploy, so report it. | +| **That command was not found.** | The shell couldn't run your test command. | Check the command, and that its tool is on your `PATH`. | +| **That run was stopped.** | The run was ended from outside the panel before it finished. | Run it again. | +| **That run ran out of memory.** | The operating system killed the test command, usually for lack of memory. | Free some memory, or run fewer tests at once. | +| **That run did not finish.** | Keploy or the test process stopped early. The line under it says why. | Follow that line. If it asks for a container, run your tests in one and enter the container's name. | +| **There is nothing to replay.** | The mock set has no recorded calls. | Record first, or replay a set that has calls in it. | +| **The offline run did not pass.** | The replay failed: your code changed since the recording, or it made a call that was never recorded. If the notification says **Keploy stopped that run.**, the replay never reached your tests, for example because a signed-in replay couldn't reach Keploy's servers. If the panel says Keploy couldn't attach to your tests, it's a permissions problem, not the recording: see **Keploy needs permissions to attach.** | Open the recording to see what's in it. Then fix the test, select **Record only the missing calls** with the dependencies running, or record again. If Keploy stopped the run, don't record again: fix what the notification names, such as signing out when you have no network, and replay. | +| **The replay did not finish.** | The replay was stopped before it finished, so the recording wasn't judged. | Run the replay again. | +| **Keploy needs a sign-in.** or **Your session expired.** | A run tried to use your account without a valid session. | Sign in again, or run without an account. | +| **Keploy would not use your API key.** | Keploy rejected the key in `KEPLOY_API_KEY` or `~/.keploy/cred.yaml`. Signing in doesn't replace that key. | Replace the key, or run without an account. | -Don't let testing slow down your development velocity. With Keploy's AI-powered VS Code extension, comprehensive test coverage is just one click away. +In a workspace you haven't trusted, the panel opens, but recording, replaying, and sign-in stay off until you trust the folder. -### **Quick Start:** +On Linux, if the CLI doesn't run at all, check your glibc version with `ldd --version`. The released CLI needs glibc 2.34 or later; see [Requirements](#requirements). -1. **[Install from VS Code Marketplace β†’](https://marketplace.visualstudio.com/items?itemName=Keploy.keployio)** -2. **Open your favorite project** and select a function -3. **Click "Generate Unit Tests"** and experience the magic -4. **Watch your test coverage soar** while maintaining development speed +## Commands -\_Transform your IDE. Elevate your code quality. Join 500k+ developers building better software with Keploy. +| Command | What it does | +| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | +| **Keploy: Open Mocks Dashboard** | Opens the Keploy panel. | +| **Keploy: Record Mocks** | Records the dependency calls of your test command. | +| **Keploy: Replay Mocks** | Replays your test command against a recording. | +| **Keploy: Add Mocking Rules for AI Agents** | Installs the Keploy skill or rules for your coding agents. | +| **Keploy: Set Up Mock Auto-Refresh (CI)** | Writes the GitHub Actions workflow that re-records mocks. | +| **Keploy: Sign In** and **Keploy: Sign Out** | Signs the CLI in to your Keploy account, or out of it. | +| **Keploy: Show CLI Version** | Shows the version of the installed CLI. | +| **Keploy: Mocking Docs** | Opens [Mock your tests](/docs/running-keploy/mock-your-tests/). | +| **Keploy: Report an Issue (GitHub)** and **Keploy: Request a Feature (GitHub)** | Opens a prefilled issue in the [keploy/keploy](https://github.com/keploy/keploy) repository. | ## Related -- [Keploy's PR Agent](/docs/running-keploy/utg-pr-agent/) β€” the same generator on pull requests. -- [Keploy Unit Test Generator](/docs/running-keploy/unit-test-generator/) β€” the generator internals. -- [Unit Test Architecture](/docs/running-keploy/unit-test-architecture/) β€” how generated unit tests are built. +- [Mock your tests](/docs/running-keploy/mock-your-tests/) β€” the `keploy mock` commands the extension runs. +- [Mock Registry](/docs/keploy-cloud/mock-registry/) β€” where signed-in runs keep mock sets. +- [Keploy on macOS](/docs/installation/macos-installation/), [Linux](/docs/installation/linux-installation/), and [Windows](/docs/installation/windows-installation/) β€” what each platform supports. +- [Keploy's PR Agent](/docs/running-keploy/utg-pr-agent/) β€” Keploy's unit test generator, on your pull requests. From 4ad3fd6b9c2e763d594459decf551e5cf3faba24 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 20:14:37 +0530 Subject: [PATCH 02/11] docs: list the VS Code extension page under Integration Testing The rewritten extension page describes recording and replaying API and database mocks, but the sidebar still filed it under Unit Testing > Integration. Its breadcrumb therefore read "Unit Testing > Integration > VS Code Extension" directly above the note saying the extension no longer generates unit tests. Move it next to Mock Your Tests, the keploy mock page the extension drives, so the sidebar and breadcrumb match what the page describes. Signed-off-by: slayerjain --- versioned_sidebars/version-4.0.0-sidebars.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_sidebars/version-4.0.0-sidebars.json b/versioned_sidebars/version-4.0.0-sidebars.json index c33c2e3b4..15bff7a8c 100644 --- a/versioned_sidebars/version-4.0.0-sidebars.json +++ b/versioned_sidebars/version-4.0.0-sidebars.json @@ -51,6 +51,7 @@ "running-keploy/mock-quickstart", "running-keploy/mock-your-tests", + "running-keploy/utg-vscode-extension", "running-keploy/keploy-templatize", "running-keploy/risk-profile-analysis", "keploy-cloud/time-freezing", @@ -248,7 +249,6 @@ "label": "Integration", "items": [ "running-keploy/utg-pr-agent", - "running-keploy/utg-vscode-extension", "keploy-explained/integrations-pr-agent" ] }, From 950a6d8a55ad6a5ac15ebac8bb8b2597fc4a650c Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 20:14:57 +0530 Subject: [PATCH 03/11] docs: say which folders the agent skill question actually names The extension page claimed the "Just this project" button lists the repository folders it writes to. It does not: the button names only the folders of agents with no home-folder location (Cursor and GitHub Copilot), while the install it runs (`keploy skill install --project`) also writes .claude/skills or .agents/skills for every other agent it found. With only Claude Code or Codex detected the button carries no folder at all. A consent prompt must not be described as disclosing more than it does. State what the question names, what each answer writes, and point at the table that lists every repository folder. Signed-off-by: slayerjain --- .../version-4.0.0/running-keploy/utg-vsc-extension.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index 3211753d2..dfe4080ad 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -131,7 +131,7 @@ When you sign in, runs go through your Keploy account instead. Depending on your The extension can teach your coding agent to record and replay your tests, so the agent works against recorded calls instead of stubs it guesses. -After your first recording that captures calls, the extension asks once whether to install the Keploy skill for the coding agents it finds. The question says where the files go, and the **Just this project** button lists the repository folders it writes to. You can also start this at any time with **Keploy: Add Mocking Rules for AI Agents**. Nothing is written before you answer, and the skill acts only in repositories that use Keploy. +After your first recording that captures calls, the extension asks once whether to install the Keploy skill for the coding agents it finds. The question names the home folders that **All my projects** writes to. **Just this project** writes into this repository for every agent the extension found, in the folders listed in the following table, but its label names only the Cursor and GitHub Copilot folders. When the extension finds only Cursor or GitHub Copilot, the question names the repository folders instead. You can also start this at any time with **Keploy: Add Mocking Rules for AI Agents**. Nothing is written before you answer, and the skill acts only in repositories that use Keploy. | Your answer | Where the files go | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | From e185b36d218999ad3526c3cfeb385d15a2500b47 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 20:15:21 +0530 Subject: [PATCH 04/11] docs: point Mock Your Tests readers at the VS Code extension guide Every docs link inside the VS Code extension opens Mock Your Tests: the Keploy: Mocking Docs command, the "What to check" button when a recording captured nothing, and the fallback of every other docs button. That page never mentioned the extension, so a user sent there from a failing panel found nothing about the panel or its messages. Add one line near the top that links the extension guide and its troubleshooting table. Signed-off-by: slayerjain --- .../version-4.0.0/running-keploy/mock-your-tests.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/versioned_docs/version-4.0.0/running-keploy/mock-your-tests.md b/versioned_docs/version-4.0.0/running-keploy/mock-your-tests.md index 1a87037c8..913d1ddfc 100644 --- a/versioned_docs/version-4.0.0/running-keploy/mock-your-tests.md +++ b/versioned_docs/version-4.0.0/running-keploy/mock-your-tests.md @@ -31,6 +31,11 @@ so your test code needs **no SDK and no changes**. Keploy propagates your test runner's **exit code**, so it drops straight into CI. +Using VS Code? The [Keploy VS Code extension](/docs/running-keploy/utg-vscode-extension/) +runs these commands for you from a panel, and its +[troubleshooting table](/docs/running-keploy/utg-vscode-extension/#troubleshoot) +explains the messages the panel shows when a run fails. + ## Quick start ```bash From 79a93404861fdb4b7423c2c4828959009b8b0eb7 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 20:15:40 +0530 Subject: [PATCH 05/11] docs: stop telling AI crawlers the VS Code extension generates tests llms-full.txt, which this repo builds for AI search engines, still said unit test generation is "available via VS Code extension, JetBrains plugins, CLI, and GitHub PR Agent", and listed the extension as "inline test generation and execution". Extension 3.0.0 records and replays API and database mocks and generates no tests. There is no Keploy JetBrains plugin on the JetBrains Marketplace, and neither Keploy CLI has a unit test generation command. Describe the extension as it is, link its guide, and name the PR Agent as the way to generate unit tests. content/llms-root.txt:58 carries the same claim but is left for a follow-up because it conflicts with the open #918. Signed-off-by: slayerjain --- content/llms-full-root.txt | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/content/llms-full-root.txt b/content/llms-full-root.txt index 499461f98..48a2aa43a 100644 --- a/content/llms-full-root.txt +++ b/content/llms-full-root.txt @@ -38,7 +38,7 @@ Keploy offers three core products: Record-replay engine using eBPF to capture API calls and generate test suites with deterministic replay. The open-source core captures HTTP/HTTPS, gRPC, WebSocket, and GraphQL traffic at the kernel level and automatically creates replayable test cases with mocked dependencies (databases, external APIs, message queues). Tests are stored as YAML files in the project directory. ### 2. Test Agent (AI Unit Test Generation) -AI-powered unit test generation from PR diffs and source code. The Test Agent analyzes code semantics using LLMs (Gemini 2.5 Pro, GPT-4) and generates unit tests that are validated by actually running them against the codebase. Only tests that build, pass, and increase coverage are kept. Available via VS Code extension, JetBrains plugins, CLI, and GitHub PR Agent. Currently supports Go with Java and JavaScript coming soon. +AI-powered unit test generation from PR diffs and source code. The Test Agent analyzes code semantics using LLMs (Gemini 2.5 Pro, GPT-4) and generates unit tests that are validated by actually running them against the codebase. Only tests that build, pass, and increase coverage are kept. Available through the GitHub PR Agent. Currently supports Go with Java and JavaScript coming soon. ### 3. API Test Generation Generate API tests from OpenAPI specs, Postman collections, cURL commands, or real user traffic. Supports functional tests, edge case tests, performance tests, security tests, and dependency tests. AI models route tasks to the best-suited model and validate results to ensure high-quality output. @@ -465,9 +465,8 @@ For current, publicly substantiated security, privacy, and compliance informatio ## Integrations ### IDE Integrations -- VS Code Extension: inline test generation and execution (see Marketplace for current installs) -- JetBrains Plugin: IntelliJ IDEA, PyCharm, WebStorm, GoLand support -- URL: https://keploy.io/docs/keploy-explained/integrations-vscode/ +- VS Code Extension (Keploy: API & DB Mocks): records the API and database calls your tests make and replays them offline with `keploy mock`, from a panel in VS Code. It also sets up AI coding agents and adds a CI job that replays the recording. It does not generate unit tests. See the Marketplace for current installs. +- URL: https://keploy.io/docs/running-keploy/utg-vscode-extension/ ### PR Agent Integration Automated test suggestions on GitHub pull requests. Analyzes PR diffs and generates targeted tests for changed code paths. From 1c9451d0e1fcf03fcad93e6e60ebeb40b1140a0f Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 20:17:00 +0530 Subject: [PATCH 06/11] docs: correct the signed-in registry note and the empty-recording fix Two statements on the VS Code extension page did not match the code: - "Depending on your plan" the CLI keeps mock sets in the Mock Registry. The enterprise CLI makes every signed-in, non --local `keploy mock` run registry-first with no plan check, and the api-server upload and download routes check the role and app, not the plan. What the page left out is that an upload or download failure falls back to the set on disk, and that signed-in replays are reported to the account as usage while --local runs are not. - "That run captured nothing" gave no macOS cause. On macOS the panel names npm, npx, yarn or pnpm when one of them started the tests, because macOS strips Keploy's instrumentation across that launcher. The panel prefills `npm test`, so this is the first failure a Mac Node user sees. Say what it means and what to run instead. Signed-off-by: slayerjain --- .../running-keploy/utg-vsc-extension.md | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index dfe4080ad..687b96889 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -121,7 +121,7 @@ Directly on macOS and Windows, Keploy supports tests written in Go, Node.js, Pyt You don't need an account to record and replay on your machine. Without one, the panel runs with `--local`, and your recordings stay on your disk. The Command Palette and the test-file actions ask whether to sign in or to run without an account. -When you sign in, runs go through your Keploy account instead. Depending on your plan, the CLI then also keeps each mock set in Keploy's [Mock Registry](/docs/keploy-cloud/mock-registry/): it uploads the set after a successful recording, and downloads it before a replay. Signed-in runs check your account with Keploy's servers first, so they need a network connection. To record and replay with no network at all, sign out. +When you sign in, runs go through your Keploy account instead, and the CLI also keeps each mock set in Keploy's [Mock Registry](/docs/keploy-cloud/mock-registry/): it uploads the set after a successful recording, and downloads it before a replay. If an upload or download fails, the run carries on with the set on your disk. Signed-in replays are reported to your Keploy account as usage; runs without an account aren't. Signed-in runs check your account with Keploy's servers first, so they need a network connection. To record and replay with no network at all, sign out. - **Sign in:** select **Sign in** in the panel, or run **Keploy: Sign In**. The extension runs `keploy login`, which opens your browser. - **One session:** the extension uses the CLI's own session, so a `keploy login` in any terminal signs the panel in too. If `KEPLOY_API_KEY` is set, or `~/.keploy/cred.yaml` holds an API key, the CLI uses that key first. @@ -185,25 +185,25 @@ To turn them off, set `keploy.telemetry.enabled` to `false`, or set VS Code's `t When a run fails, the panel names the problem and offers the action that fixes it. After a failed replay, the panel shows **The offline run did not pass.** or **The replay did not finish.**, unless Keploy refused your sign-in or API key, and the notification that appears when the run ends gives the specific reason. The messages you're most likely to see: -| Message | What it means | What to do | -| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **The install did not finish.** | The CLI download failed. The panel names the cause, such as a full disk, no write access to the install folder, a proxy asking for credentials (HTTP 407), or a refused download (HTTP 403). | Fix the cause, then select **Try the install again**. Downloads start at `keploy.io` and are served from the CDN it redirects to, so a proxy or firewall must allow both. | -| **Keploy's macOS build is Apple Silicon only.** | You're on an Intel Mac, where the extension can't run Keploy. | To use Keploy on this Mac, run the CLI inside a Lima VM. See the [macOS guide](/docs/installation/macos-installation/). | -| **The pinned Keploy version was never published.** or **This Keploy version has no build for this Mac.** | `keploy.cli.version` names a release that doesn't exist, or one older than the Apple Silicon build. | Clear the setting, or pin a published release. | -| **That run captured nothing.** | The recording finished without capturing a single call. | Check that your dependencies were running and reachable from the test command. For a container command, check that the tests ran in the container you named. | -| **That recording was cut short.** | The run stopped before your test command finished, so the recording is incomplete. | Record again before you rely on it. | -| **Keploy needs permissions to attach.** | On Linux, the CLI couldn't get the root privileges that eBPF needs. | Enter your `sudo` password when the CLI asks, set `keploy.cli.elevation` to `sudo`, or run your tests in a container. | -| **Keploy stopped that run.** | Keploy itself failed before your tests ran, for example a signed-in run that couldn't reach Keploy's servers. The line under it gives Keploy's reason. | Follow that line. With no network, sign out, then record or replay without an account. | -| **That run did not pass.** | The test command failed. Keploy passes your command's exit code through, so this is usually your tests. | Run the command on its own. If it passes there, the failure came from running it under Keploy, so report it. | -| **That command was not found.** | The shell couldn't run your test command. | Check the command, and that its tool is on your `PATH`. | -| **That run was stopped.** | The run was ended from outside the panel before it finished. | Run it again. | -| **That run ran out of memory.** | The operating system killed the test command, usually for lack of memory. | Free some memory, or run fewer tests at once. | -| **That run did not finish.** | Keploy or the test process stopped early. The line under it says why. | Follow that line. If it asks for a container, run your tests in one and enter the container's name. | -| **There is nothing to replay.** | The mock set has no recorded calls. | Record first, or replay a set that has calls in it. | -| **The offline run did not pass.** | The replay failed: your code changed since the recording, or it made a call that was never recorded. If the notification says **Keploy stopped that run.**, the replay never reached your tests, for example because a signed-in replay couldn't reach Keploy's servers. If the panel says Keploy couldn't attach to your tests, it's a permissions problem, not the recording: see **Keploy needs permissions to attach.** | Open the recording to see what's in it. Then fix the test, select **Record only the missing calls** with the dependencies running, or record again. If Keploy stopped the run, don't record again: fix what the notification names, such as signing out when you have no network, and replay. | -| **The replay did not finish.** | The replay was stopped before it finished, so the recording wasn't judged. | Run the replay again. | -| **Keploy needs a sign-in.** or **Your session expired.** | A run tried to use your account without a valid session. | Sign in again, or run without an account. | -| **Keploy would not use your API key.** | Keploy rejected the key in `KEPLOY_API_KEY` or `~/.keploy/cred.yaml`. Signing in doesn't replace that key. | Replace the key, or run without an account. | +| Message | What it means | What to do | +| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **The install did not finish.** | The CLI download failed. The panel names the cause, such as a full disk, no write access to the install folder, a proxy asking for credentials (HTTP 407), or a refused download (HTTP 403). | Fix the cause, then select **Try the install again**. Downloads start at `keploy.io` and are served from the CDN it redirects to, so a proxy or firewall must allow both. | +| **Keploy's macOS build is Apple Silicon only.** | You're on an Intel Mac, where the extension can't run Keploy. | To use Keploy on this Mac, run the CLI inside a Lima VM. See the [macOS guide](/docs/installation/macos-installation/). | +| **The pinned Keploy version was never published.** or **This Keploy version has no build for this Mac.** | `keploy.cli.version` names a release that doesn't exist, or one older than the Apple Silicon build. | Clear the setting, or pin a published release. | +| **That run captured nothing.** | The recording finished without capturing a single call. | Check that your dependencies were running and reachable from the test command. For a container command, check that the tests ran in the container you named. On macOS, if the panel names `npm`, `npx`, `yarn`, or `pnpm`, macOS removed Keploy's instrumentation when that launcher started your tests: start them with `node` itself, or run them in Docker. See the [macOS guide](/docs/installation/macos-installation/#option-1-run-keploy-natively). | +| **That recording was cut short.** | The run stopped before your test command finished, so the recording is incomplete. | Record again before you rely on it. | +| **Keploy needs permissions to attach.** | On Linux, the CLI couldn't get the root privileges that eBPF needs. | Enter your `sudo` password when the CLI asks, set `keploy.cli.elevation` to `sudo`, or run your tests in a container. | +| **Keploy stopped that run.** | Keploy itself failed before your tests ran, for example a signed-in run that couldn't reach Keploy's servers. The line under it gives Keploy's reason. | Follow that line. With no network, sign out, then record or replay without an account. | +| **That run did not pass.** | The test command failed. Keploy passes your command's exit code through, so this is usually your tests. | Run the command on its own. If it passes there, the failure came from running it under Keploy, so report it. | +| **That command was not found.** | The shell couldn't run your test command. | Check the command, and that its tool is on your `PATH`. | +| **That run was stopped.** | The run was ended from outside the panel before it finished. | Run it again. | +| **That run ran out of memory.** | The operating system killed the test command, usually for lack of memory. | Free some memory, or run fewer tests at once. | +| **That run did not finish.** | Keploy or the test process stopped early. The line under it says why. | Follow that line. If it asks for a container, run your tests in one and enter the container's name. | +| **There is nothing to replay.** | The mock set has no recorded calls. | Record first, or replay a set that has calls in it. | +| **The offline run did not pass.** | The replay failed: your code changed since the recording, or it made a call that was never recorded. If the notification says **Keploy stopped that run.**, the replay never reached your tests, for example because a signed-in replay couldn't reach Keploy's servers. If the panel says Keploy couldn't attach to your tests, it's a permissions problem, not the recording: see **Keploy needs permissions to attach.** | Open the recording to see what's in it. Then fix the test, select **Record only the missing calls** with the dependencies running, or record again. If Keploy stopped the run, don't record again: fix what the notification names, such as signing out when you have no network, and replay. | +| **The replay did not finish.** | The replay was stopped before it finished, so the recording wasn't judged. | Run the replay again. | +| **Keploy needs a sign-in.** or **Your session expired.** | A run tried to use your account without a valid session. | Sign in again, or run without an account. | +| **Keploy would not use your API key.** | Keploy rejected the key in `KEPLOY_API_KEY` or `~/.keploy/cred.yaml`. Signing in doesn't replace that key. | Replace the key, or run without an account. | In a workspace you haven't trusted, the panel opens, but recording, replaying, and sign-in stay off until you trust the folder. From f2d730ec361eb097a9dd31ee354163bf6413ca4d Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 21:28:08 +0530 Subject: [PATCH 07/11] docs: say what a signed-in recording does to git and CI The VS Code extension page said an ignored recording stays on your machine, and that committing keploy/ lets teammates and CI replay it. For a signed-in user neither is true. The extension drops --local only when you are signed in, and then the enterprise CLI uploads the whole mocks.yaml, credentials included, to the Mock Registry whatever .gitignore says. After that upload, or after a signed-in download, it appends /*/mocks.yaml to keploy/.gitignore. That line hides every set git does not already track, so the CI job the extension writes, which checks for keploy//mocks.yaml and replays with --local, fails for lack of a recording. Scope the local-only statements to runs without an account. Say what a signed-in run uploads and adds to keploy/.gitignore, and that a tracked recording stays tracked. Give git add -f as the way to get such a set into CI, and mention the keploy.yml that the first run writes. Reproduced with a unit test against enterprise main's mockRegistry.upload: with keploy/ in the root .gitignore, the upload carried the credential line and keploy/.gitignore gained /*/mocks.yaml. A scratch repository then showed that git add keploy/ skipped the new set and kept staging the tracked one. Signed-off-by: slayerjain --- .../running-keploy/utg-vsc-extension.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index 687b96889..e3e2c2ae8 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -79,7 +79,7 @@ The extension runs your tests through the Keploy CLI, which it downloads and man 2. In the Keploy panel, enter the command that runs your tests, such as `npm test`, `pytest`, or `go test ./...`. The panel suggests one when it recognizes your project. 3. Select **Start recording**. -Keploy runs your command and saves every outgoing call it captures into a mock set, in `keploy//mocks.yaml` in your workspace. While the run is in progress, the panel counts the captured calls; select **Stop** to end it early. +Keploy runs your command and saves every outgoing call it captures into a mock set, in `keploy//mocks.yaml` in your workspace. While the run is in progress, the panel counts the captured calls; select **Stop** to end it early. The first run also writes `keploy.yml`, the CLI's settings for this repository, at the root of your workspace. ### Replay with the dependencies switched off @@ -88,7 +88,7 @@ Keploy runs your command and saves every outgoing call it captures into a mock s Keploy runs the same command and answers every call from the recording. The panel replays with `--on-miss fail`, so a call that was never recorded fails the run instead of reaching a real service. When the tests pass, the panel shows **Dependencies off. Tests still passed.** -Commit the `keploy/` folder with your code so that teammates and CI replay the same recording. +Commit the `keploy/` folder and `keploy.yml` with your code so that teammates and CI replay the same recording. Signing in changes what git picks up, as [Sign in](#sign-in-optional) explains. ### Start a run from elsewhere @@ -121,7 +121,9 @@ Directly on macOS and Windows, Keploy supports tests written in Go, Node.js, Pyt You don't need an account to record and replay on your machine. Without one, the panel runs with `--local`, and your recordings stay on your disk. The Command Palette and the test-file actions ask whether to sign in or to run without an account. -When you sign in, runs go through your Keploy account instead, and the CLI also keeps each mock set in Keploy's [Mock Registry](/docs/keploy-cloud/mock-registry/): it uploads the set after a successful recording, and downloads it before a replay. If an upload or download fails, the run carries on with the set on your disk. Signed-in replays are reported to your Keploy account as usage; runs without an account aren't. Signed-in runs check your account with Keploy's servers first, so they need a network connection. To record and replay with no network at all, sign out. +When you sign in, runs go through your Keploy account instead, and the CLI also keeps each mock set in Keploy's [Mock Registry](/docs/keploy-cloud/mock-registry/): it uploads the whole set, including any credentials it holds, when a recording finishes, and downloads it before a replay. If an upload or download fails, the run carries on with the set on your disk. Signed-in replays are reported to your Keploy account as usage; runs without an account aren't. Signed-in runs check your account with Keploy's servers first, so they need a network connection. To record and replay with no network at all, sign out. + +When a signed-in recording uploads a set, or a signed-in replay downloads one, the CLI adds `/*/mocks.yaml` to `keploy/.gitignore`. That line matches the recording in every set, so git leaves out any recording that it doesn't already track. A recording that git already tracks stays tracked, so a signed-in recording's changes to it, credentials included, go into your next commit. Each set's `config.yaml` records which copy in the registry a signed-in replay downloads. For what this means in CI, see [Check your recordings in CI](#check-your-recordings-in-ci). - **Sign in:** select **Sign in** in the panel, or run **Keploy: Sign In**. The extension runs `keploy login`, which opens your browser. - **One session:** the extension uses the CLI's own session, so a `keploy login` in any terminal signs the panel in too. If `KEPLOY_API_KEY` is set, or `~/.keploy/cred.yaml` holds an API key, the CLI uses that key first. @@ -155,13 +157,17 @@ After a replay passes, the panel offers to add a CI job that replays the recordi - **GitHub Actions:** **Check this on every pull request** writes `.github/workflows/keploy-offline-tests.yml`, or `keploy-offline-.yml` for a named set. The job installs the same CLI version, fails if the recording is missing, and runs `keploy mock replay --local` with `--on-miss fail`. The replay needs no database and no API keys. In a repository with no CI yet, the button reads **Add a GitHub Actions replay job**. - **GitLab, Woodpecker, CircleCI, and others:** **Copy the replay steps for** your CI copies the steps to your clipboard, to paste into the job that runs your tests. The extension doesn't edit those pipeline files. -To keep recordings from drifting away from the real services, run **Keploy: Set Up Mock Auto-Refresh (CI)**. It writes `.github/workflows/keploy-refresh-mocks.yml`, a GitHub Actions workflow that runs every Monday at 06:00 UTC and on demand. It re-records against your real dependencies, fails if nothing was recorded, and commits the refreshed set. Before you rely on it, replace its **Bring up real dependencies** step, which is a placeholder. This workflow is for GitHub Actions only; in a repository that uses another CI, the extension doesn't write it. +The job replays the recording committed to your repository, without an account. If `keploy/.gitignore` holds the `/*/mocks.yaml` line that signed-in runs add, git skips any recording it doesn't already track, and the job fails for lack of a recording. To commit that recording anyway, run `git add -f keploy//mocks.yaml`. Once git tracks the file, the line no longer applies to it. + +To keep recordings from drifting away from the real services, run **Keploy: Set Up Mock Auto-Refresh (CI)**. It writes `.github/workflows/keploy-refresh-mocks.yml`, a GitHub Actions workflow that runs every Monday at 06:00 UTC and on demand. It re-records against your real dependencies, fails if nothing was recorded, and commits the refreshed set. If git doesn't track the set and `keploy/.gitignore` lists it, the workflow has nothing to commit. Before you rely on it, replace its **Bring up real dependencies** step, which is a placeholder. This workflow is for GitHub Actions only; in a repository that uses another CI, the extension doesn't write it. ## Keep credentials out of your repository A recording holds real traffic, so it can hold real credentials. The panel checks the start of each recording for common credential patterns, such as `Authorization` headers, cookies, bearer tokens, JWTs, and API key headers, and warns you when it finds one. The check doesn't read a whole large recording, and it doesn't recognize database passwords, so a clean result isn't a guarantee: review a recording before you commit it. The extension never rewrites recorded traffic. -When the panel finds credentials and `keploy/` isn't ignored, it offers **Add keploy/ to .gitignore**. If `keploy/` is already committed, it offers **Stop tracking keploy/** instead, and if it can't tell, **Keep keploy/ out of git**. An ignored recording stays on your machine, so teammates and CI can't replay it, and the CI job fails for lack of a recording. **Stop tracking keploy/** doesn't remove credentials that were already pushed, so rotate them. +When the panel finds credentials and `keploy/` isn't ignored, it offers **Add keploy/ to .gitignore**. If `keploy/` is already committed, it offers **Stop tracking keploy/** instead, and if it can't tell, **Keep keploy/ out of git**. Without an account, an ignored recording stays on your machine, so teammates and CI can't replay it, and the CI job fails for lack of a recording. **Stop tracking keploy/** doesn't remove credentials that were already pushed, so rotate them. + +When you're signed in, the CLI has already uploaded the recording, credentials included, to your account's Mock Registry, and ignoring `keploy/` doesn't change that. To keep credentials off Keploy's servers, record without an account, or record against test credentials. Once `keploy/` is ignored, the panel offers **Record again with fresh credentials**: record against test credentials to get a recording that's safe to commit. From 5270bcf317e79f45e1cce758639a8af7f176e434 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 21:28:09 +0530 Subject: [PATCH 08/11] docs: keep the VS Code page true across pending extension fixes Several statements on the page describe extension behaviour that changes before 3.0.0 ships: - which folders the agent question and its "Just this project" label name, and that the fallback rules writer writes without a question - that the telemetry opt-out does not reach the CLI - the list of headlines a failed replay can show Keep only what holds before and after those changes. The page now says where each agent answer writes, which files the fallback writes, and how to turn off the CLI's own telemetry (disableTele in keploy.yml or KEPLOY_DISABLETELE). The CLI's sign-in events ignore both. The advice for runs that Keploy itself stopped moves to the "Keploy stopped that run." notification, which stays. The empty-recording row also names the macOS launcher trap. A runner started through npm, npx, yarn or pnpm, or through its own env-node shebang, loses Keploy's instrumentation, and node node_modules/.bin/jest keeps it. Signed-off-by: slayerjain --- .../running-keploy/utg-vsc-extension.md | 54 ++++++++++--------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index e3e2c2ae8..8be9d7349 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -133,7 +133,9 @@ When a signed-in recording uploads a set, or a signed-in replay downloads one, t The extension can teach your coding agent to record and replay your tests, so the agent works against recorded calls instead of stubs it guesses. -After your first recording that captures calls, the extension asks once whether to install the Keploy skill for the coding agents it finds. The question names the home folders that **All my projects** writes to. **Just this project** writes into this repository for every agent the extension found, in the folders listed in the following table, but its label names only the Cursor and GitHub Copilot folders. When the extension finds only Cursor or GitHub Copilot, the question names the repository folders instead. You can also start this at any time with **Keploy: Add Mocking Rules for AI Agents**. Nothing is written before you answer, and the skill acts only in repositories that use Keploy. +After your first recording that captures calls, the extension asks once whether to set up the coding agents it finds. You can also start this at any time with **Keploy: Add Mocking Rules for AI Agents**. When the extension asks, it writes nothing before you answer. The skill acts only in repositories that use Keploy. + +When the Keploy CLI can install its skill, your answer decides where the files go: | Your answer | Where the files go | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -144,9 +146,9 @@ After your first recording that captures calls, the extension asks once whether Cursor and GitHub Copilot have no home-folder location, so only a repository install, **Just this project** or **Add it**, sets them up. -If the extension finds no coding agent, **Keploy: Add Mocking Rules for AI Agents** writes rules into the repository instead: `.claude/skills/keploy-mocking/SKILL.md`, `.cursor/rules/keploy-mocking.mdc`, `CLAUDE.md`, `AGENTS.md`, and `.github/copilot-instructions.md`. In files that can hold your own content, it edits only the block between `` and ``, and leaves the rest alone. A notification lists every file it wrote. +When the Keploy CLI has no skill to install, or the extension finds no coding agent, **Keploy: Add Mocking Rules for AI Agents** writes the extension's own rules into the repository instead: `.claude/skills/keploy-mocking/SKILL.md`, `.cursor/rules/keploy-mocking.mdc`, `CLAUDE.md`, `AGENTS.md`, and `.github/copilot-instructions.md`. In files that can hold your own content, it edits only the block between `` and ``, and leaves the rest alone. A notification lists every file it wrote. -To keep the skill out of home folders on a machine, set `keploy.agentSkill.install` to `false`. The extension then never asks, and the command writes into the current repository only. +To keep the skill out of home folders on a machine, set `keploy.agentSkill.install` to `false`. The extension then doesn't offer this on its own, and the command writes into the current repository only. In VS Code's chat, the extension also provides tools that an agent can call, or that you can reference with `#keployStatus`, `#keployMocks`, `#keployRecord`, `#keployReplay`, and `#keployVerify`. The record, replay, and verify tools ask you to confirm before they run your test command. @@ -185,31 +187,33 @@ Once `keploy/` is ignored, the panel offers **Record again with fresh credential The extension sends usage events to Keploy: which features you use, and whether recordings and replays succeed or fail. The events never include source code, file contents, paths, test commands, or request and response data. They carry VS Code's anonymous machine ID, and your Keploy account only when you're signed in. -To turn them off, set `keploy.telemetry.enabled` to `false`, or set VS Code's `telemetry.telemetryLevel` to `off`. Either one stops the extension's events. Neither setting reaches the Keploy CLI yet, so the runs that the extension starts still send the CLI's own telemetry. +To turn them off, set `keploy.telemetry.enabled` to `false`, or set VS Code's `telemetry.telemetryLevel` to `off`. Either one stops the extension's events. + +The Keploy CLI reports its own usage separately. To turn that off, set `disableTele: true` in your repository's `keploy.yml`, or set the environment variable `KEPLOY_DISABLETELE` to `true`. The CLI's sign-in events, which it sends when you sign in or when it renews your session, don't follow that setting. ## Troubleshoot -When a run fails, the panel names the problem and offers the action that fixes it. After a failed replay, the panel shows **The offline run did not pass.** or **The replay did not finish.**, unless Keploy refused your sign-in or API key, and the notification that appears when the run ends gives the specific reason. The messages you're most likely to see: - -| Message | What it means | What to do | -| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **The install did not finish.** | The CLI download failed. The panel names the cause, such as a full disk, no write access to the install folder, a proxy asking for credentials (HTTP 407), or a refused download (HTTP 403). | Fix the cause, then select **Try the install again**. Downloads start at `keploy.io` and are served from the CDN it redirects to, so a proxy or firewall must allow both. | -| **Keploy's macOS build is Apple Silicon only.** | You're on an Intel Mac, where the extension can't run Keploy. | To use Keploy on this Mac, run the CLI inside a Lima VM. See the [macOS guide](/docs/installation/macos-installation/). | -| **The pinned Keploy version was never published.** or **This Keploy version has no build for this Mac.** | `keploy.cli.version` names a release that doesn't exist, or one older than the Apple Silicon build. | Clear the setting, or pin a published release. | -| **That run captured nothing.** | The recording finished without capturing a single call. | Check that your dependencies were running and reachable from the test command. For a container command, check that the tests ran in the container you named. On macOS, if the panel names `npm`, `npx`, `yarn`, or `pnpm`, macOS removed Keploy's instrumentation when that launcher started your tests: start them with `node` itself, or run them in Docker. See the [macOS guide](/docs/installation/macos-installation/#option-1-run-keploy-natively). | -| **That recording was cut short.** | The run stopped before your test command finished, so the recording is incomplete. | Record again before you rely on it. | -| **Keploy needs permissions to attach.** | On Linux, the CLI couldn't get the root privileges that eBPF needs. | Enter your `sudo` password when the CLI asks, set `keploy.cli.elevation` to `sudo`, or run your tests in a container. | -| **Keploy stopped that run.** | Keploy itself failed before your tests ran, for example a signed-in run that couldn't reach Keploy's servers. The line under it gives Keploy's reason. | Follow that line. With no network, sign out, then record or replay without an account. | -| **That run did not pass.** | The test command failed. Keploy passes your command's exit code through, so this is usually your tests. | Run the command on its own. If it passes there, the failure came from running it under Keploy, so report it. | -| **That command was not found.** | The shell couldn't run your test command. | Check the command, and that its tool is on your `PATH`. | -| **That run was stopped.** | The run was ended from outside the panel before it finished. | Run it again. | -| **That run ran out of memory.** | The operating system killed the test command, usually for lack of memory. | Free some memory, or run fewer tests at once. | -| **That run did not finish.** | Keploy or the test process stopped early. The line under it says why. | Follow that line. If it asks for a container, run your tests in one and enter the container's name. | -| **There is nothing to replay.** | The mock set has no recorded calls. | Record first, or replay a set that has calls in it. | -| **The offline run did not pass.** | The replay failed: your code changed since the recording, or it made a call that was never recorded. If the notification says **Keploy stopped that run.**, the replay never reached your tests, for example because a signed-in replay couldn't reach Keploy's servers. If the panel says Keploy couldn't attach to your tests, it's a permissions problem, not the recording: see **Keploy needs permissions to attach.** | Open the recording to see what's in it. Then fix the test, select **Record only the missing calls** with the dependencies running, or record again. If Keploy stopped the run, don't record again: fix what the notification names, such as signing out when you have no network, and replay. | -| **The replay did not finish.** | The replay was stopped before it finished, so the recording wasn't judged. | Run the replay again. | -| **Keploy needs a sign-in.** or **Your session expired.** | A run tried to use your account without a valid session. | Sign in again, or run without an account. | -| **Keploy would not use your API key.** | Keploy rejected the key in `KEPLOY_API_KEY` or `~/.keploy/cred.yaml`. Signing in doesn't replace that key. | Replace the key, or run without an account. | +When a run fails, the panel names the problem and offers the action that fixes it, and the notification that appears when the run ends gives the specific reason. The messages you're most likely to see: + +| Message | What it means | What to do | +| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **The install did not finish.** | The CLI download failed. The panel names the cause, such as a full disk, no write access to the install folder, a proxy asking for credentials (HTTP 407), or a refused download (HTTP 403). | Fix the cause, then select **Try the install again**. Downloads start at `keploy.io` and are served from the CDN it redirects to, so a proxy or firewall must allow both. | +| **Keploy's macOS build is Apple Silicon only.** | You're on an Intel Mac, where the extension can't run Keploy. | To use Keploy on this Mac, run the CLI inside a Lima VM. See the [macOS guide](/docs/installation/macos-installation/). | +| **The pinned Keploy version was never published.** or **This Keploy version has no build for this Mac.** | `keploy.cli.version` names a release that doesn't exist, or one older than the Apple Silicon build. | Clear the setting, or pin a published release. | +| **That run captured nothing.** | The recording finished without capturing a single call. | Check that your dependencies were running and reachable from the test command. For a container command, check that the tests ran in the container you named. On macOS, if the panel names `npm`, `npx`, `yarn`, or `pnpm`, macOS removed Keploy's instrumentation when that launcher started your tests. Start the test runner with `node` itself, such as `node node_modules/.bin/jest`, or run your tests in Docker. Running `jest` on its own doesn't help, because it starts through the same kind of launcher. See the [macOS guide](/docs/installation/macos-installation/#option-1-run-keploy-natively). | +| **That recording was cut short.** | The run stopped before your test command finished, so the recording is incomplete. | Record again before you rely on it. | +| **Keploy needs permissions to attach.** | On Linux, the CLI couldn't get the root privileges that eBPF needs. | Enter your `sudo` password when the CLI asks, set `keploy.cli.elevation` to `sudo`, or run your tests in a container. | +| **Keploy stopped that run.** | Keploy itself failed before your tests ran, for example a signed-in run that couldn't reach Keploy's servers. The line under it gives Keploy's reason. After a replay, this means the recording wasn't tested. | Follow that line. With no network, sign out, then record or replay without an account. Don't record again because of it. | +| **That run did not pass.** | The test command failed. Keploy passes your command's exit code through, so this is usually your tests. | Run the command on its own. If it passes there, the failure came from running it under Keploy, so report it. | +| **That command was not found.** | The shell couldn't run your test command. | Check the command, and that its tool is on your `PATH`. | +| **That run was stopped.** | The run was ended from outside the panel before it finished. | Run it again. | +| **That run ran out of memory.** | The operating system killed the test command, usually for lack of memory. | Free some memory, or run fewer tests at once. | +| **That run did not finish.** | Keploy or the test process stopped early. The line under it says why. | Follow that line. If it asks for a container, run your tests in one and enter the container's name. | +| **There is nothing to replay.** | The mock set has no recorded calls. | Record first, or replay a set that has calls in it. | +| **The offline run did not pass.** | The replay failed: your code changed since the recording, or it made a call that was never recorded. If the notification says **Keploy stopped that run.**, the replay never reached your tests; see that message. If the panel says Keploy couldn't attach to your tests, it's a permissions problem, not the recording: see **Keploy needs permissions to attach.** | Open the recording to see what's in it. Then fix the test, select **Record only the missing calls** with the dependencies running, or record again. | +| **The replay did not finish.** | The replay was stopped before it finished, so the recording wasn't judged. | Run the replay again. | +| **Keploy needs a sign-in.** or **Your session expired.** | A run tried to use your account without a valid session. | Sign in again, or run without an account. | +| **Keploy would not use your API key.** | Keploy rejected the key in `KEPLOY_API_KEY` or `~/.keploy/cred.yaml`. Signing in doesn't replace that key. | Replace the key, or run without an account. | In a workspace you haven't trusted, the panel opens, but recording, replaying, and sign-in stay off until you trust the folder. From c1454d2c558dc99f082faca40816c206b0909eb5 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 21:28:09 +0530 Subject: [PATCH 09/11] docs: give the real size of the Keploy CLI download The page said the CLI is a single file of a few hundred MB. The released 3.8.44 builds are 160-191 MB: Linux x86-64 190.7 MB, Linux arm64 177.5 MB, macOS arm64 159.9 MB, and Windows x64 168.8 MB, going by the content-length of each download. Say about 200 MB. Signed-off-by: slayerjain --- .../version-4.0.0/running-keploy/utg-vsc-extension.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index 8be9d7349..eaa101fff 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -46,7 +46,7 @@ Version 3.0.0 of the extension replaces the earlier unit test generator. To gene | macOS | Apple Silicon only. The Keploy CLI's macOS build doesn't run on Intel Macs, so the extension doesn't install it there. | | Linux | x86-64 or arm64, with Linux kernel 5.10 or later **and** glibc 2.34 or later, such as Ubuntu 22.04, Debian 12, or RHEL 9 and later. The released CLI doesn't start on older glibc (Ubuntu 20.04, Debian 11, RHEL 8, Amazon Linux 2) or on musl-based distributions such as Alpine. | | Windows | x64. Windows on Arm isn't supported. | -| Disk | A few hundred MB in your home folder for the CLI. | +| Disk | About 200 MB in your home folder for the CLI. | | Docker (optional) | Only if your tests run in containers. | ## Install the extension @@ -67,7 +67,7 @@ The extension runs your tests through the Keploy CLI, which it downloads and man - **New install.** The download starts on its own the first time VS Code starts with the extension. A notification shows its progress; select **Cancel** to stop it. Nothing is installed, and the panel offers **Install Keploy** for when you're ready. - **Update from an earlier version of the extension.** If you don't have the Keploy CLI yet, the extension asks before it downloads it. -- **Size and location.** The CLI is a single file of a few hundred MB. It goes to `~/.keploy/bin/keploy` on macOS and Linux, and to `%APPDATA%\.keploy\bin\keploy.exe` on Windows. The extension checks the download against the SHA-256 checksum Keploy publishes and discards it if they don't match. Installing the CLI changes nothing else on your machine. +- **Size and location.** The CLI is a single file of about 200 MB. It goes to `~/.keploy/bin/keploy` on macOS and Linux, and to `%APPDATA%\.keploy\bin\keploy.exe` on Windows. The extension checks the download against the SHA-256 checksum Keploy publishes and discards it if they don't match. Installing the CLI changes nothing else on your machine. - **Updates.** When the CLI in that folder is older than the minimum version the extension needs, the extension replaces it automatically. It never touches a `keploy` binary you installed somewhere else, such as `/usr/local/bin`. - **A fixed version.** To hold one build, set `keploy.cli.version` to an exact release number. Leave it empty to follow the current release. From 9211bb04c20867c238b099a9e8cd5f56cd977038 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 21:28:09 +0530 Subject: [PATCH 10/11] docs: drop the CLI method from the unit test FAQ FAQ answers 10 and 11 offered a "CLI Tool" way to generate unit tests and said it needs an API key. Neither the open-source nor the enterprise Keploy CLI has a unit test generation command (the enterprise test-gen command generates API tests), and llms-full-root.txt on this branch already says unit tests come from the PR Agent. Point both answers at the PR Agent. Signed-off-by: slayerjain --- .../version-4.0.0/keploy-explained/unit-testing-faq.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md b/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md index f286ad347..a12b3a645 100644 --- a/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md +++ b/versioned_docs/version-4.0.0/keploy-explained/unit-testing-faq.md @@ -71,12 +71,11 @@ Yes! Keploy is built to handle large, complex projects. Processing time may vary ### 10. Which method should I use to generate tests? -- **PR Agent**: Best for automated test generation on GitHub pull requests. -- **CLI Tool**: For those who want more control or integrate into scripts. +Use the [PR Agent](/docs/running-keploy/utg-pr-agent/), which generates unit tests on your GitHub pull requests. ### 11. Do I need an API key? -Only for the CLI method. The PR Agent uses its own secure auth flow. +No. The PR Agent uses its own secure auth flow. ### 12. How does Keploy ensure the quality of generated tests? From bbeb7d5c89e70bd0996f34e6d312db2a5f3f5534 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Wed, 23 Sep 2026 22:05:06 +0530 Subject: [PATCH 11/11] docs: say which telemetry opt-out reaches the Keploy agent The VS Code page told readers that KEPLOY_DISABLETELE=true turns off the Keploy CLI's usage reporting. On native Linux that is only half true. Unless the CLI runs as root, it starts its agent with sudo, and sudo's env_reset drops the variable, so the agent still reports usage. That is the default setup for the extension on Linux. disableTele: true in keploy.yml does reach that agent, because sudo keeps the working directory and the agent reads the same keploy.yml. For a Docker command the agent runs in its own container. It gets a fixed environment and no keploy.yml, so neither setting reaches it. Say which setting covers what, and keep the note that sign-in events follow neither. Signed-off-by: slayerjain --- .../version-4.0.0/running-keploy/utg-vsc-extension.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md index eaa101fff..810ad2528 100644 --- a/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md +++ b/versioned_docs/version-4.0.0/running-keploy/utg-vsc-extension.md @@ -189,7 +189,13 @@ The extension sends usage events to Keploy: which features you use, and whether To turn them off, set `keploy.telemetry.enabled` to `false`, or set VS Code's `telemetry.telemetryLevel` to `off`. Either one stops the extension's events. -The Keploy CLI reports its own usage separately. To turn that off, set `disableTele: true` in your repository's `keploy.yml`, or set the environment variable `KEPLOY_DISABLETELE` to `true`. The CLI's sign-in events, which it sends when you sign in or when it renews your session, don't follow that setting. +The Keploy CLI reports its own usage separately, and so does the agent that the CLI starts to record and replay your tests. When your tests run directly on your machine, setting `disableTele: true` in your repository's `keploy.yml` turns off both. + +Setting the environment variable `KEPLOY_DISABLETELE` to `true` in the environment you start VS Code from also turns off the CLI's reporting, but on Linux it doesn't always reach the agent. Unless the CLI itself runs as root, it starts the agent with `sudo`, which drops the variable, so the agent still reports its usage. On Linux, use `keploy.yml`. + +When your tests run in a Docker container, the CLI runs the agent in a container of its own. Either setting still turns off the CLI's own reporting, but neither reaches that agent, so it still reports its usage. + +The CLI also sends events about signing in, such as when you sign in or when it renews your session. Those events don't follow either setting. ## Troubleshoot