docs: describe the VS Code extension 3.0.0 as API and DB mocks - #919
Merged
Merged
Conversation
The docs still sold the Keploy VS Code extension as a one-click unit test generator: running-keploy/utg-vscode-extension promised a "Generate Unit Tests" button, and integrations-vscode said the extension generates, views and manages AI unit tests. Extension 3.0.0 is a different product. It records the calls a test command makes to APIs and databases and replays them with those dependencies switched off, through `keploy mock`. It installs the Keploy CLI, works without an account, can set up AI coding agents, and writes CI jobs. It no longer generates unit tests, and no docs page described what it does. Rewrite the extension page from the extension's own behaviour: what it is for, requirements (VS Code 1.104+, Apple Silicon only on macOS, Linux kernel 5.10+ and glibc 2.34+ for the released CLI, Windows x64), the first-run CLI install, record and replay per platform, optional sign-in and what it adds, agent skill consent and where files land, CI setup, settings, the telemetry opt-out and what it does not cover yet, and the failure messages the panel shows. The page keeps its id, so the live URL keeps working; the S3 deploy has no working redirects. Point integrations-vscode at the new page, and stop the unit test generator pages (introduction, PR agent, FAQ, best practices, glossary) from sending readers to the extension for unit tests. The PR Agent is where unit test generation lives now. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The rewritten extension page describes recording and replaying API and database mocks, but the sidebar still filed it under Unit Testing > Integration. Its breadcrumb therefore read "Unit Testing > Integration > VS Code Extension" directly above the note saying the extension no longer generates unit tests. Move it next to Mock Your Tests, the keploy mock page the extension drives, so the sidebar and breadcrumb match what the page describes. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The extension page claimed the "Just this project" button lists the repository folders it writes to. It does not: the button names only the folders of agents with no home-folder location (Cursor and GitHub Copilot), while the install it runs (`keploy skill install --project`) also writes .claude/skills or .agents/skills for every other agent it found. With only Claude Code or Codex detected the button carries no folder at all. A consent prompt must not be described as disclosing more than it does. State what the question names, what each answer writes, and point at the table that lists every repository folder. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Every docs link inside the VS Code extension opens Mock Your Tests: the Keploy: Mocking Docs command, the "What to check" button when a recording captured nothing, and the fallback of every other docs button. That page never mentioned the extension, so a user sent there from a failing panel found nothing about the panel or its messages. Add one line near the top that links the extension guide and its troubleshooting table. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
llms-full.txt, which this repo builds for AI search engines, still said unit test generation is "available via VS Code extension, JetBrains plugins, CLI, and GitHub PR Agent", and listed the extension as "inline test generation and execution". Extension 3.0.0 records and replays API and database mocks and generates no tests. There is no Keploy JetBrains plugin on the JetBrains Marketplace, and neither Keploy CLI has a unit test generation command. Describe the extension as it is, link its guide, and name the PR Agent as the way to generate unit tests. content/llms-root.txt:58 carries the same claim but is left for a follow-up because it conflicts with the open #918. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Two statements on the VS Code extension page did not match the code: - "Depending on your plan" the CLI keeps mock sets in the Mock Registry. The enterprise CLI makes every signed-in, non --local `keploy mock` run registry-first with no plan check, and the api-server upload and download routes check the role and app, not the plan. What the page left out is that an upload or download failure falls back to the set on disk, and that signed-in replays are reported to the account as usage while --local runs are not. - "That run captured nothing" gave no macOS cause. On macOS the panel names npm, npx, yarn or pnpm when one of them started the tests, because macOS strips Keploy's instrumentation across that launcher. The panel prefills `npm test`, so this is the first failure a Mac Node user sees. Say what it means and what to run instead. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The VS Code extension page said an ignored recording stays on your machine, and that committing keploy/ lets teammates and CI replay it. For a signed-in user neither is true. The extension drops --local only when you are signed in, and then the enterprise CLI uploads the whole mocks.yaml, credentials included, to the Mock Registry whatever .gitignore says. After that upload, or after a signed-in download, it appends /*/mocks.yaml to keploy/.gitignore. That line hides every set git does not already track, so the CI job the extension writes, which checks for keploy/<set>/mocks.yaml and replays with --local, fails for lack of a recording. Scope the local-only statements to runs without an account. Say what a signed-in run uploads and adds to keploy/.gitignore, and that a tracked recording stays tracked. Give git add -f as the way to get such a set into CI, and mention the keploy.yml that the first run writes. Reproduced with a unit test against enterprise main's mockRegistry.upload: with keploy/ in the root .gitignore, the upload carried the credential line and keploy/.gitignore gained /*/mocks.yaml. A scratch repository then showed that git add keploy/ skipped the new set and kept staging the tracked one. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Several statements on the page describe extension behaviour that changes before 3.0.0 ships: - which folders the agent question and its "Just this project" label name, and that the fallback rules writer writes without a question - that the telemetry opt-out does not reach the CLI - the list of headlines a failed replay can show Keep only what holds before and after those changes. The page now says where each agent answer writes, which files the fallback writes, and how to turn off the CLI's own telemetry (disableTele in keploy.yml or KEPLOY_DISABLETELE). The CLI's sign-in events ignore both. The advice for runs that Keploy itself stopped moves to the "Keploy stopped that run." notification, which stays. The empty-recording row also names the macOS launcher trap. A runner started through npm, npx, yarn or pnpm, or through its own env-node shebang, loses Keploy's instrumentation, and node node_modules/.bin/jest keeps it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The page said the CLI is a single file of a few hundred MB. The released 3.8.44 builds are 160-191 MB: Linux x86-64 190.7 MB, Linux arm64 177.5 MB, macOS arm64 159.9 MB, and Windows x64 168.8 MB, going by the content-length of each download. Say about 200 MB. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
FAQ answers 10 and 11 offered a "CLI Tool" way to generate unit tests and said it needs an API key. Neither the open-source nor the enterprise Keploy CLI has a unit test generation command (the enterprise test-gen command generates API tests), and llms-full-root.txt on this branch already says unit tests come from the PR Agent. Point both answers at the PR Agent. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The VS Code page told readers that KEPLOY_DISABLETELE=true turns off the Keploy CLI's usage reporting. On native Linux that is only half true. Unless the CLI runs as root, it starts its agent with sudo, and sudo's env_reset drops the variable, so the agent still reports usage. That is the default setup for the extension on Linux. disableTele: true in keploy.yml does reach that agent, because sudo keeps the working directory and the agent reads the same keploy.yml. For a Docker command the agent runs in its own container. It gets a fixed environment and no keploy.yml, so neither setting reaches it. Say which setting covers what, and keep the note that sign-in events follow neither. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What has changed?
The docs still describe the Keploy VS Code extension as a one-click unit test generator. Extension 3.0.0, listed as "Keploy: API & DB Mocks", is a different product:
keploy mock.running-keploy/utg-vsc-extension.md(live at /docs/running-keploy/utg-vscode-extension/) is rewritten for 3.0.0. It covers:The page
idis unchanged, so the existing URL keeps working. The S3 deploy has no redirects.Other pages:
keploy-explained/integrations-vscode.md: describes the extension and links the guide.running-keploy/mock-your-tests.md: points readers at the VS Code guide.content/llms-full-root.txt: stops telling AI crawlers the extension generates tests.unit-test-generator,utg-pr-agent,unit-testing-faq,utg-best-practicesandglossary/unit-test-automationno longer claim the extension generates unit tests. The made-upkeploy.utg.*settings, an old-UI screenshot and the CLI method are removed, and those pages point to the PR Agent.Overlap with #918. Three files are also changed by #918: the sidebar,
mock-your-tests.mdandcontent/llms-full-root.txt. The edits here are small pointers to the new page. Whichever PR merges second needs a rebase, and I will rebase this one.Merge on the day extension 3.0.0 is published. Before merging, re-check these three against the final extension and CLI:
This PR resolves: no linked issue (part of the VS Code extension 3.0.0 release).
Type of change
How Has This Been Tested?
npm install && npm run buildon Node 20 succeeds, andnpm run verify-schemais OK.npx prettier@2.8.8 --checkis clean. Every commit subject isdocs: …, and commitlint is clean.Checklist: