feat(core): fetch the skills bundle from an immutable pin

[dg-coreylweathers]

First PR of the five that replace #111. It adds one new module, deepctl_core.skill_bundle, plus its tests. Nothing calls it yet.

What it does

• Fetches from an immutable pin. The default bundle is deepgram/skills commit 0fc13fad726fb78e17fb1f05ba5942f0d022990f (labeled deepgram-skills-v1.7.0). Its tarball must match sha256 5b7f9753…3980, and a mismatch raises SkillFetchError. The release tag is lightweight and could be moved, so it is kept only as a human-readable label and is never fetched.
• Validates refs. --ref and DEEPCTL_SKILLS_REF accept only [A-Za-z0-9._/-], must start with a letter or digit, and may not contain .., // or /., or end in . or /. The cap is 100 characters, and a cache dir name may be at most 120 bytes, which keeps a typical Windows cache path under MAX_PATH. A user-supplied ref has no known hash, so it is not hash-checked, unless it equals the pinned commit.
• Downloads from codeload with httpx, using a 30 s timeout and a 64 MiB cap. A 404 raises SkillRefNotFoundError, and an invalid ref raises SkillRefInvalidError.
• Extracts the tarball safely. Every member is checked before anything is written.
  • Only regular files and directories are accepted.
  • It rejects absolute paths, .., \, : (drive letters, UNC paths, alternate data streams), symlinks, hardlinks and devices.
  • It rejects Windows reserved device names (CON, nul.txt, COM1, COM¹, CONOUT$…) and path components that end in a dot or a space.
  • It caps the member count, the total size and the name length.
  • Files are opened with xb, so an existing file is never overwritten.
• Parses the manifest. It reads plugins[name=="deepgram"].skills from .claude-plugin/marketplace.json. Each entry must fully match (?:\./)?skills/<name>. The name must match ^[A-Za-z0-9][A-Za-z0-9._-]*$ and must not be a Windows reserved name, end in a dot, or duplicate another name by case. All of this is checked before any Path is built.
• Publishes the cache atomically to ~/.deepctl/skills/repo_cache/.
  • It extracts into a mkdtemp sibling, validates, then swaps with os.replace.
  • If the swap fails or is interrupted (Ctrl-C included), the previous cache goes back in place.
  • If a concurrent dg process publishes a valid cache for the same ref during the final rename, that copy is used instead of failing. Other overlaps fail cleanly and keep the old copy.
  • A symlink, or a non-empty directory without deepctl's marker, is refused and never moved or deleted. If the target changes between the ownership check and the move, the fetch is refused: a directory is put back, and anything else (a file or a symlink) is kept in staging and named in the error, so it is never overwritten. POSIX rename can replace an empty directory that appears in the rename window, but no file data is at risk.

What it does not do yet

It has no caller. It doesn't change skill_generator.py or any command, and it doesn't touch the README. Folder install and the dg skills commands come in goal 2. Concurrent fetches of the same ref don't lose data, but they aren't serialized. Goal 5's lock covers skills.json writes, not this cache. Returned paths are valid until the next fetch of the same ref replaces the cache, so goal 2's installer must treat a source folder that vanishes mid-copy as a retryable fetch error.

Greg's findings this closes (from the 2026-10-05 review on #111)

• S3: the pin is now a commit SHA plus a tarball sha256, not a movable tag. Tests: TestPinnedDefault::test_tampered_archive_is_refused_and_leaves_nothing and ::test_tampered_archive_keeps_the_previous_cache.
• B4: skill names are validated as single plain segments before any path is built. The TestManifest tests are parametrized over .., a/b, a\b, /abs, C:\x, C:x, ., con, NUL, com1 and more.

Ownership proof for every filesystem delete or move

• The one rmtree: it removes only the staging dir this call created with tempfile.mkdtemp inside the cache root. Before deleting, it re-checks that the moved-aside old copy carries the marker. Staging is kept whenever it holds the only copy of the old cache, and the error names it (a bare Ctrl-C has no error to name it in).
• The previous cache: it is moved into staging only if it is a real directory (not a symlink) carrying this module's marker.

How to review

1. Read skill_bundle.py (399 lines). Its sections, in order:

   1. constants and caps
   2. errors and RepoSkill
   3. ref validation (validate_ref, resolve_skills_ref)
   4. fetch_skill_bundle
   5. manifest parsing
   6. download
   7. tar member checks and extraction
   8. _publish

2. Check the pin yourself:

   • git ls-remote https://github.com/deepgram/skills refs/tags/deepgram-skills-v1.7.0
   • curl -sSL https://codeload.github.com/deepgram/skills/tar.gz/0fc13fad726fb78e17fb1f05ba5942f0d022990f | shasum -a 256

   Repeated downloads returned the same bytes.

3. Skim the tests. They are grouped as TestPinnedDefault (S3), TestRefs, TestDownload, TestTarSafety, TestManifest (B4) and TestPublish, and none of them use the network.

Verification

• Budget: 399 source lines against a limit of 400, measured with git diff --numstat excluding tests.
• Docker gate: ruff format and check, mypy strict, and the full pytest suite (1682 passed) all pass. The core tests also pass as uid 1000 and on Python 3.10.
• Upgrade check: not applicable. Nothing calls this module yet, so install behavior doesn't change.

Stacked series: 1/5. Tracking PR: #111.

🤖 Generated with Claude Code

[GregHolmes]

Review: deepgram/cli #123, feat(core): fetch the skills bundle from an immutable pin

Classification: code
Verdict: approve

Intent

Add an unused deepctl_core.skill_bundle module that downloads the Deepgram skills bundle from a full commit SHA, verifies the default archive hash, safely extracts it, validates manifest skill names, and atomically publishes a cache. It must not change any command or existing install behavior.

Blocking

None.

Should-fix

None.

Nits

None.

Verified (evidence)

• PASS: The default source is the full 0fc13fad726fb78e17fb1f05ba5942f0d022990f commit and the checked-in SHA-256 matches the codeload archive. git ls-remote resolves deepgram-skills-v1.7.0 to that commit; the downloaded archive hashes to 5b7f975378110372c8ae3a3c712b72ba2fa43b06f2d1d93497abf87262a23980.
• PASS: packages/deepctl-core/src/deepctl_core/skill_bundle.py rejects unsafe archive members, non-regular types, unsafe Windows components, unsafe refs, and manifest names before constructing destination paths.
• PASS: The cache only serves the pinned ref after structural manifest validation; tampered downloaded archives are rejected before extraction or cache publication.
• PASS: Docker verification with the repository lockfile passed: Ruff format/check, strict mypy for skill_bundle.py, and packages/deepctl-core/tests/unit/test_skill_bundle.py (140 passed).

Needs human

None.

Developer-facing messaging

The internal module is deliberately not wired into any command yet, so this PR introduces no new developer-facing behavior or documentation claim.