Skip to content

fix(client): validate skills in skill_refs, not in the config parser - #146

Merged
XieX merged 1 commit into
xie/skills-experimental-entry-pointfrom
xie/skills-validate-refs-on-use
Oct 7, 2026
Merged

XieX merged 1 commit into
xie/skills-experimental-entry-pointfrom
xie/skills-validate-refs-on-use

Conversation

@XieX

@XieX XieX commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A malformed skills field no longer fails core config calls. Today parse_ai_config rejects the whole AI Config when skills is malformed. config().invoke(), stream(), graph nodes and extract_variation then raise, and inspect_config returns config: None, even for apps that never use Agent Skills. Agent Skills is experimental, and TESTING.md §0.3 says experimental behaviour must not break a core call.

  • parse_ai_config passes skills through unchecked.
  • skill_refs validates the field and raises ValueError when it is present but malformed, including skills: null and a single bad entry. Before, it dropped bad entries with a warning, which was only safe because the parser had already rejected them.
  • An absent field, or a config that is not a dict, still returns [].

Raising rather than dropping keeps the reason the parser checked in the first place. write_skills with the default prune=True deletes every managed skill missing from the list it is given, so skill_refs never returns an empty or partial list for a field that is present.

Stacked on #144. JS counterpart: launchdarkly/js-ai-sdk#117.

Tests

  • Parser: a malformed skills field of every kind (null, non-list, bad entry, one bad entry among good ones) parses successfully and passes through unchanged.
  • skill_refs: the cases the parser used to reject, including null, now raise ValueError naming the field. One bad entry rejects the whole field. The error does not echo the rejected key.
  • Core calls: inspect_config and extract_variation succeed on a variation with malformed skills.

Open question

skill_refs(None) still returns [], as documented. That is the case where a failed inspect_config feeds an empty list to a pruning write_skills. The README quickstart guards against it, but raising would remove the trap for every caller. That is a contract change, so it is not in this PR.

Test plan

  • make lint, make format-check, make typecheck
  • uv run pytest: 2209 passed, 11 skipped

🤖 Generated with Claude Code


Note

Overview
Agent Skills validation moves off the core config parser so experimental skills data cannot break config().invoke(), inspect_config, or extract_variation when apps never use skills (TESTING.md §0.3).

parse_ai_config now passes skills through unchanged; _parse_skills was removed from the parse path and exposed as skills_field_rejection_reason for reference discovery only. skill_refs validates the field and raises ValueError when skills is present but malformed (including null or one bad entry in a list), instead of dropping bad entries with warnings—avoiding a shortened list that would authorize write_skills(..., prune=True) to delete on-disk skills the config still references.

Docs (README.md, agents.md, AiConfigRep docstring) and tests were updated: parser tests expect malformed skills to parse successfully; skill_refs and lifecycle tests cover the new raise behavior and that core calls succeed with invalid skills.

Reviewed by Cursor Bugbot for commit 0f303f4. Bugbot is set up for automated code reviews on this repo. Configure here.

`parse_ai_config` rejected a whole AI Config when its `skills` field was
malformed, so a bad experimental field failed `config().invoke()`,
`stream()`, graph nodes and `extract_variation` for apps that never use
Agent Skills. That breaks TESTING.md §0.3: experimental behaviour must not
break a core call.

The parser now passes `skills` through unchecked. `skill_refs` validates it
instead and raises `ValueError` when the field is present but malformed,
including `skills: null` and a single bad entry. It used to drop bad entries
with a warning, which was safe only because the parser had already rejected
them. Raising keeps the original reason for the check: `write_skills` with
`prune=True` never receives an empty or partial list for a field that is
present. An absent field, or a config that is not a dict, still returns `[]`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@XieX
XieX merged commit 40fd647 into xie/skills-experimental-entry-point Oct 7, 2026
8 checks passed
XieX added a commit to launchdarkly/js-ai-sdk that referenced this pull request Oct 7, 2026
## Summary

A malformed `skills` field no longer fails core config calls. Today
`parseAiConfig` rejects the whole AI Config when `skills` is malformed.
`config().invoke()`, `stream()`, graph nodes and `extractVariation` then
raise, and `inspectConfig` returns `config: null`, even for apps that
never use Agent Skills. Agent Skills is experimental, and TESTING.md
§0.3 says experimental behaviour must not break a core call.

- **`parseAiConfig` passes `skills` through unchecked.**
- **`skillRefs` validates the field and throws `TypeError`** when it is
present but malformed, including `skills: null` and a single bad entry.
Before, it dropped bad entries with a warning, which was only safe
because the parser had already rejected them.
- **An absent field, or a nullish config, still returns `[]`.**
- **`AiConfigRep.skills` is typed `unknown`**, since the parser no
longer checks it. The root API report changes accordingly. Skills has
never been in an npm release, so no published code reads the old type.

Throwing rather than dropping keeps the reason the parser checked in the
first place. `writeSkills` with the default `prune: true` deletes every
managed skill missing from the list it is given, so `skillRefs` never
returns an empty or partial list for a field that is present.

Stacked on #114. Python counterpart: launchdarkly/python-ai-sdk#146.

## Tests

- **Parser:** a malformed `skills` field of every kind (null, non-array,
bad entry, one bad entry among good ones) parses successfully and passes
through unchanged.
- **`skillRefs`:** the cases the parser used to reject, including
`null`, now throw `TypeError` naming the field. One bad entry rejects
the whole field. The error does not echo the rejected key.
- **Core calls:** `inspectConfig` and `extractVariation` succeed on a
variation with malformed `skills`.

## Open question

`skillRefs(null)` still returns `[]`, as documented. That is the case
where a failed `inspectConfig` feeds an empty list to a pruning
`writeSkills`. Throwing would remove that trap for every caller. That is
a contract change, so it is not in this PR.

## Test plan

- [x] Build, typecheck, Biome, Sherif, `api:check`
- [x] All workspace tests: client 1240 passed, 10 skipped
- [x] `test:integration`: 28 passed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Moves **Agent Skills** validation out of the core config parser so a
bad `skills` payload cannot break `config().invoke()`, `inspectConfig`,
or `extractVariation`.
> 
> **`parseAiConfig`** no longer validates `skills`; the field is passed
through as-is. **`AiConfigRep.skills`** is now typed **`unknown`** (API
report updated). **`skillRefs`** owns validation: it still returns `[]`
when `skills` is absent or `config` is nullish, but throws
**`TypeError`** when `skills` is present yet malformed (including
`null`, wrong shape, or one bad entry among good ones)—replacing the old
warn-and-drop behavior so **`writeSkills`** with default **`prune:
true`** never gets a shortened list that could delete on-disk skills the
config still references.
> 
> Tests and docs (README, `agents.md`) reflect the experimental-field
rule: core calls tolerate malformed `skills`; consumers validate at
**`skillRefs`**.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
42968ff. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
@XieX
XieX deleted the xie/skills-validate-refs-on-use branch October 7, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants