Add explicit CSPT detection skill, categorization, fixtures, and tests#789
Add explicit CSPT detection skill, categorization, fixtures, and tests#789afraa786 wants to merge 4 commits into
Conversation
Greptile SummaryThis PR adds dedicated Client-Side Path Traversal detection and reporting support. The main changes are:
Confidence Score: 4/5SARIF class separation and the documented browser paths need fixes before merging.
strix/report/sarif.py, tests/fixtures/cspt/positive_direct.html, tests/fixtures/cspt/positive_encoded.html, docs/cspt-evidence-runbook.md
|
| Filename | Overview |
|---|---|
| strix/tools/reporting/tool.py | Adds validation and propagation for the optional CSPT finding class while preserving the existing default. |
| strix/report/sarif.py | Exports the structured finding class, but class fingerprints still depend on title keywords. |
| strix/skills/vulnerabilities/client_side_path_traversal.md | Adds detailed CSPT detection and reporting guidance, including path examples with incorrect normalization outcomes. |
| docs/cspt-evidence-runbook.md | Adds a manual browser and proxy workflow, but its main fixture example expects the wrong outbound path. |
| tests/fixtures/cspt/positive_direct.html | Adds a direct fragment-to-fetch traversal fixture whose documented normalized path is incorrect. |
| tests/fixtures/cspt/positive_encoded.html | Adds an encoded query-to-XHR traversal fixture whose payload escapes the documented API prefix. |
| tests/test_cspt_skill.py | Tests classification and fixture structure but does not verify actual normalized request paths or title-independent fingerprints. |
Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 3
strix/report/sarif.py:786
**Structured Class Can Still Collide**
A report with `finding_class="client_side_path_traversal"` but a title such as `Path Traversal via fragment` still matches the generic `path traversal` keyword. It then receives the same class fingerprint as server-side traversal, so dismissal tooling can collapse the two classes despite the new structured field. The fingerprint should use the non-default `finding_class`, with title matching only as a legacy fallback.
### Issue 2 of 3
tests/fixtures/cspt/positive_direct.html:8-9
**Traversal Reaches A Different Path**
Joining `../../admin/keys` to `/api/orders/` while retaining `/detail` normalizes to `/admin/keys/detail`, not `/api/admin/keys`. The fixture and runbook therefore tell investigators to expect the wrong request path from the browser, which can invalidate the intended evidence comparison.
### Issue 3 of 3
tests/fixtures/cspt/positive_encoded.html:8-9
**Encoded Example Escapes API Prefix**
After decoding, `/api/resources/../../admin/config` normalizes to `/admin/config`, not `/api/admin/config`, because the two parent segments remove both `resources` and `api`. The fixture consequently demonstrates a different endpoint from the one its evidence claims.
Reviews (1): Last reviewed commit: "Add explicit CSPT detection skill, categ..." | Re-trigger Greptile
- SARIF class fingerprint now prefers the structured finding_class over title-keyword matching, so a CSPT finding whose title contains "path traversal" no longer collapses onto the server-side traversal fingerprint - Correct documented normalized paths in positive_direct/positive_encoded fixtures, the evidence runbook, and the skill (leading ../ segments pop both path prefixes, e.g. /admin/keys/detail, not /api/admin/keys) - Add regression tests covering the fingerprint precedence and the corrected fixture paths Addresses PR usestrix#789 review (Closes usestrix#776)
|
Pushed a fix (c9cbfa7) addressing the Greptile review feedback:
Ready for another look |
Review status: Changes requiredThank you for the work on this. The dedicated Client-Side Path Traversal (CSPT) skill, fixtures, reporting field, and documentation are a useful foundation. However, this PR cannot be accepted in its current form because it does not yet satisfy several acceptance criteria from issue #776, and parts of the validation guidance are technically incorrect. The following items need to be addressed before the PR is ready for another review. 1. Actual data-flow detection and browser/proxy confirmation are not testedThe acceptance criteria explicitly require:
The current tests do not exercise either capability.
This does not establish that the source actually flows into the sink. A fixture containing unrelated source and sink statements would still pass. Likewise, the normalization tests exercise a hand-written Python path function. They do not validate browser URL parsing, execute a request, inspect an HTTP Archive (HAR), or confirm what appears at the proxy/server. There is also no automated test showing that Strix:
The PR description acknowledges that live confirmation is manual, but that does not meet the issue’s automated-regression requirement. Required changes
The 2. The primary encoded-fragment example does not traverseThe runbook uses the following fragment payload: In that flow, the application does not decode the fragment before concatenating it into the request URL. Percent-encoded slashes remain part of a path segment; the browser does not turn Therefore: does not normalize to: The WHATWG URL Standard recognizes The skill currently makes the broader incorrect claim that browsers decode The encoded query fixture is not proof of that behavior because Required changes
3. SARIF primary fingerprints can still collapse CSPT and server-side traversal findingsThe latest commit correctly changed the custom class fingerprint to prefer the structured For locationless Common Weakness Enumeration (CWE)-22 findings, both results are anchored to I reproduced this using:
Both received the same The new fingerprint regression test asserts only the custom property. It does not inspect Required changes
4. The confirmation model is too restrictive and will cause false negativesThe skill says a case is CSPT only when all of the following are present:
It later classifies requests without credentials or successful reach as “not CSPT.” See the confirmation and false-positive sections. This conflates two separate questions:
A path manipulation primitive may be valid even when the immediate request is unauthenticated or returns 404. It may still become exploitable when chained with response rendering, a public state-changing endpoint, application-added authorization, or another client-side sink. Required changesSeparate the validation model into:
Credentials and successful reach should affect reportability and severity, but they should not redefine whether path traversal occurred. 5. Skill references use a non-loadable nameThe runtime-discoverable skill key is: However, the reporting tool refers agents to: See the reporting-tool documentation. The hyphenated name is not accepted by Required changes
6. Documentation and PR state need cleanupThe public skills documentation does not list the new CSPT skill, leaving the documented skill catalog incomplete. The PR description also reports 25 CSPT tests, but the current branch contains 30. The full-suite counts are similarly stale. At the time of this review, the branch is 11 commits behind Required changes
Verification performedCheck | Result -- | -- CSPT-specific tests on the PR head | 30 passed CSPT-specific tests after a virtual merge with current main | 30 passed Ruff lint and formatting | Passed mypy | Passed Bandit | Passed git diff --check | Passed Full test suite | 270 passed, 1 skipped, 2 failedThe two full-suite failures were reproduced on current Required before re-review
Once these blockers are addressed with reproducible evidence, the PR should be in a much stronger position for acceptance. ## Review status: Changes requiredThank you for the work on this. The dedicated Client-Side Path Traversal (CSPT) skill, fixtures, reporting field, and documentation are a useful foundation. However, this PR cannot be accepted in its current form because it does not yet satisfy several acceptance criteria from [issue #776](#776), and parts of the validation guidance are technically incorrect. The following items need to be addressed before the PR is ready for another review. 1. Actual data-flow detection and browser/proxy confirmation are not testedThe acceptance criteria explicitly require:
The current tests do not exercise either capability.
This does not establish that the source actually flows into the sink. A fixture containing unrelated source and sink statements would still pass. Likewise, the [normalization tests]( strix/tests/test_cspt_skill.py Lines 349 to 402 in c9cbfa7 There is also no automated test showing that Strix:
The PR description acknowledges that live confirmation is manual, but that does not meet the issue’s automated-regression requirement. Required changes
The strix/tests/fixtures/cspt/positive_normalized.html Lines 16 to 23 in c9cbfa7 2. The primary encoded-fragment example does not traverseThe runbook uses the following fragment payload: See [the runbook procedure]( strix/docs/cspt-evidence-runbook.md Lines 41 to 44 in c9cbfa7 In that flow, the application does not decode the fragment before concatenating it into the request URL. Percent-encoded slashes remain part of a path segment; the browser does not turn Therefore: does not normalize to: The [WHATWG URL Standard](https://url.spec.whatwg.org/#url-path-segment) recognizes The skill currently makes the broader incorrect claim that browsers decode strix/strix/skills/vulnerabilities/client_side_path_traversal.md Lines 119 to 124 in c9cbfa7 The encoded query fixture is not proof of that behavior because Required changes
3. SARIF primary fingerprints can still collapse CSPT and server-side traversal findingsThe latest commit correctly changed the custom class fingerprint to prefer the structured For locationless Common Weakness Enumeration (CWE)-22 findings, both results are anchored to I reproduced this using:
Both received the same The new [fingerprint regression test]( strix/tests/test_cspt_skill.py Lines 303 to 341 in c9cbfa7 partialFingerprints, so the primary collision remains undetected.
Required changes
4. The confirmation model is too restrictive and will cause false negativesThe skill says a case is CSPT only when all of the following are present:
It later classifies requests without credentials or successful reach as “not CSPT.” See the [confirmation and false-positive sections]( strix/strix/skills/vulnerabilities/client_side_path_traversal.md Lines 168 to 220 in c9cbfa7 This conflates two separate questions:
A path manipulation primitive may be valid even when the immediate request is unauthenticated or returns 404. It may still become exploitable when chained with response rendering, a public state-changing endpoint, application-added authorization, or another client-side sink. Required changesSeparate the validation model into:
Credentials and successful reach should affect reportability and severity, but they should not redefine whether path traversal occurred. 5. Skill references use a non-loadable nameThe runtime-discoverable skill key is: However, the reporting tool refers agents to: See [the reporting-tool documentation]( strix/strix/tools/reporting/tool.py Lines 565 to 578 in c9cbfa7 The hyphenated name is not accepted by Required changes
6. Documentation and PR state need cleanupThe public [skills documentation](https://github.com/usestrix/strix/blob/c9cbfa76665869b815605aa65751dc1ce78d636b/docs/advanced/skills.mdx) does not list the new CSPT skill, leaving the documented skill catalog incomplete. The PR description also reports 25 CSPT tests, but the current branch contains 30. The full-suite counts are similarly stale. At the time of this review, the branch is 11 commits behind Required changes
Verification performed
The two full-suite failures were reproduced on current Required before re-review
Once these blockers are addressed with reproducible evidence, the PR should be in a much stronger position for acceptance. |
- SARIF class fingerprint now prefers the structured finding_class over title-keyword matching, so a CSPT finding whose title contains "path traversal" no longer collapses onto the server-side traversal fingerprint - Correct documented normalized paths in positive_direct/positive_encoded fixtures, the evidence runbook, and the skill (leading ../ segments pop both path prefixes, e.g. /admin/keys/detail, not /api/admin/keys) - Add regression tests covering the fingerprint precedence and the corrected fixture paths Addresses PR usestrix#789 review (Closes usestrix#776)
…cs fixes Resolve the "changes required" review on the CSPT PR: - SARIF: add a shared `_class_token` helper preferring the structured `finding_class` over a title keyword, and use it in BOTH `_class_fingerprint` and the synthetic branch of `_primary_fingerprint`, so a locationless CSPT finding no longer collapses onto a server-side traversal's primary fingerprint. Require CSPT reports to carry a discriminator (endpoint/method or a code location) so multiple locationless CSPT findings stay distinct. - Dedup: include `finding_class` in the candidate and in `_prepare_report_for_comparison`, and teach the judge that a differing finding_class under a shared CWE is not a duplicate. - Skill/runbook: correct the %2f guidance — the browser does not decode `%2f` during normalization; only literal-slash dot-segments traverse, and `%2f`-based traversal is an app-layer decode. Switch fragment examples to raw `../../`. Split confirmation into a CSPT *primitive* (path steered) vs. an *impact* level (credentials/reach/chain) to stop false negatives. Fix non-loadable skill names (`client_side_path_traversal`, `path_traversal_lfi_rfi`). - Fixtures: drive `positive_normalized` from the fragment so it serves under a plain static server. - Tests: add `test_cspt_browser_capture.py` (headless Chromium + local request-capture server asserting the exact outbound path per positive and no traversal per negative) and `test_cspt_dataflow.py` (deterministic taint tracker that fails when source and sink are present but unconnected); add primary-fingerprint and dedup regressions. Add playwright to dev deps. - Docs: list the CSPT skill in the public skills catalog.
…cs fixes Resolve the "changes required" review on the CSPT PR: - SARIF: add a shared `_class_token` helper preferring the structured `finding_class` over a title keyword, used in BOTH `_class_fingerprint` and the synthetic branch of `_primary_fingerprint`, so a locationless CSPT finding no longer collapses onto a server-side traversal's primary fingerprint. Require CSPT reports to carry a discriminator (endpoint/method or a code location) so multiple locationless CSPT findings stay distinct. - Dedup: include `finding_class` in the candidate and in `_prepare_report_for_comparison`; teach the judge that a differing finding_class under a shared CWE is not a duplicate. - Skill/runbook: correct the %2f guidance — the browser does not decode `%2f` during normalization; only literal-slash dot-segments traverse, and `%2f`-based traversal is an app-layer decode. Switch fragment examples to raw `../../`. Split confirmation into a CSPT *primitive* (path steered) vs. an *impact* level (credentials/reach/chain) to stop false negatives. Fix non-loadable skill names to the underscore keys (`client_side_path_traversal`, `path_traversal_lfi_rfi`), including the skill frontmatter. - Fixtures: drive `positive_normalized` from the fragment so it serves under a plain static server. - Tests: add `test_cspt_browser_capture.py` — headless Chromium (Playwright) + a local request-capture server asserting the exact outbound path per positive and no traversal per negative, plus WHATWG ground-truth tests proving raw `../../` and `%2e%2e/` collapse while un-decoded `..%2f..%2f` does not. Add `test_cspt_dataflow.py` — a deterministic taint tracker that fails when a fixture's source and sink are present but unconnected. Add primary-fingerprint and dedup regressions. Add playwright to dev deps. - Docs: list the CSPT skill in the public skills catalog. Known follow-up (not blocking): `uv.lock` is left at format revision 2; run `uv lock` with the repo's pinned uv to add the playwright dev dep without the unrelated revision-2->3 rewrite. There is no test CI workflow yet; when one is added it should run `uv run playwright install chromium` so the browser tests execute (they importorskip otherwise).
c9cbfa7 to
c0cc1a8
Compare
|
@Am0stafa Pushed commits addressing all 6 points from your review, with real One known, explicitly-flagged follow-up: the uv.lock isn't yet updated with Ready for another look whenever convenient thanks for the detailed review, |
|
Strong, high-quality PR — merge-ready after one packaging fix.
|
|
@0xallam, @bearsyankees |
Add Client-Side Path Traversal (CSPT) detection support
Closes #776. Adds a dedicated CSPT skill, fixtures, a machine-readable finding_class, SARIF/dedup separation from server-side traversal, and automated regression coverage for source-to-sink detection and browser outbound-path confirmation.
Changes in response to review
The maintainer's "changes required" review raised 6 issues; all are addressed:
Real data-flow + browser/proxy detection is now tested.
tests/test_cspt_browser_capture.py drives every fixture in headless Chromium (Playwright) against a local HTTP request-capture server, and asserts the exact outbound path the server observed: positives emit the traversed path (e.g. /admin/keys/detail, /admin/config, /admin/settings, /admin/summary), negatives emit no traversed request. It also pins WHATWG URL ground truth: raw ../../ and %2e%2e/%2e%2e/ collapse, while un-decoded ..%2f..%2f does not.
tests/test_cspt_dataflow.py is a deterministic taint tracker that follows a source through assignments/transforms into a sink's path argument. It fails closed: a fixture with an unrelated source and sink (present but unconnected) is not flagged (proven by two meta-tests).
positive_normalized.html was reworked to read its route from the fragment (hash router), so it serves correctly under a plain static server.
%2f normalization guidance corrected. The skill and runbook no longer claim the browser decodes %2f/%2e during path normalization. They now distinguish browser-recognized dot-segments (.., %2e%2e with literal /), encoded separators that stay literal (%2f), and application-layer decoding before URL construction. Fragment examples use raw ../../. Every claim has a browser assertion.
SARIF primary-fingerprint collision fixed. A shared _class_token() helper prefers the structured finding_class over a title keyword and is used in both _class_fingerprint() and the synthetic branch of _primary_fingerprint(). The reviewer's exact repro (locationless CSPT vs. server-side traversal, both CWE-22) now yields different partialFingerprints.primaryLocationLineHash values; a regression test asserts this. CSPT reports must carry a discriminator (endpoint/method or code location) so multiple locationless CSPT findings don't collapse. finding_class is included in dedup (_prepare_report_for_comparison + candidate + judge prompt).
Confirmation model separated. The skill now distinguishes a confirmed CSPT primitive (outbound path differs due to attacker-controlled client input) from confirmed exploit impact (credentials / reach / chain). Credentials and reach drive reportability and severity but do not decide whether traversal occurred, preventing false negatives.
Loadable skill names fixed. All agent-facing references use the underscore keys client_side_path_traversal and path_traversal_lfi_rfi (including skill frontmatter). No hyphenated cross-references remain.
Docs updated. CSPT is listed in the public skills catalog (docs/advanced/skills.mdx). Branch rebased onto latest main (clean, no conflicts).
Verification
CSPT-specific tests: 54 passed
Full test suite: 321 passed, 1 skipped, 3 failed
ruff (lint + format): clean on changed files
mypy: clean
bandit: clean
The 3 full-suite failures (test_config_loader::test_persist_current_sets_0600_mode, two test_runner_root_prompt::…) reproduce on clean main and are not introduced by this PR.
Commands:
uv run python -m pytest tests/test_cspt_skill.py tests/test_cspt_dataflow.py tests/test_cspt_browser_capture.py # 54 passed
uv run python -m pytest # full suite
uv run ruff check strix/ tests/ && uv run mypy strix/ && uv run bandit -c pyproject.toml -r strix/
The browser tests require Playwright + Chromium (uv run playwright install chromium); they importorskip and skip cleanly where unavailable.
Known follow-up (not blocking)
uv.lock: left at format revision 2. The playwright>=1.40 dev dep is declared in pyproject.toml but not yet in the lock, because the local uv (0.11.26) rewrites the lock to revision 3 (renaming upload_time to upload-time on every entry): about 3000 lines of unrelated churn. Run uv lock with the repo's pinned uv (or pin uv via required-version) to add the dep cleanly. The release build (uv sync --frozen, default deps only) is unaffected.
CI: there is no test workflow in this repo today (only build-release.yml on tags + pre-commit.ci for lint/type/security). When a test CI is added, it should run uv run playwright install chromium so the CSPT browser tests execute rather than skip.