Skip to content

[codex] add ai-assisted detection demo and portfolio docs#18

Merged
stacknil merged 1 commit intomainfrom
codex/add-ai-assisted-detection-demo
Mar 28, 2026
Merged

[codex] add ai-assisted detection demo and portfolio docs#18
stacknil merged 1 commit intomainfrom
codex/add-ai-assisted-detection-demo

Conversation

@stacknil
Copy link
Copy Markdown
Owner

Summary

This PR turns the repository into a clearer two-demo telemetry-lab portfolio artifact and adds a constrained ai-assisted-detection-demo alongside the existing telemetry-window-demo.

The new demo is intentionally framed as a small, reviewable security engineering prototype. Detection, grouping, and ATT&CK attachment stay deterministic. The LLM stage is limited to JSON-only case drafting, requires human verification, and is fail-closed on invalid or unsafe output.

Problem

Before this change, the repository only presented the time-series telemetry workflow, which made the broader telemetry-to-detection story hard to see from the landing page. There was also no second demo showing how deterministic telemetry processing could feed an auditable AI-assisted SOC-style workflow.

For a public portfolio repository, that made the project read as a single prototype instead of a coherent lab with multiple small, credible demos.

Root cause

The repo structure, README narrative, CLI surface, and test coverage had not yet been extended to represent the second demo as a first-class artifact. The AI-assisted detection pipeline, its guardrails, and its audit semantics needed to be encoded in code, artifacts, tests, and docs rather than implied.

Fix

This PR adds demos/ai-assisted-detection-demo with:

  • sample auth, web, and process telemetry in JSONL
  • deterministic rules and ATT&CK mappings
  • a local JSON output schema
  • generated demo artifacts for rule hits, case bundles, case summaries, the analyst report, and audit traces

It also adds the corresponding Python implementation under src/telemetry_window_demo/ai_assisted_detection_demo/ and exposes the demo through python -m telemetry_window_demo.cli run-ai-demo.

The pipeline is hardened to behave conservatively:

  • telemetry is explicitly treated as untrusted data
  • instructions and evidence are separated
  • non-JSON and malformed JSON are rejected
  • schema-invalid outputs are rejected
  • action-taking and final-verdict language are rejected
  • summaries with a mismatched case_id are rejected
  • malformed rule or ATT&CK metadata fails safely and is surfaced in the audit trail and main report

The docs are updated so the repository reads clearly as a two-demo project. The top-level README now includes a concise comparison matrix, the AI-assisted demo README includes accepted / rejected / degraded reviewer walkthroughs, and the design doc states the lifecycle contract and audit schema version explicitly.

Validation

I ran the full test suite after the final documentation and integration pass:

  • python -m pytest -> 31 passed

I also verified that the documentation and generated artifacts use the same audit schema version string:

  • ai-assisted-detection-audit/v1

Notes

This remains a portfolio-grade prototype, not a production SOC system. The LLM does not make final incident decisions, does not modify rules, and does not execute response actions.

@stacknil stacknil force-pushed the codex/add-ai-assisted-detection-demo branch from 6614959 to 96a23c0 Compare March 28, 2026 04:17
Includes the rebased headless plotting backend fix needed for CI after updating onto main.
@stacknil stacknil force-pushed the codex/add-ai-assisted-detection-demo branch from 96a23c0 to 5e7154b Compare March 28, 2026 04:21
@stacknil stacknil marked this pull request as ready for review March 28, 2026 04:21
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@stacknil stacknil merged commit ed46372 into main Mar 28, 2026
2 checks passed
@stacknil stacknil deleted the codex/add-ai-assisted-detection-demo branch March 28, 2026 04:23
@stacknil stacknil restored the codex/add-ai-assisted-detection-demo branch March 28, 2026 04:24
@stacknil stacknil deleted the codex/add-ai-assisted-detection-demo branch March 28, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant