Skip to content

feat(sec): SPAC de-SPAC lifecycle — 8-K milestones, merger-proxy & redemption extraction#166

Merged
sroussey merged 39 commits into
mainfrom
claude/funny-babbage-4k162y
Jun 24, 2026
Merged

feat(sec): SPAC de-SPAC lifecycle — 8-K milestones, merger-proxy & redemption extraction#166
sroussey merged 39 commits into
mainfrom
claude/funny-babbage-4k162y

Conversation

@sroussey

Copy link
Copy Markdown
Contributor

Summary

Builds out the SPAC de-SPAC lifecycle on top of the consolidated spac report, in three increments that all land here (none were previously on main):

  • §4a — 8-K milestone dates. Deterministically map 8-K item codes for known SPACs to lifecycle events (1.01 → definitive_agreement, 1.02 → terminated, 2.01 → completed, 5.07 → vote), grouped into spac_deal attempts via the event stream (idempotent replays, stable deal_index).
  • §4b / §4b.1 — merger-proxy extraction. AI-extract target/PIPE/consideration from the 14A/14C merger & revised-proxy family (DEFM14A/PREM14A, DEFM14C/PREM14C, DEFR14A/PRER14A) into spac_merger_extraction, correlated onto the matching deal; definitive statements emit the proxy milestone. Per-extractor confidence floor (SEC_MERGER_PROXY_CONFIDENCE_FLOOR).
  • §4c — redemption actuals. AI-extract realized share redemptions from a known SPAC's post-vote 8-K narrative and roll them up into total_redemption_amount.

§4c detail (this session's work)

When a known SPAC files an 8-K carrying item 5.07/2.01/8.01, ingestion escalates the fetch to the full submission .txt (the existing S-1/DRS mechanism), reads the primary document + EX-99.x exhibits, and processRedemption8K AI-extracts realized redemptions into a per-accession spac_redemption_extraction row. deriveDeals correlates redemption_amount / redemption_shares onto the matching spac_deal using an announcement-based window (upper bound = the next deal's announcement, ignoring outcome_date, so a redemption reported at/after closing still attaches). The deal column is the sole source sumRedemptions totals, so each redemption is counted exactly once (no event is emitted). New redemption extractor id (own version slot + dead-letters), SEC_REDEMPTION_MODEL / SEC_REDEMPTION_CONFIDENCE_FLOOR config, and a sec spac backfill-redemptions command that sweeps historical known-SPAC trigger 8-Ks off the bootstrapped filing metadata.

Design specs & the §4c implementation plan live in the PRD repo under docs/superpowers/specs/ and .../plans/.

Test Plan

  • bun test src/sec/forms/miscellaneous-filings/ src/sec/forms/registration-statements/s1/ src/storage/spac/ src/storage/versioning/ src/task/spac/ src/task/forms/266 pass, 0 fail
  • bunx tsc --noEmit → clean
  • No-double-count invariant proven by an idempotency e2e test (reprocessing the same 8-K keeps total_redemption_amount single)
  • Gating verified SPAC-only with defense-in-depth (task escalation + processForm8K re-gate + processRedemption8K trigger/SPAC/deal checks)
  • Reviewer: spot-check a real SPAC's redemption 8-K end-to-end via sec fetch form <cik> 8-K and sec spac report <cik>

Note: this PR carries all three SPAC increments (§4a/§4b/§4c) because none were yet on main. The pre-existing FetchQuarterlyIndexTask / FetchDailyIndexTask network-timeout test failures are unrelated to this work.

🤖 Generated with Claude Code


Generated by Claude Code

claude added 30 commits June 23, 2026 05:49
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Skip the milestone path when neither period-of-report nor filing_date is
available, so an undated 8-K can't write junk dates onto the deal/row.
Adds a regression test.
…lestone

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Adds RedemptionOutputSchema (shares, amount, price_per_share, confidence,
source_span) and extractRedemption() in sectionExtractors.ts, mirroring the
extractMergerDeal pattern. Covered by a two-case bun test.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Thread SpacRedemptionExtractionRepo through recomputeAndSaveDeals,
replacing the [] stopgap with actual redemption data from the repo.
claude added 7 commits June 24, 2026 04:30
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
The `sec version CLI` status test's expected extractor list was missing
`merger-proxy` and `redemption`, so `db setup` (which bootstraps all extractor
ids) produced more rows than asserted. This test lives outside the feature dirs
the local runs covered, so CI surfaced it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
- deriveDeals: first deal's redemption window is unbounded below, so a
  vote-results 8-K filed before a completion-only deal's date still attaches
  (was dropped → total_redemption_amount under-counted).
- processRedemption8K: parse/render of filer HTML is wrapped so a malformed
  body dead-letters the section instead of aborting the filing (restores the
  "extraction degrades, never aborts" invariant; mirrors the merger-proxy path).
- BackfillRedemptionsTask: isolate per-filing failures so one bad 8-K doesn't
  abort the historical sweep.
- redemption schema: minimum:0 on redemption_amount / price_per_share so a
  negative/sign-error value dead-letters rather than corrupting the rollup.
- extractRedemption: drop a figure-less response (null shares AND null amount)
  so the no-redemption case doesn't persist a meaningless row.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
recomputeAndSaveDeals upserted derived deals but never deleted rows from a
prior, larger derivation. When the derived deal set shrinks (event stream or
derivation logic changes), orphaned spac_deal rows survived and their stale
columns — notably redemption_amount — were still summed by sumRedemptions,
over-counting total_redemption_amount. Now delete any existing deal whose
deal_index is absent from the recomputed set before saving. Shared by the 8-K
milestone, merger-proxy, and redemption writers. Adds SpacRepo.deleteDeal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the SPAC de‑SPAC lifecycle on top of the consolidated spac report by adding deterministic 8‑K milestone mapping, AI-based merger‑proxy extraction/correlation, and AI-based redemption actual extraction/correlation (including a historical backfill CLI/task).

Changes:

  • Map selected 8‑K item codes into SPAC lifecycle events and deterministically derive spac_deal attempts from the event stream.
  • Add merger‑proxy extraction (14A/14C + revised proxy variants) with per-accession persistence and deal correlation/rollup.
  • Add redemption extraction from full-submission 8‑Ks (primary + EX‑99 exhibits), deal correlation/rollup, plus a sec spac backfill-redemptions sweep task.

Reviewed changes

Copilot reviewed 58 out of 58 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/task/spac/BackfillRedemptionsTask.ts Adds a task to select and reprocess historical known‑SPAC trigger-item 8‑Ks for redemption extraction.
src/task/spac/BackfillRedemptionsTask.test.ts Tests accession selection and dry-run behavior for the backfill task.
src/task/forms/ProcessAccessionDocFormTask.ts Escalates fetch to full submission .txt for known‑SPAC redemption-trigger 8‑Ks; routes merger proxies to processMergerProxy; passes fullSubmissionText into 8‑K storage.
src/task/forms/ProcessAccessionDocFormTask.redemption.test.ts Tests the 8‑K redemption full-submission fetch escalation behavior.
src/storage/versioning/extractorIds.ts Registers new extractor IDs (merger-proxy, redemption) and maps merger proxy forms to merger-proxy.
src/storage/versioning/extractorIds.test.ts Updates extractor ID list assertions and tests merger-proxy form mapping.
src/storage/versioning/componentRegistry.test.ts Updates registered component count to include new extractors.
src/storage/spac/SpacReportWriter.ts Adds writers for deal milestones, merger proxies, and redemptions; recomputes and reconciles derived deals.
src/storage/spac/SpacReportWriter.test.ts Adds coverage for milestone roll-forward, idempotency, proxy correlation, and proxy event emission rules.
src/storage/spac/SpacRepo.ts Adds getAllSpacs and deleteDeal to support backfill selection and derived-deal reconciliation.
src/storage/spac/SpacRedemptionExtractionSchema.ts Introduces per-accession redemption extraction storage schema/token.
src/storage/spac/SpacRedemptionExtractionRepo.ts Adds repo wrapper for redemption extractions.
src/storage/spac/SpacRedemptionExtractionRepo.test.ts Tests redemption extraction repo behavior (overwrite, query by cik).
src/storage/spac/SpacMergerExtractionSchema.ts Introduces per-accession merger-proxy extraction storage schema/token.
src/storage/spac/SpacMergerExtractionRepo.ts Adds repo wrapper for merger-proxy extractions.
src/storage/spac/SpacMergerExtractionRepo.test.ts Tests merger-proxy extraction repo behavior (overwrite, query by cik).
src/storage/spac/SpacEventSchema.ts Updates lifecycle event vocabulary documentation to reflect new event sources.
src/storage/spac/spacDealGrouping.ts Implements deterministic deriveDeals combining events + merger proxy extractions + redemption extractions.
src/storage/spac/spacDealGrouping.test.ts Unit tests for deal derivation and merger-proxy correlation behavior.
src/storage/spac/spacDealGrouping.redemption.test.ts Unit tests for redemption correlation windowing and precedence rules.
src/storage/spac/recomputeDeals.reconcile.test.ts Tests reconciliation deleting orphan deal rows to prevent stale rollup totals.
src/sec/forms/registration-statements/s1/sectionRunner.ts Adds parseConfidenceFloor and allows per-extractor confidence floors in makeRunSection.
src/sec/forms/registration-statements/s1/sectionRunner.test.ts Tests parseConfidenceFloor and per-extractor confidenceFloor behavior.
src/sec/forms/registration-statements/s1/sectionExtractors.ts Adds AI extractors for merger deal terms and redemption actuals.
src/sec/forms/registration-statements/s1/sectionExtractors.test.ts Adds tests for extractMergerDeal behavior.
src/sec/forms/registration-statements/s1/redemptionSchema.ts Defines structured schema for redemption extraction output.
src/sec/forms/registration-statements/s1/redemptionModel.ts Adds model selection + confidence floor config for redemption extraction.
src/sec/forms/registration-statements/s1/redemptionModel.test.ts Tests redemption model env overrides and confidence floor fallback.
src/sec/forms/registration-statements/s1/parseSubmission.ts Adds parseEightKSubmission to slice primary + EX‑99 exhibits from full submissions.
src/sec/forms/registration-statements/s1/parseEightKSubmission.test.ts Tests parseEightKSubmission behavior.
src/sec/forms/registration-statements/s1/mergerModel.ts Adds model selection + confidence floor config for merger-proxy extraction.
src/sec/forms/registration-statements/s1/mergerModel.test.ts Tests merger-proxy confidence floor env overrides and fallback.
src/sec/forms/registration-statements/s1/mergerDealSchema.ts Defines structured schema for merger-proxy deal extraction output.
src/sec/forms/registration-statements/s1/extractRedemption.test.ts Tests redemption extractor behavior (nulls, span requirement, schema validation).
src/sec/forms/registration-statements/s1/DocumentSegmenter.ts Adds merger-proxy-specific section headings/patterns for segmentation.
src/sec/forms/proxies-information-statements/mock_data/merger-proxy/SOURCES.md Documents merger-proxy fixtures provenance and intent.
src/sec/forms/proxies-information-statements/mock_data/merger-proxy/defm14a_sample.txt Adds compact DEFM14A fixture for merger-proxy e2e plumbing tests.
src/sec/forms/proxies-information-statements/Form_PRER14A.ts Uses parseRegistrationSubmission parsing for PRER14A.
src/sec/forms/proxies-information-statements/Form_PREM14C.ts Uses parseRegistrationSubmission parsing for PREM14C.
src/sec/forms/proxies-information-statements/Form_PREM14A.ts Uses parseRegistrationSubmission parsing for PREM14A.
src/sec/forms/proxies-information-statements/Form_DEFR14A.ts Uses parseRegistrationSubmission parsing for DEFR14A.
src/sec/forms/proxies-information-statements/Form_DEFM14C.ts Uses parseRegistrationSubmission parsing for DEFM14C.
src/sec/forms/proxies-information-statements/Form_DEFM14A.ts Uses parseRegistrationSubmission parsing for DEFM14A.
src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.ts Implements merger-proxy extraction pipeline, persistence, provenance, and deal recompute/rollup.
src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.e2e.test.ts End-to-end tests for merger-proxy extraction/correlation across proxy form variants.
src/sec/forms/miscellaneous-filings/spac8kRedemptionTriggers.ts Defines redemption-trigger 8‑K item codes and a helper matcher.
src/sec/forms/miscellaneous-filings/spac8kMilestones.ts Maps key 8‑K item codes to SPAC lifecycle events.
src/sec/forms/miscellaneous-filings/spac8kMilestones.test.ts Tests item-to-event mapping and 8‑K → SPAC milestone wiring/idempotency.
src/sec/forms/miscellaneous-filings/redemption8k.ts Implements redemption extraction from full-submission 8‑Ks and deal correlation/rollup.
src/sec/forms/miscellaneous-filings/redemption8k.test.ts Unit tests for redemption extraction gating and persistence/correlation.
src/sec/forms/miscellaneous-filings/redemption8k.e2e.test.ts End-to-end tests for 8‑K redemption extraction and idempotent rollup.
src/sec/forms/miscellaneous-filings/Form_8_K.storage.ts Wires SPAC milestone mapping and redemption extraction into 8‑K processing.
src/config/TestingDI.ts Registers in-memory storages for new SPAC extraction tables for tests.
src/config/setupAllDatabases.ts Adds setup calls for new SPAC extraction tables.
src/config/DefaultDI.ts Registers persistent storages for new SPAC extraction tables.
src/commands/spac.ts Adds sec spac backfill-redemptions CLI command.
src/cli/groups/version.test.ts Updates CLI version tests to include new extractor IDs.
CLAUDE.md Updates repo docs to describe new SPAC milestone/proxy/redemption extraction behavior and CLI usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/sec/forms/registration-statements/s1/parseEightKSubmission.test.ts Outdated
Comment thread src/sec/forms/registration-statements/s1/sectionRunner.ts
Comment thread src/task/spac/BackfillRedemptionsTask.ts
Comment thread CLAUDE.md
- parseEightKSubmission.test.ts: use bun:test (repo convention; was the only
  file importing vitest).
- sectionRunner parseConfidenceFloor: correct the JSDoc — a 0 floor admits
  every row (disables the floor); NaN is the drop-everything case.
- BackfillRedemptionsTask: query filings by (form, cik) to use the
  ["form","cik"] index instead of loading all of a SPAC's filings and filtering.
- CLAUDE.md: the milestone grouping function is deriveDeals, not the stale
  deriveDealsFromEvents reference.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
@sroussey sroussey merged commit 7a0f271 into main Jun 24, 2026
1 check passed
@sroussey sroussey deleted the claude/funny-babbage-4k162y branch June 24, 2026 06:05
sroussey pushed a commit that referenced this pull request Jun 24, 2026
Resolves conflicts created by PR #166 (SPAC de-SPAC lifecycle / merger-proxy /
redemption extraction) landing on main after this PR opened.

Conflicts resolved:

- src/sec/forms/miscellaneous-filings/Form_8_K.storage.ts
  - Function signature combines both side's additive params:
    extractor_id, extractor_version (this PR), fullSubmissionText, model (#166).
  - Event writes go through replaceEvents() (this PR), threading
    extractor_id + extractor_version into the version-scoped delete-then-insert.
  - SPAC milestone mapping + redemption extraction blocks from #166 follow
    unchanged after the events are persisted.

- src/task/forms/ProcessAccessionDocFormTask.ts
  - Keep TypeAccessionNumber import (this PR), processMergerProxy +
    hasRedemptionTriggerItem imports (#166).
  - 8-K dispatch call site passes both extractor_id/extractor_version and
    fullSubmissionText into processForm8K; merger-proxy case from #166 follows.

- src/sec/forms/registration-statements/s1/sectionExtractors.ts (auto-merged
  cleanly by git but the new extractMergerDeal / extractRedemption functions
  still called the pre-PR wrapUntrusted shape + UNTRUSTED_PREAMBLE constant,
  which this PR removed. Both updated to the nonce-fence API
  (wrapUntrusted -> { wrapped, nonce }, buildUntrustedPreamble(nonce))
  so the new SPAC AI extractors get the per-call nonce fence + multi-stage
  defang for free. Without this, the prompts would interpolate as
  "[object Object]" and the model receives garbage.

Verification:
- targeted: bun test src/sec/forms/miscellaneous-filings/ \
    src/sec/forms/proxies-information-statements/ src/task/forms/ \
    src/storage/spac/ src/storage/form-8k-event/ \
    src/sec/forms/registration-statements/  -> 229 pass / 0 fail.
- full: bun test  -> 1410 pass / 7 fail. All 7 fails are pre-existing
  FetchDailyIndexTask + FetchQuarterlyIndexTask 5000ms network timeouts
  unrelated to this PR (sandbox can't reach SEC.gov reliably).
- bun run build  -> clean (bun build + tsc, no errors).

Co-Authored-By: Claude <noreply@anthropic.com>
sroussey pushed a commit that referenced this pull request Jun 24, 2026
…e-fence API

The new SPAC extractors added in PR #166 (extractMergerDeal,
extractRedemption) called the pre-PR wrapUntrusted shape (returning a
string) and the removed UNTRUSTED_PREAMBLE constant. After this PR
swapped wrapUntrusted to return { wrapped, nonce } and replaced the
constant with buildUntrustedPreamble(nonce), the surviving call sites
template-interpolated UNTRUSTED_PREAMBLE as a free identifier ->
compile error (TS2552), and even if the type had survived the
{ wrapped, nonce } object would have rendered as "[object Object]"
in the prompt -> the model receives garbage and silently returns
nothing (caught by Form_DEFM14A.storage.e2e.test.ts target_name=null
assertions in the post-merge run).

Both extractors now use the same nonce-fence + multi-stage defang as
the other section extractors -- a forced consequence of the merge,
extending the per-call nonce + entity decode + NFKC + zero-width
strip protection to the new SPAC AI extractors at no extra design
cost.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants