feat(sec): SPAC de-SPAC lifecycle — 8-K milestones, merger-proxy & redemption extraction#166
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Skip the milestone path when neither period-of-report nor filing_date is available, so an undated 8-K can't write junk dates onto the deal/row. Adds a regression test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
…lestone Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Adds RedemptionOutputSchema (shares, amount, price_per_share, confidence, source_span) and extractRedemption() in sectionExtractors.ts, mirroring the extractMergerDeal pattern. Covered by a two-case bun test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Thread SpacRedemptionExtractionRepo through recomputeAndSaveDeals, replacing the [] stopgap with actual redemption data from the repo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
The `sec version CLI` status test's expected extractor list was missing `merger-proxy` and `redemption`, so `db setup` (which bootstraps all extractor ids) produced more rows than asserted. This test lives outside the feature dirs the local runs covered, so CI surfaced it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
- deriveDeals: first deal's redemption window is unbounded below, so a vote-results 8-K filed before a completion-only deal's date still attaches (was dropped → total_redemption_amount under-counted). - processRedemption8K: parse/render of filer HTML is wrapped so a malformed body dead-letters the section instead of aborting the filing (restores the "extraction degrades, never aborts" invariant; mirrors the merger-proxy path). - BackfillRedemptionsTask: isolate per-filing failures so one bad 8-K doesn't abort the historical sweep. - redemption schema: minimum:0 on redemption_amount / price_per_share so a negative/sign-error value dead-letters rather than corrupting the rollup. - extractRedemption: drop a figure-less response (null shares AND null amount) so the no-redemption case doesn't persist a meaningless row. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
recomputeAndSaveDeals upserted derived deals but never deleted rows from a prior, larger derivation. When the derived deal set shrinks (event stream or derivation logic changes), orphaned spac_deal rows survived and their stale columns — notably redemption_amount — were still summed by sumRedemptions, over-counting total_redemption_amount. Now delete any existing deal whose deal_index is absent from the recomputed set before saving. Shared by the 8-K milestone, merger-proxy, and redemption writers. Adds SpacRepo.deleteDeal. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
Contributor
There was a problem hiding this comment.
Pull request overview
Implements the SPAC de‑SPAC lifecycle on top of the consolidated spac report by adding deterministic 8‑K milestone mapping, AI-based merger‑proxy extraction/correlation, and AI-based redemption actual extraction/correlation (including a historical backfill CLI/task).
Changes:
- Map selected 8‑K item codes into SPAC lifecycle events and deterministically derive
spac_dealattempts from the event stream. - Add merger‑proxy extraction (14A/14C + revised proxy variants) with per-accession persistence and deal correlation/rollup.
- Add redemption extraction from full-submission 8‑Ks (primary + EX‑99 exhibits), deal correlation/rollup, plus a
sec spac backfill-redemptionssweep task.
Reviewed changes
Copilot reviewed 58 out of 58 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/task/spac/BackfillRedemptionsTask.ts | Adds a task to select and reprocess historical known‑SPAC trigger-item 8‑Ks for redemption extraction. |
| src/task/spac/BackfillRedemptionsTask.test.ts | Tests accession selection and dry-run behavior for the backfill task. |
| src/task/forms/ProcessAccessionDocFormTask.ts | Escalates fetch to full submission .txt for known‑SPAC redemption-trigger 8‑Ks; routes merger proxies to processMergerProxy; passes fullSubmissionText into 8‑K storage. |
| src/task/forms/ProcessAccessionDocFormTask.redemption.test.ts | Tests the 8‑K redemption full-submission fetch escalation behavior. |
| src/storage/versioning/extractorIds.ts | Registers new extractor IDs (merger-proxy, redemption) and maps merger proxy forms to merger-proxy. |
| src/storage/versioning/extractorIds.test.ts | Updates extractor ID list assertions and tests merger-proxy form mapping. |
| src/storage/versioning/componentRegistry.test.ts | Updates registered component count to include new extractors. |
| src/storage/spac/SpacReportWriter.ts | Adds writers for deal milestones, merger proxies, and redemptions; recomputes and reconciles derived deals. |
| src/storage/spac/SpacReportWriter.test.ts | Adds coverage for milestone roll-forward, idempotency, proxy correlation, and proxy event emission rules. |
| src/storage/spac/SpacRepo.ts | Adds getAllSpacs and deleteDeal to support backfill selection and derived-deal reconciliation. |
| src/storage/spac/SpacRedemptionExtractionSchema.ts | Introduces per-accession redemption extraction storage schema/token. |
| src/storage/spac/SpacRedemptionExtractionRepo.ts | Adds repo wrapper for redemption extractions. |
| src/storage/spac/SpacRedemptionExtractionRepo.test.ts | Tests redemption extraction repo behavior (overwrite, query by cik). |
| src/storage/spac/SpacMergerExtractionSchema.ts | Introduces per-accession merger-proxy extraction storage schema/token. |
| src/storage/spac/SpacMergerExtractionRepo.ts | Adds repo wrapper for merger-proxy extractions. |
| src/storage/spac/SpacMergerExtractionRepo.test.ts | Tests merger-proxy extraction repo behavior (overwrite, query by cik). |
| src/storage/spac/SpacEventSchema.ts | Updates lifecycle event vocabulary documentation to reflect new event sources. |
| src/storage/spac/spacDealGrouping.ts | Implements deterministic deriveDeals combining events + merger proxy extractions + redemption extractions. |
| src/storage/spac/spacDealGrouping.test.ts | Unit tests for deal derivation and merger-proxy correlation behavior. |
| src/storage/spac/spacDealGrouping.redemption.test.ts | Unit tests for redemption correlation windowing and precedence rules. |
| src/storage/spac/recomputeDeals.reconcile.test.ts | Tests reconciliation deleting orphan deal rows to prevent stale rollup totals. |
| src/sec/forms/registration-statements/s1/sectionRunner.ts | Adds parseConfidenceFloor and allows per-extractor confidence floors in makeRunSection. |
| src/sec/forms/registration-statements/s1/sectionRunner.test.ts | Tests parseConfidenceFloor and per-extractor confidenceFloor behavior. |
| src/sec/forms/registration-statements/s1/sectionExtractors.ts | Adds AI extractors for merger deal terms and redemption actuals. |
| src/sec/forms/registration-statements/s1/sectionExtractors.test.ts | Adds tests for extractMergerDeal behavior. |
| src/sec/forms/registration-statements/s1/redemptionSchema.ts | Defines structured schema for redemption extraction output. |
| src/sec/forms/registration-statements/s1/redemptionModel.ts | Adds model selection + confidence floor config for redemption extraction. |
| src/sec/forms/registration-statements/s1/redemptionModel.test.ts | Tests redemption model env overrides and confidence floor fallback. |
| src/sec/forms/registration-statements/s1/parseSubmission.ts | Adds parseEightKSubmission to slice primary + EX‑99 exhibits from full submissions. |
| src/sec/forms/registration-statements/s1/parseEightKSubmission.test.ts | Tests parseEightKSubmission behavior. |
| src/sec/forms/registration-statements/s1/mergerModel.ts | Adds model selection + confidence floor config for merger-proxy extraction. |
| src/sec/forms/registration-statements/s1/mergerModel.test.ts | Tests merger-proxy confidence floor env overrides and fallback. |
| src/sec/forms/registration-statements/s1/mergerDealSchema.ts | Defines structured schema for merger-proxy deal extraction output. |
| src/sec/forms/registration-statements/s1/extractRedemption.test.ts | Tests redemption extractor behavior (nulls, span requirement, schema validation). |
| src/sec/forms/registration-statements/s1/DocumentSegmenter.ts | Adds merger-proxy-specific section headings/patterns for segmentation. |
| src/sec/forms/proxies-information-statements/mock_data/merger-proxy/SOURCES.md | Documents merger-proxy fixtures provenance and intent. |
| src/sec/forms/proxies-information-statements/mock_data/merger-proxy/defm14a_sample.txt | Adds compact DEFM14A fixture for merger-proxy e2e plumbing tests. |
| src/sec/forms/proxies-information-statements/Form_PRER14A.ts | Uses parseRegistrationSubmission parsing for PRER14A. |
| src/sec/forms/proxies-information-statements/Form_PREM14C.ts | Uses parseRegistrationSubmission parsing for PREM14C. |
| src/sec/forms/proxies-information-statements/Form_PREM14A.ts | Uses parseRegistrationSubmission parsing for PREM14A. |
| src/sec/forms/proxies-information-statements/Form_DEFR14A.ts | Uses parseRegistrationSubmission parsing for DEFR14A. |
| src/sec/forms/proxies-information-statements/Form_DEFM14C.ts | Uses parseRegistrationSubmission parsing for DEFM14C. |
| src/sec/forms/proxies-information-statements/Form_DEFM14A.ts | Uses parseRegistrationSubmission parsing for DEFM14A. |
| src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.ts | Implements merger-proxy extraction pipeline, persistence, provenance, and deal recompute/rollup. |
| src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.e2e.test.ts | End-to-end tests for merger-proxy extraction/correlation across proxy form variants. |
| src/sec/forms/miscellaneous-filings/spac8kRedemptionTriggers.ts | Defines redemption-trigger 8‑K item codes and a helper matcher. |
| src/sec/forms/miscellaneous-filings/spac8kMilestones.ts | Maps key 8‑K item codes to SPAC lifecycle events. |
| src/sec/forms/miscellaneous-filings/spac8kMilestones.test.ts | Tests item-to-event mapping and 8‑K → SPAC milestone wiring/idempotency. |
| src/sec/forms/miscellaneous-filings/redemption8k.ts | Implements redemption extraction from full-submission 8‑Ks and deal correlation/rollup. |
| src/sec/forms/miscellaneous-filings/redemption8k.test.ts | Unit tests for redemption extraction gating and persistence/correlation. |
| src/sec/forms/miscellaneous-filings/redemption8k.e2e.test.ts | End-to-end tests for 8‑K redemption extraction and idempotent rollup. |
| src/sec/forms/miscellaneous-filings/Form_8_K.storage.ts | Wires SPAC milestone mapping and redemption extraction into 8‑K processing. |
| src/config/TestingDI.ts | Registers in-memory storages for new SPAC extraction tables for tests. |
| src/config/setupAllDatabases.ts | Adds setup calls for new SPAC extraction tables. |
| src/config/DefaultDI.ts | Registers persistent storages for new SPAC extraction tables. |
| src/commands/spac.ts | Adds sec spac backfill-redemptions CLI command. |
| src/cli/groups/version.test.ts | Updates CLI version tests to include new extractor IDs. |
| CLAUDE.md | Updates repo docs to describe new SPAC milestone/proxy/redemption extraction behavior and CLI usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- parseEightKSubmission.test.ts: use bun:test (repo convention; was the only file importing vitest). - sectionRunner parseConfidenceFloor: correct the JSDoc — a 0 floor admits every row (disables the floor); NaN is the drop-everything case. - BackfillRedemptionsTask: query filings by (form, cik) to use the ["form","cik"] index instead of loading all of a SPAC's filings and filtering. - CLAUDE.md: the milestone grouping function is deriveDeals, not the stale deriveDealsFromEvents reference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3vG1nXpbisZQuoduW7Z1f
sroussey
pushed a commit
that referenced
this pull request
Jun 24, 2026
Resolves conflicts created by PR #166 (SPAC de-SPAC lifecycle / merger-proxy / redemption extraction) landing on main after this PR opened. Conflicts resolved: - src/sec/forms/miscellaneous-filings/Form_8_K.storage.ts - Function signature combines both side's additive params: extractor_id, extractor_version (this PR), fullSubmissionText, model (#166). - Event writes go through replaceEvents() (this PR), threading extractor_id + extractor_version into the version-scoped delete-then-insert. - SPAC milestone mapping + redemption extraction blocks from #166 follow unchanged after the events are persisted. - src/task/forms/ProcessAccessionDocFormTask.ts - Keep TypeAccessionNumber import (this PR), processMergerProxy + hasRedemptionTriggerItem imports (#166). - 8-K dispatch call site passes both extractor_id/extractor_version and fullSubmissionText into processForm8K; merger-proxy case from #166 follows. - src/sec/forms/registration-statements/s1/sectionExtractors.ts (auto-merged cleanly by git but the new extractMergerDeal / extractRedemption functions still called the pre-PR wrapUntrusted shape + UNTRUSTED_PREAMBLE constant, which this PR removed. Both updated to the nonce-fence API (wrapUntrusted -> { wrapped, nonce }, buildUntrustedPreamble(nonce)) so the new SPAC AI extractors get the per-call nonce fence + multi-stage defang for free. Without this, the prompts would interpolate as "[object Object]" and the model receives garbage. Verification: - targeted: bun test src/sec/forms/miscellaneous-filings/ \ src/sec/forms/proxies-information-statements/ src/task/forms/ \ src/storage/spac/ src/storage/form-8k-event/ \ src/sec/forms/registration-statements/ -> 229 pass / 0 fail. - full: bun test -> 1410 pass / 7 fail. All 7 fails are pre-existing FetchDailyIndexTask + FetchQuarterlyIndexTask 5000ms network timeouts unrelated to this PR (sandbox can't reach SEC.gov reliably). - bun run build -> clean (bun build + tsc, no errors). Co-Authored-By: Claude <noreply@anthropic.com>
sroussey
pushed a commit
that referenced
this pull request
Jun 24, 2026
…e-fence API The new SPAC extractors added in PR #166 (extractMergerDeal, extractRedemption) called the pre-PR wrapUntrusted shape (returning a string) and the removed UNTRUSTED_PREAMBLE constant. After this PR swapped wrapUntrusted to return { wrapped, nonce } and replaced the constant with buildUntrustedPreamble(nonce), the surviving call sites template-interpolated UNTRUSTED_PREAMBLE as a free identifier -> compile error (TS2552), and even if the type had survived the { wrapped, nonce } object would have rendered as "[object Object]" in the prompt -> the model receives garbage and silently returns nothing (caught by Form_DEFM14A.storage.e2e.test.ts target_name=null assertions in the post-merge run). Both extractors now use the same nonce-fence + multi-stage defang as the other section extractors -- a forced consequence of the merge, extending the per-call nonce + entity decode + NFKC + zero-width strip protection to the new SPAC AI extractors at no extra design cost. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds out the SPAC de-SPAC lifecycle on top of the consolidated
spacreport, in three increments that all land here (none were previously onmain):1.01→ definitive_agreement,1.02→ terminated,2.01→ completed,5.07→ vote), grouped intospac_dealattempts via the event stream (idempotent replays, stabledeal_index).DEFM14A/PREM14A,DEFM14C/PREM14C,DEFR14A/PRER14A) intospac_merger_extraction, correlated onto the matching deal; definitive statements emit theproxymilestone. Per-extractor confidence floor (SEC_MERGER_PROXY_CONFIDENCE_FLOOR).total_redemption_amount.§4c detail (this session's work)
When a known SPAC files an 8-K carrying item
5.07/2.01/8.01, ingestion escalates the fetch to the full submission.txt(the existing S-1/DRS mechanism), reads the primary document +EX-99.xexhibits, andprocessRedemption8KAI-extracts realized redemptions into a per-accessionspac_redemption_extractionrow.deriveDealscorrelatesredemption_amount/redemption_sharesonto the matchingspac_dealusing an announcement-based window (upper bound = the next deal's announcement, ignoringoutcome_date, so a redemption reported at/after closing still attaches). The deal column is the sole sourcesumRedemptionstotals, so each redemption is counted exactly once (no event is emitted). Newredemptionextractor id (own version slot + dead-letters),SEC_REDEMPTION_MODEL/SEC_REDEMPTION_CONFIDENCE_FLOORconfig, and asec spac backfill-redemptionscommand that sweeps historical known-SPAC trigger 8-Ks off the bootstrappedfilingmetadata.Design specs & the §4c implementation plan live in the PRD repo under
docs/superpowers/specs/and.../plans/.Test Plan
bun test src/sec/forms/miscellaneous-filings/ src/sec/forms/registration-statements/s1/ src/storage/spac/ src/storage/versioning/ src/task/spac/ src/task/forms/→ 266 pass, 0 failbunx tsc --noEmit→ cleantotal_redemption_amountsingle)processForm8Kre-gate +processRedemption8Ktrigger/SPAC/deal checks)sec fetch form <cik> 8-Kandsec spac report <cik>🤖 Generated with Claude Code
Generated by Claude Code