Skip to content

Tune outside-in partial rip across graph scales - #159

Draft
seveibar wants to merge 6 commits into
mainfrom
agent/adaptive-outside-in-partial-rip
Draft

Tune outside-in partial rip across graph scales#159
seveibar wants to merge 6 commits into
mainfrom
agent/adaptive-outside-in-partial-rip

Conversation

@seveibar

@seveibar seveibar commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • retain each affected trace's unchanged prefix and suffix, reopening only a bounded window around its hottest segment
  • route the reopened window outside-in from both retained ends, with a hard maximum travel distance per frontier and a safe one-ended fallback
  • provide an inclusive route-count window so callers can use partial ripping only at graph scales where it wins
  • preserve the legacy hot path when partial ripping and outside-in routing are disabled
  • choose completed states region-cost-first, allowing lower-complexity alternatives only inside explicit max-region and total-region-cost envelopes anchored to the first completed state
  • expose routing/rip/frontier/cost telemetry and keep a checked-in log of the accepted and rejected experiments

This supersedes #158. The paired integration is tscircuit/tscircuit-autorouter#2051.

Official package benchmark

The final /benchmark run on all eight committed SRJ18 Pipeline7 inputs reports:

  • 8/8 cases and 2,001/2,001 routes complete on both main and this PR
  • 107.536s total versus 256.486s on main: 2.39x faster
  • 4.877s P50 versus 29.878s on main: 6.13x faster median
  • average max region cost 1.713 versus 2.333 on main: 26.6% better
  • average iterations 429,535 versus 1,335,850 on main: 67.8% fewer

The package benchmark's P95 is 66.033s versus 47.871s because its 361-route sample is outside the reliable scale range. The autorouter integration therefore enables the strategy only for 100–350 routes; graphs below and above that window use the compatibility path.

Integration policy

  • fewer than 100 routes: compatibility behavior
  • 100–350 routes: one full-rip warmup, then bounded partial-rip attempts with region-cost-first selection
  • more than 350 routes: compatibility behavior
  • preloaded traces: partial ripping and outside-in routing disabled
  • complexity tie-breaking: capped relative to the first completed state at 20% max-region-cost growth and 10% total-region-cost growth

Verification

  • bun run typecheck
  • bun run build
  • focused outside-in, partial-rip, duplicate-topology, port-penalty, and route-completion suite: 22 passing tests
  • repository CI: test, format, typecheck, and deployment checks passing
  • git diff --check

Detailed trial commands, timings, route completion, and region-cost results are recorded in experiments/outside-in-partial-rip.md.

@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tiny-hypergraph Ready Ready Preview Aug 10, 2026 8:29am

Request Review

@tscircuitbot

Copy link
Copy Markdown

Benchmark This PR

Run benchmarks by commenting on this PR:

/benchmark [benchmark.sh args...]
/benchmark --dataset <hg07|srj18> --limit <n> --solver <core|poly> --families <default|default+deep|all> --concurrency <n|auto>

Everything after /benchmark is forwarded directly to ./benchmark.sh.
The result comment shows the matching dataset from the latest stored main-branch benchmark artifact beside the PR run; main is not rerun for each PR benchmark request.

Examples:

  • /benchmark -> all committed completed SRJ18 Pipeline7 cases, core solver
  • /benchmark --limit 4 -> first four committed Pipeline7 cases
  • /benchmark --dataset srj18 --concurrency auto -> srj18 samples with runner CPU-count concurrency
  • /benchmark --sample sample003 -> one committed SRJ18 Pipeline7 case
  • /benchmark --dataset hg07 --limit 20 --solver poly -> first 20 hg07 samples with the poly solver
  • /benchmark --dataset hg07 --limit 40 --families default+deep -> first 40 hg07 samples with deeper candidate families

Any PR whose title contains [BENCHMARK TEST] will automatically run the benchmark workflow on PR updates.

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot

tscircuitbot commented Aug 10, 2026

Copy link
Copy Markdown

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success Routes Final Cost Avg Hops P50 P95
Main srj18 core default 100.0% 100.0% 2.333 5.987 29.878s 47.871s
PR srj18 core default 100.0% 100.0% 1.713 5.794 4.686s 55.269s
Main Branch Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 256.486s
Avg baseline max region cost 2.333
Avg final max region cost 2.333
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 1335850.000
Avg route hops 5.987
Routes requiring rip 0
Avg solveGraph time 31.399s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 32.061s
P50 duration 29.878s
P95 duration 47.871s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.091 1.091 0.000 5.995 0 0 1391083 22.262s
sample003 success 145/145 6.353 6.353 0.000 7.055 0 0 862491 12.899s
sample005 success 172/172 1.492 1.492 0.000 5.959 0 0 990627 14.875s
sample007 success 263/263 0.527 0.527 0.000 5.464 0 0 1461654 43.149s
sample008 success 361/361 2.806 2.806 0.000 6.695 0 0 1503642 40.554s
sample009 success 256/256 1.492 1.492 0.000 5.777 0 0 935782 29.878s
sample010 success 310/310 1.686 1.686 0.000 5.406 0 0 1645055 44.999s
sample011 success 289/289 3.217 3.217 0.000 5.543 0 0 1896466 47.871s
PR Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 97.678s
Avg baseline max region cost 1.713
Avg final max region cost 1.713
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 429534.750
Avg route hops 5.794
Routes requiring rip 0
Avg solveGraph time 11.540s
Avg section search time 0.000s
Avg optimizeSection time 0.002s
Avg duration 12.210s
P50 duration 4.686s
P95 duration 55.269s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.440 1.440 0.000 5.693 0 0 221769 4.686s
sample003 success 145/145 1.543 1.543 0.000 6.910 0 0 108861 3.103s
sample005 success 172/172 0.766 0.766 0.000 5.744 0 0 132080 2.676s
sample007 success 263/263 0.542 0.542 0.000 5.148 0 0 308090 10.799s
sample008 success 361/361 3.183 3.183 0.000 6.778 0 0 2000004 55.269s
sample009 success 256/256 1.492 1.492 0.000 5.500 0 0 155144 4.463s
sample010 success 310/310 1.278 1.278 0.000 4.739 0 0 141861 5.076s
sample011 success 289/289 3.464 3.464 0.000 5.841 0 0 368469 11.605s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/31365052185

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot

tscircuitbot commented Aug 10, 2026

Copy link
Copy Markdown

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success Routes Final Cost Avg Hops P50 P95
Main srj18 core default 100.0% 100.0% 2.333 5.987 29.878s 47.871s
PR srj18 core default 100.0% 100.0% 1.713 5.794 4.610s 61.457s
Main Branch Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 256.486s
Avg baseline max region cost 2.333
Avg final max region cost 2.333
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 1335850.000
Avg route hops 5.987
Routes requiring rip 0
Avg solveGraph time 31.399s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 32.061s
P50 duration 29.878s
P95 duration 47.871s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.091 1.091 0.000 5.995 0 0 1391083 22.262s
sample003 success 145/145 6.353 6.353 0.000 7.055 0 0 862491 12.899s
sample005 success 172/172 1.492 1.492 0.000 5.959 0 0 990627 14.875s
sample007 success 263/263 0.527 0.527 0.000 5.464 0 0 1461654 43.149s
sample008 success 361/361 2.806 2.806 0.000 6.695 0 0 1503642 40.554s
sample009 success 256/256 1.492 1.492 0.000 5.777 0 0 935782 29.878s
sample010 success 310/310 1.686 1.686 0.000 5.406 0 0 1645055 44.999s
sample011 success 289/289 3.217 3.217 0.000 5.543 0 0 1896466 47.871s
PR Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 101.400s
Avg baseline max region cost 1.713
Avg final max region cost 1.713
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 429534.750
Avg route hops 5.794
Routes requiring rip 0
Avg solveGraph time 12.016s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 12.675s
P50 duration 4.610s
P95 duration 61.457s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.440 1.440 0.000 5.693 0 0 221769 4.610s
sample003 success 145/145 1.543 1.543 0.000 6.910 0 0 108861 3.159s
sample005 success 172/172 0.766 0.766 0.000 5.744 0 0 132080 2.853s
sample007 success 263/263 0.542 0.542 0.000 5.148 0 0 308090 10.526s
sample008 success 361/361 3.183 3.183 0.000 6.778 0 0 2000004 61.457s
sample009 success 256/256 1.492 1.492 0.000 5.500 0 0 155144 4.441s
sample010 success 310/310 1.278 1.278 0.000 4.739 0 0 141861 4.933s
sample011 success 289/289 3.464 3.464 0.000 5.841 0 0 368469 9.421s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/31365799539

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot

tscircuitbot commented Aug 10, 2026

Copy link
Copy Markdown

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success Routes Final Cost Avg Hops P50 P95
Main srj18 core default 100.0% 100.0% 2.333 5.987 29.878s 47.871s
PR srj18 core default 100.0% 100.0% 1.713 5.794 4.664s 61.787s
Main Branch Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 256.486s
Avg baseline max region cost 2.333
Avg final max region cost 2.333
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 1335850.000
Avg route hops 5.987
Routes requiring rip 0
Avg solveGraph time 31.399s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 32.061s
P50 duration 29.878s
P95 duration 47.871s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.091 1.091 0.000 5.995 0 0 1391083 22.262s
sample003 success 145/145 6.353 6.353 0.000 7.055 0 0 862491 12.899s
sample005 success 172/172 1.492 1.492 0.000 5.959 0 0 990627 14.875s
sample007 success 263/263 0.527 0.527 0.000 5.464 0 0 1461654 43.149s
sample008 success 361/361 2.806 2.806 0.000 6.695 0 0 1503642 40.554s
sample009 success 256/256 1.492 1.492 0.000 5.777 0 0 935782 29.878s
sample010 success 310/310 1.686 1.686 0.000 5.406 0 0 1645055 44.999s
sample011 success 289/289 3.217 3.217 0.000 5.543 0 0 1896466 47.871s
PR Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 101.177s
Avg baseline max region cost 1.713
Avg final max region cost 1.713
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 429534.750
Avg route hops 5.794
Routes requiring rip 0
Avg solveGraph time 12.020s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 12.647s
P50 duration 4.664s
P95 duration 61.787s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.440 1.440 0.000 5.693 0 0 221769 4.664s
sample003 success 145/145 1.543 1.543 0.000 6.910 0 0 108861 2.967s
sample005 success 172/172 0.766 0.766 0.000 5.744 0 0 132080 2.564s
sample007 success 263/263 0.542 0.542 0.000 5.148 0 0 308090 10.292s
sample008 success 361/361 3.183 3.183 0.000 6.778 0 0 2000004 61.787s
sample009 success 256/256 1.492 1.492 0.000 5.500 0 0 155144 4.435s
sample010 success 310/310 1.278 1.278 0.000 4.739 0 0 141861 4.885s
sample011 success 289/289 3.464 3.464 0.000 5.841 0 0 368469 9.584s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/31367023548

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot

tscircuitbot commented Aug 10, 2026

Copy link
Copy Markdown

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success Routes Final Cost Avg Hops P50 P95
Main srj18 core default 100.0% 100.0% 2.333 5.987 29.878s 47.871s
PR srj18 core default 100.0% 100.0% 1.713 5.794 4.877s 66.033s
Main Branch Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 256.486s
Avg baseline max region cost 2.333
Avg final max region cost 2.333
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 1335850.000
Avg route hops 5.987
Routes requiring rip 0
Avg solveGraph time 31.399s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 32.061s
P50 duration 29.878s
P95 duration 47.871s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.091 1.091 0.000 5.995 0 0 1391083 22.262s
sample003 success 145/145 6.353 6.353 0.000 7.055 0 0 862491 12.899s
sample005 success 172/172 1.492 1.492 0.000 5.959 0 0 990627 14.875s
sample007 success 263/263 0.527 0.527 0.000 5.464 0 0 1461654 43.149s
sample008 success 361/361 2.806 2.806 0.000 6.695 0 0 1503642 40.554s
sample009 success 256/256 1.492 1.492 0.000 5.777 0 0 935782 29.878s
sample010 success 310/310 1.686 1.686 0.000 5.406 0 0 1645055 44.999s
sample011 success 289/289 3.217 3.217 0.000 5.543 0 0 1896466 47.871s
PR Details
Metric Value
Dataset srj18
Source dataset-srj18 committed Pipeline7 tiny-hypergraph inputs
Revision source dataset-srj18@c0aad90256a95256fcac814f9f7da81a82a2fdea; tscircuit-autorouter@ac87ba2740c450c226ef84a5bbcef7fb31369297
Sample selection all cases completed in autorouter run 31336663867
Solver core
Families default
Samples 8
Success rate 100.0%
Solved-route completion rate 100.0%
Improved rate 0.0%
Zero final max region cost rate 0.0%
Total completion time 107.536s
Avg baseline max region cost 1.713
Avg final max region cost 1.713
Avg max region delta 0.000
Avg candidate count 0.000
Avg generated candidate count 0.000
Avg duplicate candidate count 0.000
Avg iterations 429534.750
Avg route hops 5.794
Routes requiring rip 0
Avg solveGraph time 12.727s
Avg section search time 0.000s
Avg optimizeSection time 0.001s
Avg duration 13.442s
P50 duration 4.877s
P95 duration 66.033s
Sample Status Routes Baseline Final Delta Hops Rips Attempts Iterations Duration Error
sample001 success 205/205 1.440 1.440 0.000 5.693 0 0 221769 4.877s
sample003 success 145/145 1.543 1.543 0.000 6.910 0 0 108861 3.019s
sample005 success 172/172 0.766 0.766 0.000 5.744 0 0 132080 2.916s
sample007 success 263/263 0.542 0.542 0.000 5.148 0 0 308090 10.488s
sample008 success 361/361 3.183 3.183 0.000 6.778 0 0 2000004 66.033s
sample009 success 256/256 1.492 1.492 0.000 5.500 0 0 155144 4.877s
sample010 success 310/310 1.278 1.278 0.000 4.739 0 0 141861 5.246s
sample011 success 289/289 3.464 3.464 0.000 5.841 0 0 368469 10.080s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/31370429106

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants