feat(extract): add --memory-limit-mb so a budget overrun aborts cleanly instead of an OOM kill (#3011) by abhay-codes07 · Pull Request #3076 · Graphify-Labs/graphify

abhay-codes07 · 2026-08-25T08:57:06Z

Closes #3011.

The problem

graphify extract inside a memory-limited container can grow past the cgroup allowance and be OOM-killed. --max-workers bounds the AST pool, but the later JS/TS symbol-resolution passes retain source buffers and syntax trees for the whole corpus, and GRAPHIFY_REBUILD_MEMORY_LIMIT_MB only ever applied to hook/watch rebuilds — never to the extract CLI. The kill leaves no graphify-specific failure, no stable exit status, and whatever had been written behind (the reporter measured 4–6.5 GiB peaks and OOM kills in an 8 GiB pod).

The change

--memory-limit-mb N / GRAPHIFY_MEMORY_LIMIT_MB=N on extract, update, and the bare graphify <path> form (which re-enters extract):

Applies the cap with setrlimit (RLIMIT_AS; RLIMIT_DATA on macOS, whose allocator ignores RLIMIT_AS) to the CLI process and, through a pool initializer, to every extraction worker. Workers start fresh under spawn, so they read the cap from the environment; the flag is written back there so flag and env behave identically. An existing lower hard limit is never raised.
Stops demoting MemoryError. Two places used to swallow it: _safe_extract recorded it as a skipped file, and the pool's per-future handler warned and retried the file in-process — which would hit the same wall in the parent. Either way a run could finish and publish a graph silently missing whatever came after. Now the pool cancels its queued work and the error propagates as a typed MemoryBudgetExceeded (a MemoryError subclass, so existing handling still applies).
Fails cleanly. The CLI reports the configured limit, the phase, and the observed peak RSS, exits with status 3 (distinct from 1 = extraction failed, 2 = bad arguments, so a wrapper can tell "give it more memory" from "the corpus is broken" without parsing stderr), and writes no graph.json — the previous graph is untouched. --allow-partial does not apply here: the operator asked to be stopped.
Is honest about its limits. A malformed value is refused (exit 2) rather than silently running with no budget — a budget that quietly vanished is the failure this exists to prevent. On Windows, where there is no setrlimit, the CLI says the budget cannot be enforced and continues; it does not pretend.

The enforcement lives in a dependency-free graphify/memory_budget.py. watch._apply_resource_limits now delegates to it and honours the general variable too, with the hook-specific GRAPHIFY_REBUILD_MEMORY_LIMIT_MB keeping precedence on that path, so one setting covers every rebuild.

RLIMIT_AS bounds virtual address space, which over-approximates the RSS a cgroup accounts — the README says to set the budget somewhat below the container limit. It is a per-process cap rather than an aggregate tree budget; that is what the platform offers portably without a dependency, and it is the same semantic the existing hook limit has always had.

What it looks like

$ graphify extract . --memory-limit-mb 6144
[graphify extract] memory budget: 6144 MB (applies to this process and its extraction workers)
...
error: memory budget of 6144 MB exceeded during AST extraction of src/generated/api.ts (peak observed in this process: ~6210 MB)
  configured: 6144 MB (--memory-limit-mb / GRAPHIFY_MEMORY_LIMIT_MB); the previous graph.json, if any, was left untouched.
  Raise the budget, narrow the corpus (.graphifyignore, --exclude), or lower --max-workers to reduce peak usage.
  exit status 3

On Windows:

[graphify extract] warning: memory budget of 512 MB cannot be enforced on this platform (no setrlimit); continuing without one

Tests

tests/test_memory_budget.py — 29 tests: value parsing and the refused env value; the typed error; setrlimit really applied and really biting (a 2 GiB allocation under a 256 MB cap raises MemoryError, run in a subprocess so the test process is never capped) and a lower existing hard limit preserved; the pool initializer and its construction; watch._apply_resource_limits precedence; _safe_extract still swallowing ordinary failures but letting MemoryError through; sequential extraction stopping rather than finishing partial; a worker hitting the budget aborting the pool with the typed error instead of a "worker failed" warning; the CLI exiting 3 with no graph.json for both flag forms and the env var; bad values exiting 2; the unenforceable-platform warning; the ordinary #2445 failure path unchanged; and update taking the flag, exiting 3, and still rejecting unknown options.

With the wiring reverted and only the module kept, 12 of them fail; with it, 27 pass and the 2 setrlimit tests skip on Windows (they run on the Linux CI; I also ran the enforcement path under WSL: cap 256 MB → MemoryError on the 2 GiB allocation). The full suite matches the v8 baseline.

README: an env-table row and a command-reference line.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

graphify-labs

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) tested, no difference found (not proven).

Graphify review — findings

Adds an opt-in memory budget for extract and update via --memory-limit-mb / GRAPHIFY_MEMORY_LIMIT_MB, which caps the CLI process and its extraction workers with setrlimit so a runaway run aborts with exit status 3 and no partial graph.json instead of being OOM-killed. _arm_memory_budget resolves the flag (which wins over the env var and is written back so spawn workers and nested rebuilds inherit it), rejects a malformed value with exit 2, and warns once and continues where the platform can't enforce a limit. MemoryError now propagates through _safe_extract and pool workers rather than being logged as a skipped file, ensuring the graph is never silently published missing whatever came after the budget was hit.

Worth a look

Memory budget flag races through process-global environment — graphify/cli.py:613 · Escalate · medium
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Process pool abort still waits for running workers after budget hit — graphify/extract.py:5578 · Escalate · medium
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
ThreadPool substitution runs the process initializer in the pytest process — tests/test_memory_budget.py:227 · Escalate · medium
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1885 functions depend on the 421 functions this change touches.

Health — this change adds coupling hotspots:

new: extract() — 494 callers, 42 callees
new: _rebuild_code() — 98 callers, 50 callees
new: extract_xaml() — 19 callers, 17 callees
new: dispatch_command() — 2 callers, 127 callees
new: extract_js() — 80 callers, 3 callees
new: _get_extractor() — 26 callers, 6 callees
new: run_pipeline() — 8 callers, 13 callees
new: collect_files() — 17 callers, 6 callees
…and 27 more — each is listed as a finding

Verification — 1885 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

verification_scope: 1838 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_extract\_parallel.

The verifier did not have enough to check \_extract\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_safe\_extract.

The verifier did not have enough to check \_safe\_extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `extractor` is annotated `Callable` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_apply\_resource\_limits (not a proof).

The verifier ran both versions of \_apply\_resource\_limits on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 2 grounded finding(s) anchored inline below; 33 more finding(s) on lines outside this diff (see the check run).

graphify-labs · 2026-08-25T09:07:03Z

+    apply_memory_budget()
+
+
 def _extract_single_file(args: tuple) -> tuple[int, dict]:


⚠️ Health regression — _extract_single_file()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

graphify-labs · 2026-08-25T09:07:03Z

+    return ru / 1024.0 if sys.platform != "darwin" else ru / (1024.0 * 1024.0)
+
+
+def budget_error(exc: BaseException, *, phase: str) -> MemoryBudgetExceeded:


⚠️ Health regression — budget_error()

6 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

shaulga · 2026-09-08T05:38:35Z

@abhay-codes07 thanks for taking this fix forward. do you know what else is needed to push it ?

…ad of an OOM kill (Graphify-Labs#3011) `graphify extract` inside a memory-limited container could grow past the cgroup allowance and be OOM-killed: --max-workers bounds the AST pool, but the later JS/TS resolution passes retain source buffers and syntax trees for the whole corpus, and GRAPHIFY_REBUILD_MEMORY_LIMIT_MB only ever applied to hook/watch rebuilds. The kill left no graphify-specific failure, no stable exit status, and whatever had been written behind. `--memory-limit-mb N` / `GRAPHIFY_MEMORY_LIMIT_MB=N` on `extract`, `update` and the bare `graphify <path>` form: * caps the CLI process with setrlimit (RLIMIT_AS; RLIMIT_DATA on macOS) and, through a pool initializer, every extraction worker - workers start fresh under `spawn`, so they read the cap from the environment; * lets MemoryError propagate where the pipeline used to demote it: _safe_extract recorded it as a skipped file, and the pool's per-future handler warned and retried the file in-process, which would hit the same wall - either way a run could finish and publish a graph silently missing whatever came after; * reports the configured limit, the phase, and the observed peak, exits with status 3 (distinct from 1 = extraction failed, 2 = bad arguments) and writes no graph.json - the previous graph is left untouched; * refuses a malformed value (exit 2) rather than silently running with no budget, and on Windows says the budget cannot be enforced and continues, rather than pretending. The enforcement lives in a dependency-free graphify.memory_budget; watch._apply_resource_limits now delegates to it and also honours the general variable, with the hook-specific one keeping precedence.

…oesn't wait Review follow-up: the MemoryError handler called shutdown(wait=False, cancel_futures=True), but raising out of the enclosing 'with ProcessPoolExecutor as pool' block triggers the executor's __exit__, which calls shutdown(wait=True) and blocks on the other running workers — each potentially churning the very memory the budget tripped on. Terminate the live workers before re-raising so the implicit exit-shutdown returns at once and the budget error surfaces promptly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q

abhay-codes07 · 2026-09-12T17:44:38Z

Worked through all three (rebased onto 0.9.59):

Pool abort waited for running workers (real, fixed). Correct — although the handler calls shutdown(wait=False, cancel_futures=True), it re-raises out of the with ProcessPoolExecutor() as pool block, so the executor's __exit__ then calls shutdown(wait=True) and blocks on the other in-flight workers — each potentially churning the memory the budget just tripped on, the opposite of a prompt abort. The handler now terminates the live workers before re-raising, so the implicit exit-shutdown finds them gone and returns at once.

Flag races through process-global environment. _arm_memory_budget runs once at CLI dispatch in the main process, before any pool exists; writing GRAPHIFY_MEMORY_LIMIT_MB to the environment is the intended mechanism for the spawn-based workers (and nested rebuilds) to inherit the cap — spawn workers don't inherit Python globals, only the environment. There's no concurrent second extract in the same process to race it. Left as-is by design.

ThreadPool substitution runs the initializer in the pytest process. Fair observation about the harness: the thread-pool substitution exercises the budget-application and abort-propagation logic without spawning real subprocesses (so it runs deterministically in CI), which is deliberate — the process-pool path itself is covered by the worker-init and BrokenProcessPool tests. Kept.

graphify-labs

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) tested, no difference found (not proven).

Graphify review — findings

Adds an opt-in memory budget for graphify extract and graphify update, configurable via --memory-limit-mb or GRAPHIFY_MEMORY_LIMIT_MB, that caps the CLI process and its extraction workers via setrlimit. When the cap is exceeded, the run aborts with exit status 3 and never publishes a partial graph.json (even with --allow-partial), since _safe_extract and the pool initializer re-raise MemoryError rather than treating it as a skipped file. On platforms without setrlimit (Windows) it warns once and continues unenforced, and a malformed env/flag value is refused with exit 2.

Worth a look

Memory budget abort path masks the intended error after shutdown clears worker list — graphify/extract.py:6258 · Escalate · high
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
MemoryError abort path can be masked by AttributeError after executor shutdown — graphify/extract.py:6262 · Escalate · high
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
MemoryError abort path reads ProcessPoolExecutor._processes after shutdown clears it — graphify/extract.py:6264 · Escalate · high
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Global environment mutation in extract leaks across concurrent/subsequent runs — graphify/extract.py · Escalate · medium
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
apply_memory_budget violates its never-raises contract — graphify/memory_budget.py:125 · Escalate · medium
- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2181 functions depend on the 460 functions this change touches.

Health — this change adds coupling hotspots:

new: extract() — 589 callers, 44 callees
new: _rebuild_code() — 115 callers, 51 callees
new: extract_js() — 85 callers, 4 callees
new: extract_xaml() — 19 callers, 17 callees
new: dispatch_command() — 2 callers, 129 callees
new: _get_extractor() — 26 callers, 6 callees
new: run_pipeline() — 8 callers, 13 callees
new: collect_files() — 17 callers, 6 callees
…and 33 more — each is listed as a finding

Verification — 2181 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

verification_scope: 2119 function(s) in the blast radius were not formally verified this run

Test selection

274 of 274 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.

tests/test_affected_cli.py — impact, full-run-safety
tests/test_affected_member_seed.py — full-run-safety
tests/test_agents_platform.py — impact, full-run-safety
tests/test_analyze.py — full-run-safety
tests/test_anthropic_custom_endpoint.py — full-run-safety
tests/test_antigravity_install.py — full-run-safety
tests/test_apm_fallback_version.py — full-run-safety
tests/test_architecture_doc.py — full-run-safety
tests/test_astro_extraction.py — impact, full-run-safety
tests/test_astro_import_ids.py — impact, full-run-safety
tests/test_atomic_canvas_export.py — full-run-safety
tests/test_atomic_version_stamp.py — full-run-safety
tests/test_atomic_writes.py — full-run-safety
tests/test_backend_env_isolation.py — full-run-safety
tests/test_backend_extras.py — full-run-safety
tests/test_benchmark.py — full-run-safety
tests/test_benchmark_raw_graph.py — full-run-safety
tests/test_build.py — impact, full-run-safety
tests/test_build_merge_dedup_scope.py — full-run-safety
tests/test_build_merge_hyperedges_and_prune.py — full-run-safety
tests/test_build_merge_shrink_guard.py — full-run-safety
tests/test_builtin_global_type_refs.py — impact, full-run-safety
tests/test_cache.py — full-run-safety
tests/test_callflow_html.py — full-run-safety
tests/test_cargo_introspect.py — full-run-safety
tests/test_carried_hyperedge_remap.py — full-run-safety
tests/test_case_sensitive_resolution.py — impact, full-run-safety
tests/test_charmap_encoding.py — full-run-safety
tests/test_chunking.py — full-run-safety
tests/test_cjs_module_extension.py — impact, full-run-safety
tests/test_claude_cli_backend.py — full-run-safety
tests/test_claude_md.py — full-run-safety
tests/test_cli_broken_pipe.py — full-run-safety
tests/test_cli_export.py — full-run-safety
tests/test_cli_help.py — full-run-safety
tests/test_cluster.py — full-run-safety
tests/test_codebuddy.py — impact, full-run-safety
tests/test_community_hub_labels.py — full-run-safety
tests/test_community_labels_skill.py — full-run-safety
tests/test_confidence.py — full-run-safety
tests/test_corrupt_graph_json.py — full-run-safety
tests/test_cpp_nested_and_cli.py — impact, full-run-safety
tests/test_cpp_objc_cross_file_calls.py — impact, full-run-safety
tests/test_cpp_preprocess.py — full-run-safety
tests/test_cross_extension_reexport_self_cycle.py — impact, full-run-safety
tests/test_cross_language_call_resolution.py — impact, full-run-safety
tests/test_cross_repo_member_calls.py — impact, full-run-safety
tests/test_cross_repo_shared_types.py — full-run-safety
tests/test_csharp_call_site_generic_args.py — impact, full-run-safety
tests/test_csharp_enum_members.py — impact, full-run-safety
… and 224 more

non-code file(s) changed (README.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (README.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_extract\_parallel.

The verifier did not have enough to check \_extract\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_safe\_extract.

The verifier did not have enough to check \_safe\_extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `extractor` is annotated `Callable` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_apply\_resource\_limits (not a proof).

The verifier ran both versions of \_apply\_resource\_limits on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 2 grounded finding(s) anchored inline below; 39 more finding(s) on lines outside this diff (see the check run).

graphify-labs · 2026-09-12T17:49:32Z

+    apply_memory_budget()
+
+
 def _extract_single_file(args: tuple) -> tuple[int, dict]:


⚠️ Health regression — _extract_single_file()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

graphify-labs · 2026-09-12T17:49:32Z

+    return ru / 1024.0 if sys.platform != "darwin" else ru / (1024.0 * 1024.0)
+
+
+def budget_error(exc: BaseException, *, phase: str) -> MemoryBudgetExceeded:


⚠️ Health regression — budget_error()

6 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Copilot AI lite review requested due to automatic review settings August 25, 2026 08:57

Copilot AI reviewed Aug 25, 2026

graphify-labs Bot reviewed Aug 25, 2026

View reviewed changes

abhay-codes07 and others added 2 commits September 12, 2026 23:12

abhay-codes07 force-pushed the feat/extract-memory-budget branch from 862e43f to e5310de Compare September 12, 2026 17:44

graphify-labs Bot reviewed Sep 12, 2026

View reviewed changes

Navigation Menu

Uh oh!

Uh oh!

feat(extract): add --memory-limit-mb so a budget overrun aborts cleanly instead of an OOM kill (#3011) - #3076

feat(extract): add --memory-limit-mb so a budget overrun aborts cleanly instead of an OOM kill (#3011)#3076
abhay-codes07 wants to merge 2 commits into
Graphify-Labs:v8from
abhay-codes07:feat/extract-memory-budget

abhay-codes07 commented Aug 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

graphify-labs Bot left a comment •

edited

Loading

Uh oh!

graphify-labs Bot Aug 25, 2026

Uh oh!

graphify-labs Bot Aug 25, 2026

Uh oh!

shaulga commented Sep 8, 2026

Uh oh!

abhay-codes07 commented Sep 12, 2026

Uh oh!

graphify-labs Bot left a comment •

edited

Loading

Uh oh!

graphify-labs Bot Sep 12, 2026

Uh oh!

graphify-labs Bot Sep 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		apply_memory_budget()


		def _extract_single_file(args: tuple) -> tuple[int, dict]:

		return ru / 1024.0 if sys.platform != "darwin" else ru / (1024.0 * 1024.0)


		def budget_error(exc: BaseException, *, phase: str) -> MemoryBudgetExceeded:

Uh oh!

Uh oh!

Conversation

abhay-codes07 commented Aug 25, 2026

The problem

The change

What it looks like

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

graphify-labs Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Graphify review — findings

Impact & health

Graphify review

Gate & verification

graphify gate

Formal verification

Uh oh!

graphify-labs Bot Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

graphify-labs Bot Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

shaulga commented Sep 8, 2026

Uh oh!

abhay-codes07 commented Sep 12, 2026

Uh oh!

graphify-labs Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Graphify review — findings

Impact & health

Graphify review

Gate & verification

graphify gate

Test selection

Test selection

Formal verification

Uh oh!

graphify-labs Bot Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

graphify-labs Bot Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

graphify-labs Bot left a comment •

edited

Loading

graphify-labs Bot left a comment •

edited

Loading