feat(memory): adopt mimalloc allocator

.gitignore

@@ -32,6 +32,7 @@ vendor/
 coverage/
 .cache/
 tmp/
+.gradle/
 
 
 # Added by cargo

README.md

@@ -296,6 +296,7 @@ Adding a command means implementing it in Rust (`crates/aft/src/commands/`) and
 ---
 
 ## Documentation
+- [Architecture for new contributors](docs/architecture-for-contributors.md): a visual guide to the request path and main code areas
 
 - [Tool reference](docs/tools.md): complete documentation for every tool
 - [Configuration](docs/config.md): config schema, LSP, auto-install

STRUCTURE.md

@@ -73,10 +73,15 @@ opencode-aft/
 - Key files: `crates/aft/src/lsp/manager.rs`, `crates/aft/src/lsp/client.rs`, `crates/aft/src/lsp/diagnostics.rs`, `crates/aft/src/lsp/roots.rs`, `crates/aft/src/lsp/child_registry.rs`
 
 **`crates/aft/src/executor/`:**
-- Purpose: Orchestrate background maintenance, interactive tools, and job queues.
-- Contains: Actor scheduler, job classes and priority queues, worker thread loop, and cooperative cancellation tokens.
+- Purpose: Orchestrate bounded background maintenance and interactive tool queues across project-root actors.
+- Contains: Process-wide and per-actor capacity accounting, interactive and maintenance job classes, reader-first admission, deadline-aware writer promotion, queue-deadline pruning, deficit round-robin actor scheduling, worker lanes, dispatch telemetry, and cooperative cancellation tokens.
 - Key files: `crates/aft/src/executor/mod.rs`, `crates/aft/src/executor/tests.rs`
 
+**Standing-root scheduling and resource control:**
+- Purpose: Share cold-build slots fairly across standing roots without making a developer laptop unresponsive.
+- Contains: Process-wide deficit round-robin root scheduling, balanced and performance resource policies, pressure sampling with hysteresis, durable slice coordination, and cross-platform background thread priority control.
+- Key files: `crates/aft/src/standing_scheduler.rs`, `crates/aft/src/resource_policy.rs`, `crates/aft/src/subc/standing.rs`, `crates/aft/src/thread_priority.rs`
+
 **`crates/aft/src/bash_background/`:**
 - Purpose: Manage background bash tasks, PTY sessions, async pattern watches, and output compression.
 - Contains: Process pool, PTY runtime, watchdog thread, persistence, restart fate preservation (`FateUnknown`), process start-time liveness checks, buffer management, async pattern watches

benchmarks/allocator-daemon/README.md

@@ -0,0 +1,84 @@
+# AFT allocator daemon benchmark
+
+Compare the parent system-allocator build with the mimalloc build under the same long-lived SubC daemon workload.
+
+This benchmark is an evidence protocol. It does not contain accepted allocator results. Record results only after both arms run on the same host with the same repository roots and configuration.
+
+## Coverage boundary
+
+The mimalloc arm installs mimalloc through Rust `GlobalAlloc`. Rust-owned heap allocations use mimalloc. Native libraries can still allocate through the platform allocator. This includes SQLite, tree-sitter, ONNX Runtime, and other C or C++ dependencies unless their build explicitly routes `malloc` through mimalloc.
+
+The idle relief pass therefore covers both domains:
+
+- `mi_collect(true)` releases unused mimalloc pages.
+- `malloc_trim(0)` requests glibc native-heap relief on Linux.
+- `malloc_zone_pressure_relief(NULL, 0)` requests native-zone relief on macOS.
+
+Process RSS, macOS physical footprint, SQLite bytes, and subsystem estimates remain independent checks. Mimalloc statistics do not represent the full process.
+
+## Required arms
+
+| Arm | Build | Purpose |
+|---|---|---|
+| `system` | Parent commit of the mimalloc change | Baseline platform allocator behavior |
+| `mimalloc` | PR branch | Rust allocator change with dual-domain idle relief |
+
+Build both binaries from clean worktrees. Do not compare binaries with different AFT features or root-index code.
+
+## Required workload
+
+Use at least seven real Git roots. Include small, medium, and large roots. Use the same absolute root paths and selected search, semantic, and callgraph indexes for both arms.
+
+Run these phases in order:
+
+1. **Cold build**: Clear only AFT index storage. Start the isolated SubC daemon. Wait until every selected root artifact reaches a terminal state.
+2. **Steady serving**: Issue a fixed reader corpus at a fixed rate while the daemon remains bound. Include read, grep, glob, outline, and callgraph queries.
+3. **Idle eviction**: Close every route. Wait for the configured idle-root eviction boundary. Confirm that the daemon reports each root eviction.
+4. **Post-relief idle**: Keep the daemon alive for at least two allocator scan intervals. Do not submit new work.
+
+Use an isolated connection file, config root, data root, and log root for each arm. Never point this benchmark at the production SubC daemon.
+
+## Sampling
+
+Sample at five-second intervals. Record these columns:
+
+```text
+timestamp,arm,phase,pid,rss_bytes,phys_footprint_bytes,vm_swap_bytes,cpu_percent,thread_count,open_routes,live_actor_roots,allocator_slack_bytes,allocator_slack_measured,sqlite_bytes,total_attributed_bytes
+```
+
+Linux obtains RSS and swap from `/proc/<pid>/status`. macOS obtains RSS and physical footprint from `proc_pidinfo` and `proc_pid_rusage`, matching AFT's `memory.rs` implementation. Obtain allocator, SQLite, root, and route values from the existing SubC health memory and runtime rollups. Keep field names and byte units unchanged.
+
+Capture these events with timestamps:
+
+- daemon ready
+- each root artifact completion
+- steady-serving start and stop
+- each idle-root eviction
+- each allocator pressure-relief log
+- daemon shutdown
+
+## Controls
+
+- Use the same host without other build or indexing work.
+- Run the arms in alternating order across at least three pairs.
+- Reboot or allow the host to return to the same memory-pressure baseline before each pair.
+- Keep power mode, CPU governor, semantic backend, model cache, and root revisions fixed.
+- Preserve model downloads between arms. Clear generated AFT indexes between arms.
+- Exclude a pair when either arm has a root failure, daemon restart, transport timeout, or changed Git revision.
+
+## Report
+
+Report each pair separately and then report the median difference. Include:
+
+- peak RSS during cold build
+- peak macOS physical footprint during cold build
+- p50 and p99 reader latency during steady serving
+- artifact build completion time
+- RSS and physical footprint immediately before eviction
+- RSS and physical footprint after each relief pass
+- final RSS, physical footprint, and swap after post-relief idle
+- allocator slack, SQLite bytes, and attributed bytes at every phase boundary
+
+Do not use RSS alone on macOS. `MADV_FREE` can leave reclaimable pages visible in RSS after the allocator surrendered them. Physical footprint is the user-visible held-memory check for that platform.
+
+Do not claim that mimalloc reclaims native allocations from mimalloc statistics. Attribute a reduction to the combined relief pass unless a dedicated native-allocation experiment isolates the allocator domain.

crates/aft/Cargo.toml

@@ -32,6 +32,7 @@ crossbeam-channel = "0.5"
 parking_lot = "0.12"
 portable-pty = "0.9"
 libc = "0.2"
+mimalloc = { version = "0.1.52", features = ["extended"] }
 getrandom = "0.3"
 tree-sitter = "0.26"
 tree-sitter-typescript = "0.23.2"

crates/aft/src/callgraph_store/mod.rs

@@ -623,6 +623,7 @@ thread_local! {
         const { std::cell::RefCell::new(None) };
     static REFRESH_COMMIT_ADMISSION: std::cell::RefCell<Option<(SubcLifecycleAdmission, Arc<std::sync::atomic::AtomicU64>, u64)>> =
         const { std::cell::RefCell::new(None) };
+    static COLD_BUILD_SLICE_BUDGET: std::cell::Cell<Option<usize>> = const { std::cell::Cell::new(None) };
 }
 
 mod dead_code_projection;
@@ -711,6 +712,21 @@ impl Drop for PublishAdmissionGuard {
     }
 }
 
+struct ColdBuildSliceBudgetGuard {
+    previous: Option<usize>,
+}
+
+impl Drop for ColdBuildSliceBudgetGuard {
+    fn drop(&mut self) {
+        COLD_BUILD_SLICE_BUDGET.with(|slot| slot.set(self.previous));
+    }
+}
+
+fn with_cold_build_slice_budget<R>(budget: usize, run: impl FnOnce() -> R) -> R {
+    let previous = COLD_BUILD_SLICE_BUDGET.with(|slot| slot.replace(Some(budget.max(1))));
+    let _guard = ColdBuildSliceBudgetGuard { previous };
+    run()
+}
 pub(crate) fn with_publish_epoch<R>(
     epoch: crate::root_cache::ArtifactPublishEpoch,
     expected: u64,
@@ -724,16 +740,31 @@ pub(crate) fn with_publish_epoch<R>(
 fn ensure_cold_build_current(stage: &'static str, completed: usize, total: usize) -> Result<()> {
     notify_cold_build_slice_observer(stage, completed, total);
     let admission = PUBLISH_ADMISSION.with(|slot| slot.borrow().clone());
-    if admission.is_none_or(|(epoch, expected)| epoch.is_current(expected)) {
-        return Ok(());
+    if admission.is_some_and(|(epoch, expected)| !epoch.is_current(expected)) {
+        crate::slog_info!(
+            "callgraph cold build superseded, stopping after {}/{} ({})",
+            completed,
+            total,
+            stage
+        );
+        return Err(CallGraphStoreError::Superseded);
     }
-    crate::slog_info!(
-        "callgraph cold build superseded, stopping after {}/{} ({})",
-        completed,
-        total,
-        stage
-    );
-    Err(CallGraphStoreError::Superseded)
+    let exhausted = COLD_BUILD_SLICE_BUDGET.with(|slot| match slot.get() {
+        Some(remaining) if completed > 0 && remaining <= 1 => true,
+        Some(remaining) if completed > 0 => {
+            slot.set(Some(remaining - 1));
+            false
+        }
+        _ => false,
+    });
+    if exhausted {
+        return Err(CallGraphStoreError::SliceProgress {
+            phase: stage.to_string(),
+            completed,
+            total,
+        });
+    }
+    Ok(())
 }
 
 fn publish_if_current<R>(publish: impl FnOnce() -> Result<R>) -> Result<R> {
@@ -815,6 +846,11 @@ pub enum CallGraphStoreError {
     Suspended(crate::build_breaker::BuildSuspension),
     Superseded,
     StaleFiles(Vec<String>),
+    SliceProgress {
+        phase: String,
+        completed: usize,
+        total: usize,
+    },
 }
 
 impl CallGraphStoreError {
@@ -860,6 +896,14 @@ impl fmt::Display for CallGraphStoreError {
             Self::Superseded => {
                 write!(formatter, "callgraph store build superseded before publish")
             }
+            Self::SliceProgress {
+                phase,
+                completed,
+                total,
+            } => write!(
+                formatter,
+                "callgraph cold-build slice completed: {phase} {completed}/{total}"
+            ),
             Self::StaleFiles(files) => {
                 write!(
                     formatter,
@@ -1172,7 +1216,10 @@ impl RefreshWorker {
         let thread_shared = Arc::clone(&shared);
         let thread = std::thread::Builder::new()
             .name("aft-callgraph-refresh".to_string())
-            .spawn(move || callgraph_refresh_worker_loop(&thread_shared))
+            .spawn(move || {
+                crate::thread_priority::demote_background();
+                callgraph_refresh_worker_loop(&thread_shared)
+            })
             .expect("failed to spawn callgraph refresh worker");
         Arc::new(Self {
             shared,
@@ -1906,6 +1953,20 @@ pub struct IncrementalStats {
     pub unchanged_extract_files: usize,
 }
 
+#[derive(Debug)]
+pub enum ColdBuildSlice {
+    Progress {
+        phase: String,
+        completed: usize,
+        total: usize,
+    },
+    Complete {
+        store: CallGraphStore,
+        stats: ColdBuildStats,
+    },
+    Superseded,
+}
+
 /// Phase timings for the copy-based incremental refresh benchmark.
 #[doc(hidden)]
 #[derive(Debug, Clone, Default, PartialEq, Eq)]
@@ -2881,6 +2942,37 @@ impl CallGraphStore {
         )
     }
 
+    pub fn resume_cold_build_slice_with_lease(
+        callgraph_dir: PathBuf,
+        project_root: PathBuf,
+        files: &[PathBuf],
+        chunk_size: usize,
+    ) -> Result<ColdBuildSlice> {
+        let result = with_cold_build_slice_budget(1, || {
+            Self::cold_build_with_lease_chunked_inner(
+                callgraph_dir,
+                project_root,
+                files,
+                chunk_size,
+                false,
+            )
+        });
+        match result {
+            Ok((store, stats)) => Ok(ColdBuildSlice::Complete { store, stats }),
+            Err(CallGraphStoreError::SliceProgress {
+                phase,
+                completed,
+                total,
+            }) => Ok(ColdBuildSlice::Progress {
+                phase,
+                completed,
+                total,
+            }),
+            Err(CallGraphStoreError::Superseded) => Ok(ColdBuildSlice::Superseded),
+            Err(error) => Err(error),
+        }
+    }
+
     pub(crate) fn force_cold_build_with_lease_chunked(
         callgraph_dir: PathBuf,
         project_root: PathBuf,
@@ -3584,7 +3676,6 @@ impl CallGraphStore {
             &module_resolution_memo,
         )
     }
-
     #[cfg(test)]
     fn cold_build_chunked_with_resolution_memo_for_test(
         &self,
@@ -8489,6 +8580,11 @@ fn build_extracts_parallel(project_root: &Path, files: &[PathBuf]) -> BuildExtra
         .num_threads(build_pool_size())
         .thread_name(|index| format!("aft-callgraph-build-{index}"))
         .stack_size(8 * 1024 * 1024)
+        .start_handler(|_| {
+            // Callgraph builds are background maintenance: keep interactive
+            // reads ahead in the OS scheduler (CPU and I/O).
+            crate::thread_priority::demote_background();
+        })
         .build()
     {
         Ok(pool) => pool.install(run),