Skip to content

perf(batch-lint): batch file processing to reduce memory usage#49

Closed
luojiyin1987 wants to merge 2 commits into
masterfrom
perf/batch-lint-memory
Closed

perf(batch-lint): batch file processing to reduce memory usage#49
luojiyin1987 wants to merge 2 commits into
masterfrom
perf/batch-lint-memory

Conversation

@luojiyin1987

Copy link
Copy Markdown
Contributor

Closes #48

Problem

Current batchLint reads all Markdown files into memory before processing, and each worker call handles only 1 file. For large repositories, this causes high memory usage and excessive IPC overhead.

Changes

  • Process files in batches of concurrency size (matching thread count)
  • Each batch reads files and sends them to a single worker call (1 IPC per batch instead of N)
  • Memory bounded by concurrency files at a time instead of all files
  • Add batch content list test for lint-worker

Verification

  • npm test: 26/26 passed
  • npm run build: successful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(batch-lint): batch file processing to reduce memory usage

1 participant