chore(bench): add cbm-vs-graphify benchmark suite#628
Conversation
|
Huge thanks for opening this PR and for the work you put into it. The maintainer shop is currently full, so this may sit for a bit before it gets a proper review. We will come back to this as soon as possible with real feedback; I wanted to make sure it did not sit unacknowledged in the meantime. |
|
Thank you for putting this together — head-to-head benchmarking against graphify is genuinely useful signal. We're going to close this one, for a couple of practical reasons rather than the idea: the DCO check is failing (sign-off email mismatch), and the suite bakes in machine-specific state — hardcoded absolute paths ( |
Summary
codebase-memory (cbm) と graphify を比較するベンチマーク一式を
bench/配下に追加します。内容
build_oracle.py,generate_qbank.py,compare_layer1.py,measure_layer3.py,analyze_layer2.py,qa_workflow.js,headtohead.shoracle.json,layer2_full.json,qbank.jsonl,summary.jsonほか (bench/cbm-vs-graphify/results/)REPORT_cbm_vs_graphify.md,BASELINE.md,README.md19 files changed, 2015 insertions(+).
Notes
ドキュメント/ベンチマークの追加のみで、製品コードへの変更はありません。