Observed during corpus-management validation (#5): right after an ingest job reaches ok, the Manage tab's node_count for that entry can show a lower count than what actually landed in kg_nodes (e.g. 96 vs the 140 nodes the job itself reported).
Two candidate explanations, need to determine which (could be both):
- Mutable-stream read lag —
kg_nodes writes have a known ~100-300ms propagation lag before being visible to a fresh table(...) read (see tests/test_ingest.py, _eventually helpers used throughout the test suite). GET /api/repos's node_count query may just be racing that lag if polled immediately after a job flips to ok.
- Counting bug — something in how
/api/repos derives node_count (e.g. the GROUP BY repo count query, or a key mismatch between the job's entry_key and the repo column) could be undercounting independent of timing.
Needs investigation: reproduce with a controlled ingest, then poll node_count at increasing delays after the job hits ok to see if it converges (lag) or stays wrong (bug).
Ref: PR #5
Observed during corpus-management validation (#5): right after an ingest job reaches
ok, the Manage tab'snode_countfor that entry can show a lower count than what actually landed inkg_nodes(e.g. 96 vs the 140 nodes the job itself reported).Two candidate explanations, need to determine which (could be both):
kg_nodeswrites have a known ~100-300ms propagation lag before being visible to a freshtable(...)read (seetests/test_ingest.py,_eventuallyhelpers used throughout the test suite).GET /api/repos'snode_countquery may just be racing that lag if polled immediately after a job flips took./api/reposderivesnode_count(e.g. theGROUP BY repocount query, or a key mismatch between the job'sentry_keyand therepocolumn) could be undercounting independent of timing.Needs investigation: reproduce with a controlled ingest, then poll
node_countat increasing delays after the job hitsokto see if it converges (lag) or stays wrong (bug).Ref: PR #5