fix(changelog): the reconcile script leaves out upstream entries the fork removed

.claude/skills/codegraph-build-validation/SKILL.md

@@ -70,7 +70,7 @@ Every engine test file runs in a throwaway home dir (`__tests__/setup-home-sandb
 
 Working copies of the real corpora (e.g. `~/codegraph-corpora/linux`, multi-GB SQLite DBs) follow the scratch rule in `AGENTS.md`: off `/tmp`, under `~/cg-scratch/` or the corpus tree.
 
-Before pushing a branch meant for upstream, run `node <fork checkout>/scripts/pr-guard.mjs` from the branch: it fails when the branch adds a `bun.lock`, `yarn.lock` or `pnpm-lock.yaml` (upstream tracks only `package-lock.json`). To check what a change does to the graph, index a corpus with each build and compare them with `node scripts/index-metrics.mjs <before.db> <after.db>`: counts plus the call edges lost or gained, keyed by qualified name so shifted lines do not count. `CHANGELOG.md` merges with git's `union` driver (`.gitattributes`), so an upstream merge keeps both sides' bullets. Union misfiles, doubles and drops fork entries when the merge brings in an upstream release, so after every upstream merge run `node scripts/changelog-reconcile.mjs`. It rebuilds the file from the merge's parents and lists the fork entries that read like a newly released upstream entry; decide each one by hand, keeping only the fork-only part. `--check` exits 1 when the file differs from the rebuild.
+Before pushing a branch meant for upstream, run `node <fork checkout>/scripts/pr-guard.mjs` from the branch: it fails when the branch adds a `bun.lock`, `yarn.lock` or `pnpm-lock.yaml` (upstream tracks only `package-lock.json`). To check what a change does to the graph, index a corpus with each build and compare them with `node scripts/index-metrics.mjs <before.db> <after.db>`: counts plus the call edges lost or gained, keyed by qualified name so shifted lines do not count. `CHANGELOG.md` merges with git's `union` driver (`.gitattributes`), so an upstream merge keeps both sides' bullets. Union misfiles, doubles and drops fork entries when the merge brings in an upstream release, so after every upstream merge run `node scripts/changelog-reconcile.mjs`. It rebuilds the file from the merge's parents and lists the fork entries that read like a newly released upstream entry; decide each one by hand, keeping only the fork-only part. An entry in upstream's `[Unreleased]` that the merge base's `[Unreleased]` already had and the fork lacks is not added back; it prints as `left out` so you can check that the removal was deliberate. `--check` exits 1 when the file differs from the rebuild.
 
 A worker dispatched to fix reproduced resolution or extraction bugs gets [`worker-fix-brief.md`](worker-fix-brief.md) plus its task specifics.
 

CHANGELOG.md

@@ -572,11 +572,6 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 - A file named the way Google names its tests, like `wire_format_unittest.cc` in protocolbuffers/protobuf, `logging_unittest.cc` in google/glog or a Python `foo_unittest.py`, now counts as a test, as `wire_format_test.cc` already did. Before, it was taken for production code: search and `codegraph_explore` ranked it alongside the code it tests, `codegraph affected` never listed it, and calls in production code could link to a class or type a unittest declares for itself, as google/breakpad's `stack_frame_entries_.size()` did to a unittest's `StackHelper`. Re-index projects that have such files after upgrading.
 
-- `codegraph sync` and the file watcher now link a file to a module it imports under a name of its own when the module is added after the file was indexed, comes back after being deleted, or gets the export in a later edit: a default import like `import tagsController from './tag/tag.controller'`, a namespace import like `import * as api from './api'`, or an aliased one like `import { Component as GroupsPageWrapper } from './groups_page'`, in JavaScript, TypeScript, Vue, Svelte and Astro. Before, the import and every use of the name, such as a call, `new`, a base class, a JSX or template tag or an Express `app.use` mount, stayed unlinked until the importing file changed or the project was re-indexed, so the module looked unused and `codegraph affected` missed the files that use it. Re-index after upgrading to pick up links an earlier version missed.
-- `codegraph sync` and the file watcher now do the same for a Python import under a name of its own, like `from .models import User as UserModel`, `from . import helpers as h` or `from app.pkg import sub as subpkg`: when the module or package is added after the importing file was indexed, comes back after being deleted, or gets the imported name in a later edit, the import and every use of the alias, such as a call, a base class or the class passed as a value, now link to it. Before, they stayed unlinked until the importing file changed or the project was re-indexed. Re-index Python projects after upgrading to pick up links an earlier version missed.
-
-- `codegraph sync` and the file watcher now also link a Python import of a name a package re-exports, like `from fastapi import FastAPI` through `fastapi/__init__.py`, when that `__init__.py` is added after the importing files were indexed or gains the re-export in a later edit. Before, calls through the name stayed unlinked until each importing file changed or the project was re-indexed. Large projects gain the most: sync also used to skip, in any language, a module or package that more than 500 references were waiting for, and now restoring Django's `django/urls/__init__.py` relinks every file that imports `path` from it. Re-index Python projects after upgrading to pick up links an earlier version missed.
-
 - In C++, a call on a variable, parameter or member declared with template arguments nested inside its template arguments, like `autovector<std::pair<int, FileMetaData*>>` or `Striped<CacheAlignedWrapper<port::Mutex>>`, now links to the method of the class it is declared as. Before, the nested arguments hid that class: in RocksDB, `files_marked_for_compaction_.clear()` on such an `autovector` linked to an unrelated `CompactionInputFiles::clear` that only shares words with the variable's name, and `mutex_.Get(key)` on a `Striped` linked to nothing. Re-index C++ projects after upgrading.
 
 - In C++, a class written with attribute macros the parser can't read now keeps its members. That covers several macros between `class` and the class name, like protocolbuffers/protobuf's generated `class PROTOBUF_EXPORT PROTOBUF_FUTURE_ADD_EARLY_WARN_UNUSED Any final`, a macro with arguments there, like RocksDB's `struct ALIGN_AS(64U) HandleImpl`, or one before a template specialization, and macros on a member: in front of it (`PROTOBUF_FUTURE_ADD_EARLY_NODISCARD`), between a pointer and its name (`PROTOBUF_NONNULL`), after its parameters or its name (`ABSL_ATTRIBUTE_LIFETIME_BOUND`, google/leveldb's `LOCKS_EXCLUDED(mu_)` and `GUARDED_BY(mu_)`), or alone on a line above a comment ({fmt}'s `FMT_BEGIN_EXPORT`). Before, protobuf's generated message classes were missing and their members, like `GetDescriptor`, `GetReflection` and `unknown_fields`, were indexed as functions outside any class, and leveldb's annotated methods were indexed under the annotation's name, so calls to them linked to nothing. Re-index C++ projects after upgrading.
@@ -618,12 +613,6 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 - In JavaScript and TypeScript, a default import now links to what the module's `export default` statement exports, wherever the statement sits in the file. Before, a module that declares its default, like `export default function Vans() {…}` or `export default class Store {…}`, had its default import linked to the first function it exports instead: in a React Router data-router app, where a page exports its `loader` or `action` above the page component, `import Vans, { loader as vansLoader } from './pages/Vans'` made the route `/vans` and every use of `Vans` lead to the loader. A component file that also exports a styled or memoized component no longer hands its default import to that component, and an anonymous default, like `export default function () {…}` or `export default () => …`, is no longer linked to a function exported beside it or nested inside it. A module loaded with `require('./x').default` is read the same way. Re-index JavaScript and TypeScript projects after upgrading.
 
-- `codegraph sync` and the file watcher now update the files that import through a module that re-exports what it imports, like a package's `__init__.py` or an `index.ts` barrel, when that module is deleted or added, and in Python also the files that reach it through another package re-exporting it. Before, deleting `shop/__init__.py` left every `from shop import Order` linked to the class it re-exported, and deleting `components/index.ts` left every `import { Button } from './components'` linked, until each importing file changed or the project was re-indexed. When such a module came back, the imports stayed linked the way they were while it was missing, and in a large Python project an import of a module whose name is as common as `models` or `test`, like `from django.test import TestCase`, stayed unlinked. Python, JavaScript, TypeScript, Vue, Svelte and Astro.
-- In JavaScript and TypeScript, a file that re-exports a module, like an `index.ts` with `export * from './button'` or `export { Button } from './button'`, now counts as importing that module, the way an `import` statement does. `codegraph affected` and a module's dependents now include the barrels that re-export it; before, a barrel made only of `export *` lines showed no dependency at all. Re-index JavaScript and TypeScript projects after upgrading.
-- `codegraph sync` and the file watcher now also update the files that import through an `index.ts` re-exporting another barrel, like `export * from './dropdown-menu'`, when that inner barrel is deleted or added. Before, their imports stayed linked through the deleted barrel, or stayed linked the way they were while it was missing, until each importing file changed or the project was re-indexed.
-- An import in a Vue, Svelte or Astro component now links the component to the file it imports, as an import in a script does, and so does an import of a scoped package that a `tsconfig` path or a workspace package maps to a project file, like `import { debounce } from '@excalidraw/common'`. Before, only the names such an import binds were linked, so a barrel or a module imported for its side effects showed no dependency from those files in `codegraph affected`, and `codegraph sync` could not update them when a barrel they import through was deleted or added. `codegraph sync` and the file watcher also link such an import again, and one written as `.` or `..`, when the file it names comes back. Re-index JavaScript, TypeScript, Vue, Svelte and Astro projects after upgrading.
-- `codegraph sync` and the file watcher now move an import written without an extension, like `import { sizeOf } from './button'`, to a file added beside the one it found before when the new file comes first, such as `button.ts` next to `button.vue`. Before, the import and the names it binds stayed linked to `button.vue` until the importing file changed or the project was re-indexed. JavaScript, TypeScript, Vue, Svelte and Astro.
-
 - In Kotlin, a call on a typed receiver now links to an extension function declared in a package, like `value.shout()` or the infix `a join b` with `value: Foo` and `fun Foo.shout()` under `package demo`, including one declared in another file of the package. Before, it linked only when the file had no `package` line, so the usual Kotlin project got no edge. On JetBrains Exposed, 1,020 such calls now link (`Table.selectAll()` to `FieldSet.selectAll`, `transaction.throwUnsupportedException(…)`), and none that linked before is lost. A member of the receiver's class still wins over an extension of the same name.
 
 - In C++, a call on a variable, parameter or member declared through a `typedef` or `using` alias now links to the method of the type the alias names, with the alias looked up the way C++ does it: in the calling function, its class and the classes that class inherits from, then the namespaces around it. In google/leveldb, `table_.Insert(buf)` on a `Table table_;` declared next to `typedef SkipList<const char*, KeyComparator> Table;` now reaches `SkipList::Insert` instead of the unrelated `HandleTable::Insert`, and `Table::Iterator iter(&table_);` reaches `SkipList::Iterator`'s constructor. A class that only shares the alias's name is no longer taken for it, and when the aliased type has no such method, as with an alias of `std::vector` or of a template parameter, no method is guessed from the receiver's name. An alias of a type written from the global scope names that global type, also inside a namespace of the same name: protocolbuffers/protobuf's `using Arena = ::upb::Arena;` in `hpb::internal::backend::upb`, so hpb's `arena_.Fuse(…)` on a `backend::Arena` now reaches `upb::Arena::Fuse` instead of the calling method itself. Re-index C++ projects after upgrading.

__tests__/fork-tooling-scripts.test.ts

@@ -245,6 +245,21 @@ describe('changelog-reconcile', () => {
     expect(result.added).toEqual(['- New upstream fix.', '- New upstream feature.']);
   });
 
+  it("does not re-add an upstream entry the fork removed since the merge base, and still adds a new one", () => {
+    const base = log('\n### Fixes\n\n- Upstream fix the fork dropped.\n\n- Shared fix.\n', OLD);
+    const fork = log('\n### Fixes\n\n- Shared fix.\n\n- Fork fix.\n', OLD);
+    const upstream = log(
+      '\n### Fixes\n\n- Upstream fix the fork dropped.\n\n- Shared fix.\n\n- New upstream fix.\n',
+      OLD,
+    );
+    const result = reconcileChangelog({ fork, upstream, base });
+    expect(unreleasedOf(result.text)).toBe('\n### Fixes\n\n- Shared fix.\n\n- Fork fix.\n\n- New upstream fix.\n');
+    expect(result.added).toEqual(['- New upstream fix.']);
+    expect(result.omitted).toEqual(['- Upstream fix the fork dropped.']);
+    // Without the merge base nothing says the fork removed it, so it comes back.
+    expect(reconcileChangelog({ fork, upstream }).added).toContain('- Upstream fix the fork dropped.');
+  });
+
   it('leaves the file alone when upstream released nothing, and notes a released section the fork edited', () => {
     const fork = log('\n### Fixes\n\n- Fork fix.\n\n\n- Spaced fork fix.\n', OLD);
     expect(reconcileChangelog({ fork, upstream: log('', OLD), base: log('', OLD) })).toMatchObject({

scripts/changelog-reconcile.mjs

@@ -17,7 +17,8 @@
  *     carried unchanged from upstream's `[Unreleased]` at the merge base that
  *     upstream no longer lists there (released under new wording, or removed);
  *   - upstream's own `[Unreleased]` entries the fork lacks, under their `###`
- *     heading;
+ *     heading, except those the merge base's `[Unreleased]` already had: the
+ *     fork removed or reworded them, so they are listed as left out;
  *   - upstream's file, verbatim, from its newest release heading to the end.
  *
  * A remaining fork entry that reads like an entry upstream released since the
@@ -346,11 +347,19 @@ export function reconcileChangelog({ fork, upstream, base = null }) {
   const emptyBefore = new Set(f.body.map((_, i) => i).filter((i) => emptyHeadingAt(f.body, forkLevel, i)));
   dropEmptiedHeadings(body, origin, emptyBefore);
 
-  // Upstream's own Unreleased entries the fork lacks.
+  // Upstream's own Unreleased entries the fork lacks. One already in the base's
+  // Unreleased was there when the fork last merged, so the fork's absence is its
+  // own doing (removed, reworded or filed elsewhere): it stays out and is reported.
   const added = [];
+  const omitted = [];
   const have = new Set(entries(body).map((e) => e.key));
+  const inBase = new Set(base === null ? [] : entries(splitChangelog(base).body).map((e) => e.key));
   for (const e of entries(u.body)) {
     if (have.has(e.key)) continue;
+    if (inBase.has(e.key)) {
+      omitted.push(e.key);
+      continue;
+    }
     insertEntry(body, e.path, u.body.slice(e.start, e.end));
     have.add(e.key);
     added.push(e.key);
@@ -370,7 +379,7 @@ export function reconcileChangelog({ fork, upstream, base = null }) {
   const tailDiffers = start >= 0 && f.tail.join('\n') !== u.tail.slice(start).join('\n');
 
   const text = [...f.header, ...body, '', ...u.tail].join('\n');
-  return { text, dropped, added, review, tailDiffers, newReleases: [...newReleases].map(version) };
+  return { text, dropped, added, omitted, review, tailDiffers, newReleases: [...newReleases].map(version) };
 }
 
 function git(args) {
@@ -424,6 +433,10 @@ function main() {
   console.log(`dropped ${result.dropped.length}, added from upstream's [Unreleased] ${result.added.length}`);
   for (const d of result.dropped) console.log(`  dropped (${d.reason}): ${first(d.entry)}`);
   for (const a of result.added) console.log(`  added: ${first(a)}`);
+  if (result.omitted.length > 0) {
+    console.log(`left out ${result.omitted.length}: in upstream's [Unreleased] and the merge base's, but not the fork's`);
+    for (const o of result.omitted) console.log(`  left out: ${first(o)}`);
+  }
   if (result.tailDiffers) {
     console.log("note: the fork parent's release sections differ from upstream's; the rebuild keeps upstream's");
   }