fix(explore): a symbol written with its package resolves, like refresh.Discovery.refresh

CHANGELOG.md

@@ -198,6 +198,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In JavaScript, an import of a TypeScript module written without an extension, like `import AudioModal from './audio_modal'` in a `.jsx` file next to `audio_modal.tsx`, now links to that module, as it already did from a TypeScript file. Before, a `.js` or `.jsx` file's imports of TypeScript modules were never followed, so in a codebase moving to TypeScript they were matched by name instead: an import under another name linked nothing, and a name that repeats could link to the wrong file, so mastodon's `<IconButton>` and `<Status>` tags rendered other components of those names than the ones each file imports. A JavaScript module beside a TypeScript one of the same name still comes first. Re-index JavaScript projects after upgrading.
 - `codegraph_explore`'s Flow now goes through the implementation your query names when an interface method has several that lead to the same place. Before, it took whichever one came first in the index: asking about prometheus's `Engine.execEvalStmt Queryable.Querier fanout.Querier NewMergeQuerier` traced the call through the TSDB's `DB.Querier` and left the `fanout.Querier` you named off the Flow. Any two routes of the same length are now settled this way, in favor of the one that passes through more of the symbols you named.
 - `codegraph_explore`'s "Dynamic-dispatch links among your symbols" list now shows every link on its own line. Before, the links from an interface method to its implementations all read alike, like `TrySet → TrySet`, so only one was listed, chosen by the order of the index rather than by your query: asking about gin's `tryToSetValue setter.TrySet multipartRequest.TrySet setByForm` showed `headerSource`'s implementation and hid `formSource`'s. Both ends of such a link are now named with their type, like `setter.TrySet → formSource.TrySet`. When there are more links than the list shows, the implementations your query names come first and those in tests or examples last, and a hop the Flow already shows is not repeated.
+- `codegraph_explore`, `codegraph_node`, `codegraph_callers`, `codegraph_callees`, `codegraph_impact` and the `codegraph callers`, `callees` and `impact` commands now find a symbol written with its package, the way Go code names it: `refresh.Discovery.refresh`, `storage.Queryable.Querier`, or `gin.Context.Next` in gin's root package. Before, such a name found nothing, so a question written that way lost the symbol and explore's Flow had nothing to connect. A name the search ranked below other results, like gin's `binding.Default`, is found as well, and a Python or Rust module followed by a class and method works the same way.
 - In C and C++, a struct, class, union or enum defined in a variable declaration, like `struct Foo { … } foo;`, is now indexed with its methods and enum values, next to the variable. Before, at most the variable was, so the type was missing from search and `codegraph_explore`, a class deriving from it had no base, and calls to its methods linked to nothing: in redis, the Sentinel's `sentinelState` and `redis-cli`'s `config` struct were missing. A struct, union or enum with no name of its own now takes the name of the variable it declares, so redis's `static struct { … } SPT;` is found as `SPT`, and one declared inside a function is named after its variable instead of `<anonymous>`. The comment above the declaration becomes the type's description. Re-index C and C++ projects after upgrading.
 - In Objective-C, a struct, union or enum defined in a variable declaration, like `static struct { Class isa; } placeholder;` or `enum { stateType, stateParamName } state;`, is now indexed with its enum values, next to the variable, as in C and C++. Before, only the variable was, so the type was missing from search and `codegraph_explore`. The same went for a `typedef enum { … } Name;` or `typedef struct … Name;` written after a line holding only a macro, like `OF_ASSUME_NONNULL_BEGIN` or `RCT_EXTERN_C_BEGIN`: ObjFW's `OFZIPArchiveEntryCompressionMethod` and every value in it were missing, and so was React Native's `RCTMethodInfo`. A struct, union or enum with no name of its own takes the name of the variable it declares, so ObjFW's class-cluster placeholders are found as `placeholder`, one declared inside a function or method is named after its variable instead of `<anonymous>`, and the comment above the declaration becomes the type's description. As in C, a type that a `.m` file defines links only from that file, so a call elsewhere that just shares its name, like ObjFW's calls of AmigaOS's `Lock(path, mode)`, doesn't link to the runtime's private `struct Lock`. Re-index Objective-C projects after upgrading.
 - In Objective-C, a macro written alone on its line, like `NS_ASSUME_NONNULL_BEGIN`, `OF_SUBCLASSING_RESTRICTED` or React Native's `RCT_EXTERN_C_BEGIN`, no longer hides the declaration below it. Before, the parser read such a macro as the start of a declaration, so an `@interface` under it could come out as a function and a `typedef` under it as two variables: many of ObjFW's classes, with their properties and methods and the classes they extend and protocols they adopt, were missing, and so were classes and protocols in React Native and KSCrash headers. A message to `super` from a class whose `@interface` follows an attribute macro like `NS_SWIFT_NAME(…)` still reaches the superclass. Re-index Objective-C projects after upgrading.

__tests__/symbol-lookup-go-packages.test.ts

@@ -0,0 +1,169 @@
+/**
+ * Regression: a symbol written with its package resolves.
+ *
+ * Explore's token resolution (and codegraph_node's lookup) dropped every Go
+ * symbol written the way Go code names it: `refresh.Discovery.refresh`,
+ * `storage.Queryable.Querier` and `gin.Context.Next` resolved to nothing, and so
+ * did `binding.Default`. Three gaps, each pinned below:
+ *   - a qualified name was matched either against the qualifiedName (which holds
+ *     the type, not the package) or entirely against the path (which holds the
+ *     package, not the type), never as a package followed by a type;
+ *   - a module's root package has no directory, so its name is only in the
+ *     files' `package` clause (gin's `context.go` is `package gin`);
+ *   - a qualified token was looked up in the full-text search's top hits only,
+ *     and for `binding.Default` those were three `binding/default_validator*.go`
+ *     files, not the function; the exact-name index is complete.
+ */
+import { describe, it, expect, afterAll, beforeAll } from 'vitest';
+import * as fs from 'node:fs';
+import * as path from 'node:path';
+import * as os from 'node:os';
+import CodeGraph from '../src/index';
+import type { Node } from '../src/types';
+import { matchesSymbol, lookupSymbolNodes } from '../src/graph/symbol-lookup';
+import { goPackageClause, goPackageReaderFor } from '../src/graph/go-package';
+import { findAllSymbols, resolveNamedTokens } from '../src/graph/named-symbol-flow';
+import { ToolHandler } from '../src/mcp/tools';
+
+function node(over: Partial<Node>): Node {
+  return {
+    id: `${over.kind ?? 'method'}:${over.filePath}:${over.qualifiedName}`,
+    kind: 'method',
+    name: 'x',
+    qualifiedName: 'x',
+    filePath: 'x.go',
+    language: 'go',
+    startLine: 1,
+    endLine: 1,
+    startColumn: 0,
+    endColumn: 0,
+    updatedAt: 0,
+    ...over,
+  } as Node;
+}
+
+describe('matchesSymbol — a package, then the scope inside it', () => {
+  const refresh = node({ name: 'refresh', qualifiedName: 'Discovery::refresh', filePath: 'discovery/refresh/refresh.go' });
+  const uyuni = node({ name: 'refresh', qualifiedName: 'Discovery::refresh', filePath: 'discovery/uyuni/uyuni.go' });
+
+  it('reads `pkg.Type.method` as the package in the path and the type in the scope', () => {
+    expect(matchesSymbol(refresh, 'refresh.Discovery.refresh')).toBe(true);
+    expect(matchesSymbol(uyuni, 'uyuni.Discovery.refresh')).toBe(true);
+    // Every leading qualifier names the package — `discovery/refresh`.
+    expect(matchesSymbol(refresh, 'discovery.refresh.Discovery.refresh')).toBe(true);
+  });
+
+  it('keeps the package: the same type in another package is not it', () => {
+    expect(matchesSymbol(uyuni, 'refresh.Discovery.refresh')).toBe(false);
+    expect(matchesSymbol(refresh, 'uyuni.Discovery.refresh')).toBe(false);
+    expect(matchesSymbol(refresh, 'nope.Discovery.refresh')).toBe(false);
+  });
+
+  it('keeps the type: the scope must end with what the query writes', () => {
+    expect(matchesSymbol(refresh, 'refresh.Other.refresh')).toBe(false);
+    // A package can't stand in for the type.
+    expect(matchesSymbol(refresh, 'Discovery.refresh.refresh')).toBe(false);
+  });
+
+  it("names a root package by the file's `package` clause", () => {
+    const next = node({ name: 'Next', qualifiedName: 'Context::Next', filePath: 'context.go' });
+    const fn = node({ kind: 'function', name: 'Default', qualifiedName: 'Default', filePath: 'utils.go' });
+    const gin = (p: string) => (p === 'context.go' || p === 'utils.go' ? 'gin' : null);
+    expect(matchesSymbol(next, 'gin.Context.Next', gin)).toBe(true);
+    expect(matchesSymbol(fn, 'gin.Default', gin)).toBe(true);
+    // Without the clause the path has no `gin`; another package name is not it.
+    expect(matchesSymbol(next, 'gin.Context.Next')).toBe(false);
+    expect(matchesSymbol(next, 'echo.Context.Next', gin)).toBe(false);
+  });
+
+  it('reads the package clause for Go files only', () => {
+    const ts = node({ name: 'Next', qualifiedName: 'Context::Next', filePath: 'context.ts', language: 'typescript' });
+    expect(matchesSymbol(ts, 'gin.Context.Next', () => 'gin')).toBe(false);
+  });
+
+  it('reads a Python `pkg.mod.Class.method` the same way', () => {
+    const save = node({ name: 'save', qualifiedName: 'Model::save', filePath: 'app/models.py', language: 'python' });
+    expect(matchesSymbol(save, 'app.models.Model.save')).toBe(true);
+    expect(matchesSymbol(save, 'other.models.Model.save')).toBe(false);
+  });
+});
+
+describe('goPackageClause', () => {
+  it('reads the clause past comments and build constraints', () => {
+    expect(goPackageClause('// Copyright\n\n//go:build !nomsgpack\n\npackage binding\n\nimport "x"\n')).toBe('binding');
+    expect(goPackageClause('/* license\n package notthis */\npackage gin // the root\n')).toBe('gin');
+    expect(goPackageClause('package main\n')).toBe('main');
+    expect(goPackageClause('// no clause\nfunc x() {}\n')).toBeNull();
+  });
+});
+
+describe('package-qualified Go names resolve on every surface', () => {
+  let dir = '';
+  let cg: CodeGraph;
+
+  beforeAll(async () => {
+    dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-go-package-names-'));
+    const write = (rel: string, content: string) => {
+      fs.mkdirSync(path.dirname(path.join(dir, rel)), { recursive: true });
+      fs.writeFileSync(path.join(dir, rel), content);
+    };
+    write('go.mod', 'module example.com/gin\n\ngo 1.22\n');
+    // The root package: its name is in the clause only.
+    write('context.go', '// Package gin is the root.\npackage gin\n\ntype Context struct{}\n\nfunc (c *Context) Next() {}\n');
+    write(
+      'binding/binding.go',
+      'package binding\n\ntype Binding interface {\n\tBind(req string) error\n}\n\nfunc Default(method string) Binding {\n\treturn nil\n}\n'
+    );
+    // Gives the full-text search `binding` + `default` hits that are not the function.
+    write(
+      'binding/default_validator.go',
+      'package binding\n\ntype defaultValidator struct{}\n\nfunc (v *defaultValidator) ValidateStruct(obj any) error {\n\treturn nil\n}\n'
+    );
+    write(
+      'discovery/refresh/refresh.go',
+      'package refresh\n\ntype Discovery struct {\n\trefreshf func() error\n}\n\nfunc (d *Discovery) refresh() error {\n\treturn d.refreshf()\n}\n'
+    );
+    write(
+      'discovery/uyuni/uyuni.go',
+      'package uyuni\n\ntype Discovery struct{}\n\nfunc (d *Discovery) refresh() error {\n\treturn nil\n}\n'
+    );
+    cg = CodeGraph.initSync(dir, { config: { include: ['**/*.go'], exclude: [] } });
+    await cg.indexAll();
+  }, 120_000);
+
+  afterAll(() => {
+    cg?.destroy();
+    if (dir) fs.rmSync(dir, { recursive: true, force: true });
+  });
+
+  const where = (nodes: Node[]) => nodes.map((n) => `${n.qualifiedName}@${n.filePath}`).sort();
+
+  it("explore's resolution finds each package's own symbol", () => {
+    expect(where(findAllSymbols(cg, 'refresh.Discovery.refresh').nodes)).toEqual(['Discovery::refresh@discovery/refresh/refresh.go']);
+    expect(where(findAllSymbols(cg, 'uyuni.Discovery.refresh').nodes)).toEqual(['Discovery::refresh@discovery/uyuni/uyuni.go']);
+    expect(where(findAllSymbols(cg, 'gin.Context.Next').nodes)).toEqual(['Context::Next@context.go']);
+    expect(findAllSymbols(cg, 'nope.Discovery.refresh').nodes).toEqual([]);
+  });
+
+  it('finds a definition the full-text search ranks below its cut', () => {
+    // Precondition: the search answers `binding.Default`, with other nodes.
+    const hits = cg.searchNodes('binding.Default', { limit: 50 });
+    expect(hits.length).toBeGreaterThan(0);
+    expect(hits.some((h) => h.node.name === 'Default')).toBe(false);
+    expect(where(findAllSymbols(cg, 'binding.Default').nodes)).toEqual(['Default@binding/binding.go']);
+  });
+
+  it('feeds explore its named symbols', () => {
+    const flow = resolveNamedTokens(cg, 'refresh.Discovery.refresh gin.Context.Next');
+    const named = [...flow.named.values()];
+    expect(where(named)).toEqual(['Context::Next@context.go', 'Discovery::refresh@discovery/refresh/refresh.go']);
+  });
+
+  it('codegraph_node and the CLI lookup agree', async () => {
+    const res = await new ToolHandler(cg).execute('codegraph_node', { symbol: 'uyuni.Discovery.refresh' });
+    const text = res.content[0].text as string;
+    expect(text).toContain('discovery/uyuni/uyuni.go');
+    expect(text).not.toContain('discovery/refresh/refresh.go');
+    expect(where(lookupSymbolNodes(cg, 'gin.Context.Next', goPackageReaderFor(cg)).nodes)).toEqual(['Context::Next@context.go']);
+  });
+});

src/bin/codegraph.ts

@@ -72,6 +72,7 @@ import { getTelemetry, TELEMETRY_DOCS, recordIndexEvent } from '../telemetry';
 import { BROWSER_ENV, DEFAULT_UI_PORT } from '../ui-server/constants';
 import type { UiServerHandle } from '../ui-server';
 import { lookupSymbolNodes, describeSymbolNode, groupDefinitions } from '../graph/symbol-lookup';
+import { goPackageReaderFor } from '../graph/go-package';
 import type { Node, Edge } from '../types';
 import { isTestPath } from '../search/query-utils';
 
@@ -2332,7 +2333,7 @@ for (const direction of ['callers', 'callees'] as const) {
         const cg = await CodeGraph.open(projectPath);
         try {
           const limit = parseInt(options.limit || '20', 10);
-          const { nodes: targets } = lookupSymbolNodes(cg, symbol);
+          const { nodes: targets } = lookupSymbolNodes(cg, symbol, goPackageReaderFor(cg));
           if (targets.length === 0) {
             info(formatSymbolNotFound(symbol, cg.searchNodes(symbol, { limit: 5 }).map((m) => m.node.name)));
             return;
@@ -2456,7 +2457,7 @@ program
       const cg = await CodeGraph.open(projectPath);
       try {
         const depth = Math.min(Math.max(parseInt(options.depth || '2', 10), 1), 10);
-        const { nodes: targets } = lookupSymbolNodes(cg, symbol);
+        const { nodes: targets } = lookupSymbolNodes(cg, symbol, goPackageReaderFor(cg));
         if (targets.length === 0) {
           info(formatSymbolNotFound(symbol, cg.searchNodes(symbol, { limit: 5 }).map((m) => m.node.name)));
           return;

src/graph/go-package.ts

@@ -0,0 +1,67 @@
+/**
+ * The package a Go file declares — the name a `pkg.Type.method` query writes.
+ *
+ * It is usually the name of the file's directory, which is why a package
+ * qualifier is first compared with the path. Not always: a module's root
+ * package has no directory of its own (gin's `context.go` is `package gin` at
+ * the top of the repository), and a versioned directory (`v2/`) keeps its
+ * parent's name. The `package` clause is the authority, and it is not in the
+ * index, so it is read from the file when a lookup needs it.
+ */
+
+import { closeSync, openSync, readSync } from 'fs';
+import { validatePathWithinRoot } from '../utils';
+
+/** The package a Go file declares, by its project-relative path; null when unknown. */
+export type GoPackageOf = (filePath: string) => string | null;
+
+/** Enough of a file to get past any license header and build tags. */
+const HEAD_BYTES = 8192;
+
+/**
+ * The name in a Go source's `package` clause: the first token after its
+ * comments and build constraints. Null when there is none.
+ */
+export function goPackageClause(source: string): string | null {
+  const code = source.replace(/\/\*[\s\S]*?\*\//g, ' ').replace(/\/\/[^\n]*/g, '');
+  return /^\s*package\s+([A-Za-z_]\w*)/.exec(code)?.[1] ?? null;
+}
+
+/** `goPackageReader` for a graph's project; none when the root can't be had. */
+export function goPackageReaderFor(cg: { getProjectRoot(): string }): GoPackageOf | undefined {
+  try {
+    return goPackageReader(cg.getProjectRoot());
+  } catch {
+    return undefined;
+  }
+}
+
+/**
+ * A `GoPackageOf` for files under `projectRoot`, reading each file's head once.
+ * A file it cannot read declares no package, so the qualifier does not match.
+ */
+export function goPackageReader(projectRoot: string): GoPackageOf {
+  const cache = new Map<string, string | null>();
+  return (filePath) => {
+    const hit = cache.get(filePath);
+    if (hit !== undefined) return hit;
+    let pkg: string | null = null;
+    const absPath = validatePathWithinRoot(projectRoot, filePath);
+    if (absPath) {
+      try {
+        const fd = openSync(absPath, 'r');
+        try {
+          const buf = Buffer.alloc(HEAD_BYTES);
+          const n = readSync(fd, buf, 0, HEAD_BYTES, 0);
+          pkg = goPackageClause(buf.toString('utf8', 0, n));
+        } finally {
+          closeSync(fd);
+        }
+      } catch {
+        pkg = null;
+      }
+    }
+    cache.set(filePath, pkg);
+    return pkg;
+  };
+}

src/graph/named-symbol-flow.ts

@@ -36,7 +36,8 @@ import type CodeGraph from '../index';
 import type { Node, Edge } from '../types';
 import { isTestFile } from '../search/query-utils';
 
-import { lastQualifierPart, matchesSymbol } from './symbol-lookup';
+import { qualifiedSymbolMatches } from './symbol-lookup';
+import { goPackageReaderFor } from './go-package';
 
 // Preserve the existing imports while sharing the matcher with the CLI and MCP.
 export { RUST_PATH_PREFIXES, lastQualifierPart, matchesSymbol } from './symbol-lookup';
@@ -79,15 +80,9 @@ export function findAllSymbols(cg: CodeGraph, symbol: string): { nodes: Node[];
     // ranking a differently-cased sibling above the real node (#1473 Fetch).
     exactNodes = cg.getNodesByName(symbol);
   } else {
-    let results = cg.searchNodes(symbol, { limit: 50 });
-    // Mirror findSymbolMatches — FTS strips colons, so re-search by bare tail.
-    if (results.length === 0) {
-      const tail = lastQualifierPart(symbol);
-      if (tail && tail !== symbol) results = cg.searchNodes(tail, { limit: 50 });
-    }
-    exactNodes = results
-      .filter((r) => matchesSymbol(r.node, symbol))
-      .map((r) => r.node);
+    // Shared with codegraph_node's lookup: search rank first, the complete
+    // exact-name index when the search's cut misses the definition.
+    exactNodes = qualifiedSymbolMatches(cg, symbol, goPackageReaderFor(cg));
   }
 
   if (exactNodes.length === 0) {