fix(common-utils): recover select-alias map from unparseable queries#2486
fix(common-utils): recover select-alias map from unparseable queries#2486alex-fedotyev wants to merge 2 commits into
Conversation
chSqlToAliasMap returned an empty map whenever the rendered query contained ClickHouse-specific SQL that node-sql-parser's Postgresql dialect cannot parse, for example a sampling CTE with `greatest(CAST(total / N AS UInt32), 1)`. An empty alias map drops the WITH clauses that define a source's select aliases, so filters on aliased columns (Event Patterns, histogram, alerts) failed with "Unknown identifier". On a parse failure it now falls back to parsing only the outer SELECT projection, which is all the alias map needs, so the aliases are recovered even when the rest of the statement is unparseable. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 7742d30 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🔵 Tier 2 — Low RiskSmall, isolated change with no API route or data model modifications. Why this tier:
Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns. Stats
|
Greptile SummaryThis PR fixes a silent alias-map regression where
Confidence Score: 5/5The change is additive and gracefully degrades: chSqlToAliasMap can only return more aliases than before, and still returns an empty map on total failure, so no existing caller can receive a worse result than on main. The fallback path is well-isolated, the scanner correctly handles parens, single/double/backtick quotes, -- line comments, and block comments. All pre-existing happy-path tests still pass and seven targeted new tests cover the sampling-CTE scenario plus edge cases. The only theoretical gap is ''-escaped single quotes, which does not affect production SQL today. No files require special attention; extractOuterSelectProjection in clickhouse/index.ts is the only genuinely new logic and is well-tested. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[chSqlToAliasMap called] --> B{chSql null or empty?}
B -->|yes| Z[return empty map]
B -->|no| C[parameterizedQueryToSql]
C --> D[extractSettingsClauseFromEnd]
D --> E[replaceJsonExpressions]
E --> F[selectColumnsToAliasMap full SQL]
F -->|success| G[return alias map]
F -->|parse throws| H[extractOuterSelectProjection]
H -->|returns null| I[rethrow fullParseError]
I --> J[outer catch: log error, return empty map]
H -->|returns projection| K["selectColumnsToAliasMap SELECT projection FROM __hdx_alias_src"]
K -->|success| G
K -->|parse throws| I
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[chSqlToAliasMap called] --> B{chSql null or empty?}
B -->|yes| Z[return empty map]
B -->|no| C[parameterizedQueryToSql]
C --> D[extractSettingsClauseFromEnd]
D --> E[replaceJsonExpressions]
E --> F[selectColumnsToAliasMap full SQL]
F -->|success| G[return alias map]
F -->|parse throws| H[extractOuterSelectProjection]
H -->|returns null| I[rethrow fullParseError]
I --> J[outer catch: log error, return empty map]
H -->|returns projection| K["selectColumnsToAliasMap SELECT projection FROM __hdx_alias_src"]
K -->|success| G
K -->|parse throws| I
Reviews (2): Last reviewed commit: "fix(common-utils): skip SQL comments in ..." | Re-trigger Greptile |
Make extractOuterSelectProjection skip `--` line comments and `/* */` block comments so SELECT / FROM keywords inside a comment are not mistaken for the projection boundaries. Raw SQL select or filter expressions can contain comments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
E2E Test Results✅ All tests passed • 201 passed • 3 skipped • 1202s
Tests ran across 4 shards in parallel. |
The select-alias map went empty whenever a rendered query contained ClickHouse-specific SQL that node-sql-parser's Postgresql dialect can't parse. The clearest case is a sampling CTE: filter value distributions render
greatest(CAST(total / N AS UInt32), 1), and theCAST(... AS UInt32)cast makes the whole statement unparseable. When the parse threw,chSqlToAliasMapreturned{}, which drops theWITHclauses that define a source's select aliases. Anything filtering on an aliased column (ServiceName as service, aResourceAttributes['x']map alias) then failed withUnknown identifier.Summary
chSqlToAliasMapnow falls back to parsing only the outerSELECTprojection when the full statement doesn't parse. The alias map only needs theexpr AS aliaspairs in the projection, so stripping the CTEs and theWHEREclause (where the unparseable SQL usually lives) recovers the aliases.extractOuterSelectProjection, a paren- and quote-aware scan that returns the text between the top-levelSELECTandFROM. Nested subqueries and CTE bodies are skipped because their keywords sit inside parentheses, and keywords inside string literals are ignored.SELECT <projection> FROM <table>once before giving up. If that also fails the map is empty, exactly as before.Why
This makes
aliasWithreliable for every consumer of the alias map (the search results table, histogram, alerts, the CLI), not just the simple-select sources that already parsed. It is the groundwork for fixing alias filters on the Event Patterns view.Test plan
make ci-lint(eslint + tsc) on@hyperdx/common-utilsmake ci-uniton@hyperdx/common-utils(1329 tests pass)clickhouse.test.tsfor the sampling-CTE /CASTcase, bracket aliases, expression aliases, JSON-path aliases, and the string-literal edge case. Each fails onmainand passes here.[as-cast: allow] the
as SQLParser.Selectassertion is carried over verbatim from the existing implementation; node-sql-parser'sastifyreturns a union type and the line keeps itseslint-disablecomment.