Skip to content

fix(common-utils): recover select-alias map from unparseable queries#2486

Open
alex-fedotyev wants to merge 2 commits into
mainfrom
alex/HDX-1879-aliasmap-resilient
Open

fix(common-utils): recover select-alias map from unparseable queries#2486
alex-fedotyev wants to merge 2 commits into
mainfrom
alex/HDX-1879-aliasmap-resilient

Conversation

@alex-fedotyev

Copy link
Copy Markdown
Contributor

The select-alias map went empty whenever a rendered query contained ClickHouse-specific SQL that node-sql-parser's Postgresql dialect can't parse. The clearest case is a sampling CTE: filter value distributions render greatest(CAST(total / N AS UInt32), 1), and the CAST(... AS UInt32) cast makes the whole statement unparseable. When the parse threw, chSqlToAliasMap returned {}, which drops the WITH clauses that define a source's select aliases. Anything filtering on an aliased column (ServiceName as service, a ResourceAttributes['x'] map alias) then failed with Unknown identifier.

Summary

chSqlToAliasMap now falls back to parsing only the outer SELECT projection when the full statement doesn't parse. The alias map only needs the expr AS alias pairs in the projection, so stripping the CTEs and the WHERE clause (where the unparseable SQL usually lives) recovers the aliases.

  • Split the existing AST-to-alias-map extraction into a small helper so the happy path and the fallback share it (no behavior change on inputs that already parsed).
  • Added extractOuterSelectProjection, a paren- and quote-aware scan that returns the text between the top-level SELECT and FROM. Nested subqueries and CTE bodies are skipped because their keywords sit inside parentheses, and keywords inside string literals are ignored.
  • On a parse failure, re-parse SELECT <projection> FROM <table> once before giving up. If that also fails the map is empty, exactly as before.

Why

This makes aliasWith reliable for every consumer of the alias map (the search results table, histogram, alerts, the CLI), not just the simple-select sources that already parsed. It is the groundwork for fixing alias filters on the Event Patterns view.

Test plan

  • make ci-lint (eslint + tsc) on @hyperdx/common-utils
  • make ci-unit on @hyperdx/common-utils (1329 tests pass)
  • new unit tests in clickhouse.test.ts for the sampling-CTE / CAST case, bracket aliases, expression aliases, JSON-path aliases, and the string-literal edge case. Each fails on main and passes here.
  • confirmed the existing happy-path alias tests still return identical maps

[as-cast: allow] the as SQLParser.Select assertion is carried over verbatim from the existing implementation; node-sql-parser's astify returns a union type and the line keeps its eslint-disable comment.

chSqlToAliasMap returned an empty map whenever the rendered query
contained ClickHouse-specific SQL that node-sql-parser's Postgresql
dialect cannot parse, for example a sampling CTE with
`greatest(CAST(total / N AS UInt32), 1)`. An empty alias map drops the
WITH clauses that define a source's select aliases, so filters on
aliased columns (Event Patterns, histogram, alerts) failed with
"Unknown identifier".

On a parse failure it now falls back to parsing only the outer SELECT
projection, which is all the alias map needs, so the aliases are
recovered even when the rest of the statement is unparseable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jun 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7742d30

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@hyperdx/common-utils Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview, Comment Jun 17, 2026 11:26pm
hyperdx-storybook Ready Ready Preview, Comment Jun 17, 2026 11:26pm

Request Review

@github-actions github-actions Bot added the review/tier-2 Low risk — AI review + quick human skim label Jun 17, 2026
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

🔵 Tier 2 — Low Risk

Small, isolated change with no API route or data model modifications.

Why this tier:

  • Standard feature/fix — introduces new logic or modifies core functionality

Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns.
SLA: Resolve within 4 business hours.

Stats
  • Production files changed: 1
  • Production lines changed: 212 (+ 85 in test files, excluded from tier calculation)
  • Branch: alex/HDX-1879-aliasmap-resilient
  • Author: alex-fedotyev

To override this classification, remove the review/tier-2 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@greptile-apps

greptile-apps Bot commented Jun 17, 2026

Copy link
Copy Markdown

Greptile Summary

This PR fixes a silent alias-map regression where chSqlToAliasMap returned {} whenever the rendered query contained ClickHouse-specific SQL (e.g. CAST(x AS UInt32) in a sampling CTE) that node-sql-parser's Postgresql dialect couldn't parse. The empty map dropped WITH clause aliases, causing Unknown identifier errors on aliased columns in Event Patterns, histogram, and alerts.

  • Extracts a reusable selectColumnsToAliasMap helper (no behavior change for queries that already parsed successfully).
  • Adds extractOuterSelectProjection, a paren-, quote-, and comment-aware character scanner that returns only the text between the top-level SELECT and FROM, skipping CTEs and WHERE-clause subqueries where unparseable ClickHouse syntax lives.
  • chSqlToAliasMap now retries on a parse failure with SELECT <projection> FROM <table>, recovering all expr AS alias pairs while remaining a no-op if both attempts fail.

Confidence Score: 5/5

The change is additive and gracefully degrades: chSqlToAliasMap can only return more aliases than before, and still returns an empty map on total failure, so no existing caller can receive a worse result than on main.

The fallback path is well-isolated, the scanner correctly handles parens, single/double/backtick quotes, -- line comments, and block comments. All pre-existing happy-path tests still pass and seven targeted new tests cover the sampling-CTE scenario plus edge cases. The only theoretical gap is ''-escaped single quotes, which does not affect production SQL today.

No files require special attention; extractOuterSelectProjection in clickhouse/index.ts is the only genuinely new logic and is well-tested.

Important Files Changed

Filename Overview
packages/common-utils/src/clickhouse/index.ts Adds selectColumnsToAliasMap helper, extractOuterSelectProjection scanner (paren/quote/comment-aware), and a two-tier fallback in chSqlToAliasMap so the alias map is recovered even when ClickHouse-specific CTE syntax defeats the parser.
packages/common-utils/src/tests/clickhouse.test.ts Adds a dedicated resilient-parsing test suite covering sampling-CTE/CAST, bracket aliases, expression aliases, JSON-path aliases, string-literal edge cases, SQL comment edge cases, and total failure fallback.
.changeset/alias-map-projection-fallback.md Patch-level changeset entry for @hyperdx/common-utils describing the alias-map fallback fix.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[chSqlToAliasMap called] --> B{chSql null or empty?}
    B -->|yes| Z[return empty map]
    B -->|no| C[parameterizedQueryToSql]
    C --> D[extractSettingsClauseFromEnd]
    D --> E[replaceJsonExpressions]
    E --> F[selectColumnsToAliasMap full SQL]
    F -->|success| G[return alias map]
    F -->|parse throws| H[extractOuterSelectProjection]
    H -->|returns null| I[rethrow fullParseError]
    I --> J[outer catch: log error, return empty map]
    H -->|returns projection| K["selectColumnsToAliasMap SELECT projection FROM __hdx_alias_src"]
    K -->|success| G
    K -->|parse throws| I
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[chSqlToAliasMap called] --> B{chSql null or empty?}
    B -->|yes| Z[return empty map]
    B -->|no| C[parameterizedQueryToSql]
    C --> D[extractSettingsClauseFromEnd]
    D --> E[replaceJsonExpressions]
    E --> F[selectColumnsToAliasMap full SQL]
    F -->|success| G[return alias map]
    F -->|parse throws| H[extractOuterSelectProjection]
    H -->|returns null| I[rethrow fullParseError]
    I --> J[outer catch: log error, return empty map]
    H -->|returns projection| K["selectColumnsToAliasMap SELECT projection FROM __hdx_alias_src"]
    K -->|success| G
    K -->|parse throws| I
Loading

Reviews (2): Last reviewed commit: "fix(common-utils): skip SQL comments in ..." | Re-trigger Greptile

Comment thread packages/common-utils/src/clickhouse/index.ts
Make extractOuterSelectProjection skip `--` line comments and
`/* */` block comments so SELECT / FROM keywords inside a comment are
not mistaken for the projection boundaries. Raw SQL select or filter
expressions can contain comments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

E2E Test Results

All tests passed • 201 passed • 3 skipped • 1202s

Status Count
✅ Passed 201
❌ Failed 0
⚠️ Flaky 3
⏭️ Skipped 3

Tests ran across 4 shards in parallel.

View full report →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-2 Low risk — AI review + quick human skim

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant