Repository navigation
feat: let SQL query dashboard filters use an optional source - #3691
karl-power wants to merge 3 commits into
Conversation
Add an optional source to SQL query filters so their SQL can use $__sourceTable and $__filters, as raw SQL tiles can. - Add `source` to the filter schema and expand $__sourceTable and $__filters when it is set; without one, both are still rejected - Expand $__filters to the dashboard's broadcast selections for that source, in both the dropdown and the editor preview - Add a clearable source picker next to the connection picker, limited to Log, Trace, Session and Metric sources on the chosen connection - Move resolveConnectionSourceSync out of the raw SQL chart editor into a shared util so the filter editor can use it - Check on save that the source exists, is a supported kind, and is on the filter's connection - Read and write it as `sourceId` in the external API and MCP, and map it by name on dashboard export and import Refs HDX-5524
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 4ca7b11 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| id => ({ id, kinds }), | ||
| ); | ||
| } | ||
| if (isSqlQueryFilter(filter) && filter.source) { |
There was a problem hiding this comment.
🟠 major — Internal API and dashboard import accept a SQL filter source on a different connection from the filter
The external API now rejects this case through getSourceConnectionMismatches, but the internal API does not. getFilterSourceRefs and validateFilterSources only check the source's kind. The import mapping schema (DBDashboardImportPage.tsx buildMappingFormSchema, around L579) maps filterConnectionMappings and filterSourceMappings separately. So an import can pair connection A with a source on connection B. The values query then runs on A against B's $__sourceTable, and $__filters resolves column metadata against B. Fix: in validateFilterSources, reject a SQL_QUERY filter whose source.connection !== filter.connection. Also add an import-schema issue on filterSourceMappings.${idx} when the mapped source's connection differs from the mapped connection.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
There was a problem hiding this comment.
Fixed in 81d2f9c: validateFilterSources now rejects a SQL query filter whose source is on another connection. Imports save through this endpoint, so they're covered too. I've left out the extra import-form check, since the server already blocks the mismatched save.
PR Review
5 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 5 minor 5 posted as inline comment(s) on the changed lines. Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Duplicate code analysis2 new clones introduced by this PR.
If one of these duplicates a helper that already exists, import it instead jscpd compared this branch against |
E2E Test Results✅ All tests passed • 468 passed • 1 skipped • 1187s
Tests ran across 4 shards in parallel. |
🔴 Tier 4 — CriticalTouches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI. Why this tier:
Additional context: touches the query rendering engine lightly (47 lines, under the 150-line bar for Tier 4) Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
| ]; | ||
|
|
||
| /** The source kinds a SQL_QUERY filter can use, the same as raw SQL tiles. */ | ||
| export const SQL_QUERY_FILTER_SOURCE_KINDS: SourceKind[] = [ |
There was a problem hiding this comment.
🔵 minor — SQL_QUERY_FILTER_SOURCE_KINDS duplicates QUERY_EXPRESSION_FILTER_SOURCE_KINDS
The new array is identical (Log, Trace, Session, Metric) to QUERY_EXPRESSION_FILTER_SOURCE_KINDS a few lines above in the same file. That gives two lists to keep in sync by hand, which breaks the REQUIRED DRY rule. Alias it (export const SQL_QUERY_FILTER_SOURCE_KINDS = QUERY_EXPRESSION_FILTER_SOURCE_KINDS), or derive both from one constant such as a shared NON_PROMQL_SOURCE_KINDS, with a doc comment.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
+1 it would be good to re-use if reasonable
| // The source is optional, so drop it rather than export an empty name | ||
| // the template schema would reject. | ||
| if (filter.source) { | ||
| filter.source = getSourceName(filter.source) || undefined; |
There was a problem hiding this comment.
🔵 minor — Export silently drops a deleted source, so the imported SQL filter can't be remapped
When the source is gone, the filter is exported with source: undefined but its sqlTemplate still uses $__sourceTable/$__filters. On import, isSourceBackedTemplateFilter is false, so there is no 'Data source' mapping row, and validateDashboardFilterQueries only shows a non-blocking warning. The dashboard imports with a filter that always errors with 'no source is selected'. Export a placeholder name, or keep a marker that the filter had a source, so the import UI shows a mapping row and requires one.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| * Returns an error message if a SQL_QUERY filter's source is a kind its | ||
| * $__sourceTable macro can't expand (PromQL), or null. | ||
| */ | ||
| function getSqlQueryFilterSourceKindError( |
There was a problem hiding this comment.
🔵 minor — SQL filter source rules are implemented twice, once per API
getSqlQueryFilterSourceKindError plus the new filter branch in getSourceConnectionMismatches restate what getFilterSourceRefs/validateFilterSources in packages/api/src/routers/api/dashboards.ts already enforce: an allowed kind and the same connection. The internal version also skips unchanged refs, which the external one doesn't, so the two can already disagree when a source has since been deleted or changed. Move the kind and connection check into one shared helper (for example in packages/api/src/utils/) that both routers call.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| await user.click(screen.getByRole('button', { name: 'Save' })); | ||
|
|
||
| expect( | ||
| screen.queryByText(/no source is selected|cannot be used/), |
There was a problem hiding this comment.
🔵 minor — Absence assertion runs before async validation, so 'accepts the source macros' can't fail
handleSubmit validates asynchronously, and queryByText(...) runs right after the click, before any error could render. The test would still pass if the form rejected $__sourceTable with a source selected. Assert on a positive outcome instead: pass an onSubmit spy to the Harness and await waitFor(() => expect(onSubmit).toHaveBeenCalled()), or flush the validation first, as the sibling test does with findByText.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| stretches it to the full width. */} | ||
| <div | ||
| data-testid="filter-sql-source-select" | ||
| style={{ display: 'grid' }} |
There was a problem hiding this comment.
🔵 minor — Raw inline style used where a Mantine style prop exists
<div style={{ display: 'grid' }}> breaks the convention 'Prefer individual Mantine style props over raw styles'. Use <Box display="grid" data-testid="filter-sql-source-select"> instead.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Deep Review✅ No critical issues found. No P0/P1 ship-blockers. Security, correctness, API-contract, and performance reviewers returned clean, and both fixes the author flagged in prior comments ( 🟡 P2 -- recommended
🔵 P3 nitpicks (4)
Reviewers (11): correctness, security, adversarial, kieran-typescript, api-contract, testing, maintainability, performance, previous-comments, project-standards, agent-native. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
Summary
Lets SQL query dashboard filters pick an optional source, so their SQL can use
$__sourceTableand$__filterslike raw SQL tiles. Follows up on review feedback on #3674.sourceto the filter schema. Without one,$__sourceTableand$__filtersare still rejected.$__filtersexpands to the dashboard's broadcast selections for that source, via a newrenderSourceFiltersToSqlwrapper around the existingrenderFiltersToSql.resolveConnectionSourceSyncfromRawSqlChartEditor.tsxtoutils/connectionSourceSync.tsso the filter editor can reuse it.sourceIdand also check that the source is on the filter's connection.Screenshots or video
How to reproduce
Starting point:
yarn devwith a "Local ClickHouse" connection and a Logs source ondefault.otel_logs.On
main:SELECT DISTINCT ServiceName FROM $__sourceTable. The editor rejects it.On this branch:
Severityon Logs with expressionSeverityText.Servicewith connection "Local ClickHouse", Source Logs, variablesvc, and:References