Repository navigation
feat: create SQL query dashboard filters and fill their dropdowns - #3678
karl-power wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 517403d The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
cf6ccc5 to
d411792
Compare
Duplicate code analysis1 new clone introduced by this PR.
If one of these duplicates a helper that already exists, import it instead jscpd compared this branch against |
E2E Test Results✅ All tests passed • 464 passed • 1 skipped • 1019s
Tests ran across 4 shards in parallel. |
🟡 Tier 3 — StandardIntroduces new logic, modifies core functionality, or touches areas with non-trivial risk. Why this tier:
Review process: Full human review — logic, architecture, edge cases. Stats
|
PR Review
3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor 2 posted as inline comment(s) on the changed lines. 1 listed below. Findings outside the changed lines1 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Deep ReviewScope: Note on method: The specialist reviewer sub-agents were dispatched but had not returned findings when output was required. The findings below come from a direct, full read of every changed source file. Treat the reviewer footer accordingly — the per-finding ✅ No critical issues found. The new code is defensively written: the value-query wrapper puts the user SQL on its own lines so a trailing 🟡 P2 -- recommended
🔵 P3 nitpicks (3)
Reviewers (1): direct-diff-analysis (orchestrator). Dispatched but unreturned at output time: correctness, security, kieran-typescript, testing, maintainability, adversarial, julik-frontend-races, previous-comments, project-standards, performance. Testing gaps: New behavior is well covered ( No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
|
Addressed the findings outside the changed lines in e9c4dca:
The jscpd clone with |
e9c4dca to
c5edc1a
Compare
| ], | ||
| // Keep the last values while a new range or selection loads, rather than | ||
| // blanking the dropdown. | ||
| placeholderData: keepPreviousData, |
There was a problem hiding this comment.
🔵 minor — keepPreviousData has no effect inside useQueries when the key changes, so the dropdown goes blank on every reload
In react-query v5, useQueries matches observers by query hash. When the expanded SQL or the time params change, the query gets a new observer that has no previous data, so keepPreviousData returns undefined and values drops to [] until the fetch finishes. This contradicts the comment on the line above it. Follow the sibling hooks (usePromqlLabelFilterValues.tsx:102, useQueriedDashboardFilterValues.tsx:373): build a key prefix of ['dashboard-filter-sql-query-values', call.connection], then in placeholderData return the most recently updated cached entry from queryClient.getQueriesData({ queryKey: prefix }). This filter's own variables can change the SQL, so the prefix probably also needs the filter id.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| // Only the params the SQL reads, so a query that ignores the time range | ||
| // keeps its cache key, and isn't re-run, when the range changes. | ||
| params: Object.fromEntries( | ||
| Object.entries(timeParams).filter(([name]) => sql.includes(`{${name}:`)), |
There was a problem hiding this comment.
🔵 minor — A raw {intervalSeconds:Int64} / {intervalMilliseconds:Int64} param passes validation but is never sent, so the query fails at runtime
expandSqlQueryFilterTemplate rejects the interval macros (line 888), but the params only include start/end. A template that writes {intervalSeconds:Int64} directly, as TIME_CHART_EXAMPLE_SQL in rawSqlParams.ts shows, passes validateSqlQueryFilterTemplate in the form, API and MCP. Every run then fails in ClickHouse with a missing-substitution error. Reject templates that reference {intervalSeconds: or {intervalMilliseconds: in expandSqlQueryFilterTemplate, using the same "have no time buckets" message, so saving fails up front.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Add a "SQL query values" type to the dashboard filter editor and populate the filter's dropdown from its query. - Render the filter's query in common-utils: expand time macros and other filters' variables, select the first column as a string, and cap the result at 10,000 rows - Run each SQL query filter's query on its connection and merge the values and errors into the dashboard's filter values - Keep the query's row order in the dropdown and show a caution while a referenced variable has no selection - Add the editor form: connection picker, SQL editor with macro and variable autocomplete, variable name, and an on-demand preview of the values over the dashboard's time range Refs HDX-5524
- Reject interval macros in SQL query filters, which have no time buckets - Validate the SQL on save against the dashboard's other variables - Build the editor's unsupported-macro list from the shared constants - Show the preview loader while a re-run fetches
c5edc1a to
517403d
Compare
pulpdrew
left a comment
There was a problem hiding this comment.
Couple of thoughts, looks pretty good though and works well! One other thing that could be nice is a couple of E2E exercising this filter type
| onClick={runPreview} | ||
| data-testid="filter-sql-preview-button" | ||
| > | ||
| Preview values |
There was a problem hiding this comment.
This is neat! One thing I notice is that when the query is invalid, it retries 3 times, which creates a long delay before we show the error. It also retries when I leave the tab and come back. It would be nice to fail after one retry here, and not refresh on tab focus.
|
|
||
| <CustomInputWrapper | ||
| label="SQL query" | ||
| tooltipText="The first column of the results fills the dropdown, in the order the query returns them. The query may use the time macros and reference the dashboard's other variables." |
| [RawSqlQueryParam.endDateMilliseconds]: dateRange[1].getTime(), | ||
| }; | ||
| return { | ||
| sql: `SELECT toString(tupleElement(tuple(*), 1)) AS value FROM (\n${sql}\n) LIMIT ${SQL_QUERY_FILTER_VALUES_LIMIT}`, |
There was a problem hiding this comment.
Should we add distinct here and let clickhouse do the dedupe? Also should we add a result overflow break setting so that partial queries (that time out) show partial results
| label: 'Queried values', | ||
| }, | ||
| { value: DashboardFilterType.enum.STATIC_LIST, label: 'Static values' }, | ||
| { value: DashboardFilterType.enum.SQL_QUERY, label: 'SQL query values' }, |
There was a problem hiding this comment.
nit: it's a little confusing that we now have Queried values and SQL query values as options. Maybe we can think of a new name for one or both?
| <SQLEditorControlled | ||
| control={control} | ||
| name="sqlTemplate" | ||
| rules={{ | ||
| validate: (value: unknown) => { | ||
| if (typeof value !== 'string' || !value.trim()) { | ||
| return 'This field is required'; | ||
| } | ||
| // `variables` leaves out this filter's own, so a reference to | ||
| // it is rejected as unknown. | ||
| return ( | ||
| validateSqlQueryFilterTemplate(value, variables ?? []) ?? true | ||
| ); | ||
| }, | ||
| }} | ||
| height="150px" | ||
| enableLineWrapping | ||
| placeholder={PLACEHOLDER_SQL} | ||
| additionalCompletions={completions} | ||
| onSubmit={runPreview} | ||
| /> |
There was a problem hiding this comment.
It would be nice if this had auto complete (table and column names) like the Raw SQL Tile Input


Summary
Adds the "SQL query values" dashboard filter type to the UI: users can create the filter, and its dropdown lists the first column of a ClickHouse query. This is the last PR in the stack; earlier PRs added the schema (#3672) and API/MCP support (#3674).
renderSqlQueryFilterValuesQueryin common-utils expands time macros and other filters' variables, wraps the SQL asSELECT toString(tupleElement(tuple(*), 1)) AS value FROM (...) LIMIT 10000, and strips a trailing;. Time params are sent only when the SQL uses them.useSqlQueryFilterValuesruns each filter's query on its connection withmax_execution_time: 30. Nulls, empty strings and duplicates are dropped, and the query's order is kept, so the dropdown doesn't sort these values.useDashboardFilterValuesmerges these values and errors with the other filter types. A query error shows as the dropdown's caution, as do variables the query references that have no selection yet.$__sourceTableand$__filtersare left out, since the filter has no source), and the variable name. The filter is variable-only, so there are no broadcast or variable toggles.Screenshots or video
Untitled.mov
How to reproduce
On
main:On this branch:
Service, pick the demo connection, and enter:svc, click Save filter, and close the modal. The Service dropdown lists the same services in the query's order.Severitywith optionsINFOandERRORand variablesev. Change the SQL to addAND $__filter(SeverityText, $sev). Picking a severity narrows the Service dropdown.SELECT nope FROM default.otel_logsand preview. The preview shows the ClickHouse error, and after saving, the dropdown shows it as a caution.References