docs(metrics-sql-api): document date parts (EXTRACT, DATE_PART, DATE_TRUNC) - #1211
Merged
Conversation
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
Contributor
Documentation validation✅ All blocking checks passed. Component system0 finding(s) across the corpus. Findings on lines this pull request touched are annotated inline in Files changed. |
notgiorgi
added a commit
to lightdash/lightdash
that referenced
this pull request
Sep 4, 2026
…frames (#28645) ## Problem BI tools derive date parts from a date field as `CAST(EXTRACT(YEAR FROM <dim>::TIMESTAMP) AS INT)` and sort by the same expression, while selecting only the field's interval columns. The Metrics SQL API compiled that as a table calculation, which can only reference selected fields, so the statement failed with `42601 Expression references "<dim>" which is not in the SELECT list` and the chart never loaded. ## Change Date parts now compile to the dimension's Lightdash time frame, at the query grain, in the warehouse: ```diff compileSelect SELECT item plain ref → dimension / metric + [CAST(]EXTRACT(part FROM dim[::TIMESTAMP])[ AS INT)] + DATE_PART('part', dim) / DATE_TRUNC('part', dim)[::DATE] + part → TimeFrames (YEAR→YEAR_NUM, DATE_TRUNC('month')→MONTH, …) + explore has dim_<frame> → select that field + else → customDimensions[] (sql from timeFrameConfigs[frame].getSql(explore.targetDatabase, …)) + DOW / ISODOW → explicit "not supported" error anything else → table calculation (unchanged) ORDER BY / GROUP BY integer | ref → existing paths - expression → "only column names or positions" + expression repeated from the SELECT list → that column ``` ```mermaid sequenceDiagram participant L as BI tool participant P as pgwire participant C as sqlToMetricQuery participant W as warehouse L->>P: SELECT date_day, CAST(EXTRACT(YEAR FROM date)…) AS "Year" … ORDER BY CAST(EXTRACT(…)) P->>C: compile against catalog C-->>P: MetricQuery { dimensions: [date_day, <year_num>], customDimensions?: [...], sorts: [<year_num>] } P->>W: DATE_PART('YEAR', "orders".order_date) grouped with date_day W-->>L: rows sorted by year ``` - The catalog now records each field's time interval (`frame` + base dimension name) so an existing `_year_num` / `_year` column is reused, including when the part is taken from a sibling interval like `<dim>_day`. Tables record the explore's `targetDatabase` so synthesised SQL matches the warehouse dialect. - Synthesised parts are custom SQL dimensions and go through the same custom-fields permission as calculated expressions already do on the wire server; parts the model lists as intervals work for every role. - Parts without a frame (`EPOCH`, `CENTURY`, …) keep the table-calculation path. A synthesised `WEEK` follows the warehouse default rather than the project start of week, like the SQL the client wrote. ## Verified - Unit tests for every mapping, reuse vs synthesis, dialect, ORDER BY / GROUP BY expressions, dedup, and each error case. - End to end through the local wire server with psql: the exact statement from the issue returns `orders_order_date_day | Year` sorted by year; quarter/day-of-year/month shapes work; DOW and non-date columns return explicit errors. ## Proof of work Baseline `ab0f81fafc` and this branch run side by side against the same seeded project, each statement sent over the wire protocol with `psql`. Compiled warehouse SQL captured from the warehouse statement log. | Scenario | Baseline | This branch | | --- | --- | --- | | Interval columns, `ORDER BY 1` (control) | rows | identical rows | | Looker Studio shape: `CAST(EXTRACT(YEAR FROM d::TIMESTAMP) AS INT)` selected + sorted, base dim not selected | `42601` not in SELECT list | `_day, Year` sorted by year; warehouse gets `DATE_PART('YEAR', "orders".order_date) … GROUP BY 1,2 ORDER BY <year>` | | Base dim selected, `ORDER BY <expr> DESC` | `ORDER BY only supports column names or positions` | rows sorted by year | | `DATE_TRUNC('month', d)` in SELECT / GROUP BY / ORDER BY | `42601` | month buckets; reuses the model's `orders_order_date_month` | | `QUARTER`, `DOY`, `DATE_PART('month')`, `DATE_TRUNC('quarter')::DATE`, none in the model | `42601` | correct values via synthesised `orders_order_date_pgwire_*` dimensions | | `EXTRACT(DOW)` | generic `42601` | `EXTRACT(DOW) is not supported` + hint | | `EXTRACT(EPOCH)` | table-calc error | unchanged (by design) | | `ORDER BY` expression absent from SELECT | old error | `ORDER BY expression must appear in the SELECT list` | Not exercised end to end: non-Postgres warehouses (covered by a unit test for the BigQuery dialect), timestamp `HOUR`/`MINUTE` parts, week-start alignment of a synthesised `WEEK`. Docs: lightdash/mintlify-docs#1211 Closes: PROD-10942 Closes: #28635
lightdash-bot
pushed a commit
to lightdash/lightdash
that referenced
this pull request
Sep 4, 2026
# [2.126.0](2.125.0...2.126.0) (2026-09-04) ### Bug Fixes * **ai-agents:** hide error bubbles for unanswered Slack context turns ([#28607](#28607)) ([7ca3645](7ca3645)) * **pgwire:** compile EXTRACT/DATE_TRUNC over date dimensions to time frames ([#28645](#28645)) ([766d29a](766d29a)), closes [lightdash/mintlify-docs#1211](lightdash/mintlify-docs#1211) [#28635](#28635) ### Features * **chart-registry:** beta badge on beta-channel library charts ([#28649](#28649)) ([ec2e539](ec2e539)), closes [#28653](#28653)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the date-part support added in lightdash/lightdash#28645:
EXTRACT,DATE_PARTandDATE_TRUNCover a date or timestamp dimension compile to the dimension's time frame,ORDER BY/GROUP BYcan repeat the expression,DOWis rejected, and synthesised parts need the developer custom-fields permission.Relates: PROD-10942
Relates: lightdash/lightdash#28635