fix(service-analytics, lint)!: a dataset or cube dimension over a formula field is refused where it is authored, instead of on the first query that groups by it

.changeset/30773-dataset-dimension-formula-refused.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/service-analytics': minor
+'@objectstack/lint': minor
+---
+
+fix(service-analytics, lint)!: a dataset or cube dimension over a formula field is refused where it is authored — the dataset compile and `os validate` / `os build` / `os lint` / the runtime dataset save — instead of on the first query that groups by it
+
+Clause-②: no (narrowing) — the dataset compile (`AnalyticsService.registerDataset`, `queryDataset` and every door that compiles through them) refuses a dataset declaring a dimension over a virtual (`formula`) field of its base object, and the authoring rules refuse a dataset or `analyticsCubes` dimension over one with the new id `dimension-virtual-field-refused`, bodies they accepted before. Nothing is widened: the one new export (`DIMENSION_VIRTUAL_FIELD_REFUSED`, the rule id's constant beside its two siblings in `@objectstack/lint`) adds no accepted input, and no error code, status or answer member is added.
+Metadata-protocol: narrowed
+Skills-docs: updated here — `content/docs/deployment/validating-metadata.mdx` names `dimension-virtual-field-refused` beside `dimension-json-stored-field-refused` and in its cube paragraph; follow-up #30824 moves `skills/objectstack-data/rules/field-types.md`'s formula row (the skills lane). The prescription ships in the refusal text and below.
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped: DatasetDimension.field and a cube dimension's sql keep their shape and meaning. A dimension over a formula never served a group (since the query-time refusal every query selecting it answers 400 INVALID_FIELD, and before it a 500, a driver backstop or one null group), so there is no rewrite that keeps what the author meant: the author stores the value on a field of its own, or removes the dimension, and the refusal names the dimension and the field. -->
+
+**BREAKING** accept-set narrowing at dataset authoring (`@objectstack/service-analytics`, `@objectstack/lint`), for a dimension over a `formula` field only.
+
+- **Before.** Measured on `main` `2d04cd2b`. A dataset whose dimension's `field` is a `formula` (any `returnType`) compiled and registered (`registerDataset`), and a query naming only its other, stored dimensions was served; a query grouping by the formula dimension was refused at query time (`INVALID_FIELD` / 400 at `groupBy.N`), in front of whoever opened the dashboard tile. `os validate`, `os build`, `os lint` and the runtime dataset save (`PUT /api/v1/meta/dataset/NAME`, the gate Studio, REST and MCP authors share) reported nothing for it, on a dataset or an `analyticsCubes` cube. The measure side already refused a total over a formula at both doors.
+- **Now.** The dataset compile refuses such a dataset with `DATASET_INVALID` / 400 naming the dataset, the dimension, its position (`dimensions.N`), the field and its type, and the remedy, whichever members a query names: `registerDataset`, `queryDataset` (native, ObjectQL and the draft preview alike) and the build probes that query through it. The authoring rules refuse the dimension with `dimension-virtual-field-refused` (error) at `datasets[N].dimensions[M].field` or `analyticsCubes[N].dimensions.KEY.sql`, judged on the leaf's own object for a dotted path, on all three CLI commands and at the runtime dataset save (`422`). The verdict is the spec's `isVirtualSearchField` (`SEARCH_VIRTUAL_TYPES`), the predicate the query-time doors use; `os explain dimension-virtual-field-refused` gives the reasoning.
+- **Unchanged.** A dimension over a stored field, a date or datetime dimension with or without `dateGranularity`, and every measure verdict, a `count` over a formula and the stored-number remedy for a total included. The compile does not judge a dotted dimension (its hook answers for the base object only); the lint leg and the query-time refusal still do.
+- **Who moves.** An author whose dataset or cube declares a dimension over a formula field: the dataset stops compiling until the dimension moves, and its stored dimensions stop being served with it.
+
+**The remedy.** Denormalise the value: store it in a field of its own on the object (written by a hook or a flow when its source changes) and point the dimension at that field, or remove the dimension.

content/docs/deployment/validating-metadata.mdx

@@ -247,11 +247,23 @@ above declared `multiple: true`), is refused
 Group by a field that stores one value instead. Like the measure check, it
 stays silent when the field's type cannot be resolved.
 
+A dimension over a **virtual** field, a `formula`, is refused under a second
+id. A formula is computed on read and never stored, whatever its `returnType`,
+so no driver has a column to group by: the analytics service refuses a query
+grouped by one with `400 INVALID_FIELD`, and refuses to compile a dataset that
+declares one over its own object with `400 DATASET_INVALID`, whichever members
+the query names. So such a dimension is refused
+(`dimension-virtual-field-refused`) at `datasets[N].dimensions[M].field`; a
+dimension through a relationship path is judged on the object the path
+reaches. Store the value in a field of its own (written by a hook or a flow
+when its source changes) and point the dimension at that field.
+
 Both checks also judge the members of an analytics cube (`analyticsCubes`),
 because an authored cube is queried through the same analytics door. A cube
 dimension whose `sql` column is declared with a structured-JSON type or as a
 multi-value field is refused (`dimension-json-stored-field-refused`) at
-`analyticsCubes[N].dimensions.<key>.sql`. A cube measure is judged exactly as a
+`analyticsCubes[N].dimensions.<key>.sql`, and one whose `sql` column is a
+formula is refused (`dimension-virtual-field-refused`) at the same path. A cube measure is judged exactly as a
 dataset measure is, its `type` being the aggregate: a pair outside the table is
 refused (`measure-aggregate-field-type-refused`, at
 `analyticsCubes[N].measures.<key>.type`). That covers `sum`, `avg`, `min` or

packages/lint/src/index.ts

@@ -64,6 +64,7 @@ export {
   validateDatasetMeasureAggregates,
   MEASURE_AGGREGATE_FIELD_TYPE_REFUSED,
   DIMENSION_JSON_STORED_FIELD_REFUSED,
+  DIMENSION_VIRTUAL_FIELD_REFUSED,
   MEASURE_DURATION_FIELD_TYPE_REFUSED,
   // [#23235] The dimension leg's group-key verdict on its own, so a consumer
   // deriving dataset dimensions (cloud's AI build) asks the door's two

packages/lint/src/rule-explanations.ts

@@ -2360,6 +2360,25 @@ const DIMENSION_JSON_STORED_FIELD_REFUSED_EXPLANATION: RuleExplanation = {
   ],
 };
 
+const DIMENSION_VIRTUAL_FIELD_REFUSED_EXPLANATION: RuleExplanation = {
+  rule: 'dimension-virtual-field-refused',
+  covers: 'why analytics does not group by a formula field',
+  paragraphs: [
+    'A dimension is a GROUP KEY: a dataset dimension compiles to a cube dimension whose `sql` is its ' +
+      '`field`, and a query that selects it groups (or date-buckets) by that column. A virtual field, a ' +
+      '`formula`, is computed on read and never stored, whatever its `returnType`: no driver materialises ' +
+      'a column for it, so there is nothing to group by. Every analytics face refuses such a group with ' +
+      '`400 INVALID_FIELD` at `groupBy.N`, and the dataset compile refuses a dataset declaring one over its ' +
+      'own object with `400 DATASET_INVALID`, so no report, dashboard or query is ever served from it. ' +
+      'Before those refusals the group key reached the driver as a column that does not exist, or an ' +
+      'engine that aggregates in memory put every row into one null group while the answer looked served.',
+    'Denormalise the value: store it in a field of its own on the object (written by a hook or a flow ' +
+      'when its source changes), and point the dimension at that stored field. This rule makes the same ' +
+      'refusal where the author is standing. A dotted path is judged on the leaf\'s own declaration.',
+    ANALYTICS_MEMBER_WALK,
+  ],
+};
+
 const MEASURE_DURATION_FIELD_TYPE_REFUSED_EXPLANATION: RuleExplanation = {
   rule: 'measure-duration-field-type-refused',
   covers: 'which fields a duration measure measures between',
@@ -5737,6 +5756,7 @@ export const RULE_EXPLANATIONS: Readonly<Record<string, RuleExplanation>> = Obje
   [VALIDATION_RULE_JSON_SCHEMA_UNKNOWN_FORMAT_EXPLANATION.rule]: VALIDATION_RULE_JSON_SCHEMA_UNKNOWN_FORMAT_EXPLANATION,
   [MEASURE_AGGREGATE_FIELD_TYPE_REFUSED_EXPLANATION.rule]: MEASURE_AGGREGATE_FIELD_TYPE_REFUSED_EXPLANATION,
   [DIMENSION_JSON_STORED_FIELD_REFUSED_EXPLANATION.rule]: DIMENSION_JSON_STORED_FIELD_REFUSED_EXPLANATION,
+  [DIMENSION_VIRTUAL_FIELD_REFUSED_EXPLANATION.rule]: DIMENSION_VIRTUAL_FIELD_REFUSED_EXPLANATION,
   [MEASURE_DURATION_FIELD_TYPE_REFUSED_EXPLANATION.rule]: MEASURE_DURATION_FIELD_TYPE_REFUSED_EXPLANATION,
   [REACT_CHART_DRILLDOWN_INVALID_EXPLANATION.rule]: REACT_CHART_DRILLDOWN_INVALID_EXPLANATION,
   [REACT_CHART_AGGREGATE_INVALID_EXPLANATION.rule]: REACT_CHART_AGGREGATE_INVALID_EXPLANATION,

packages/lint/src/runtime-gate.dataset-writes.test.ts

@@ -61,6 +61,8 @@ const OBJECTS = [
       issued_on: { type: 'date', label: 'Issued' },
       total: { type: 'currency', label: 'Total' },
       account: { type: 'lookup', label: 'Account', reference: 'acme_account' },
+      // [#30773] A virtual field: computed on read, never stored.
+      total_label: { type: 'formula', label: 'Total label', expression: 'record.region', returnType: 'text' },
     },
   },
   {
@@ -229,6 +231,21 @@ describe('the door refuses each crossed rule s own defect (#19143)', () => {
     expect(result.rulesRun).toContain('validateDatasetMeasureAggregates');
   });
 
+  it('⭐ LIT — a dimension over a formula field, which no query can group by (#30773)', () => {
+    // The dataset compile refuses it with `400 DATASET_INVALID` and every
+    // analytics face refuses the group with `400 INVALID_FIELD`; this is the
+    // same refusal at the save, where the author is standing.
+    const result = expectRefusal(
+      cleanDataset({ dimensions: [{ name: 'status', field: 'status' }, { name: 'total_label', field: 'total_label' }] }),
+      'dimension-virtual-field-refused',
+    );
+    expect(result.errors.find((e) => e.rule === 'dimension-virtual-field-refused')?.path).toBe(
+      // The door rewrites the snapshot's index to the written item's name.
+      'datasets.acme_invoice_metrics.dimensions[1].field',
+    );
+    expect(result.rulesRun).toContain('validateDatasetMeasureAggregates');
+  });
+
   it('⭐ LIT — a base object that resolves to nothing (the split-off rung)', () => {
     // `validateObjectReferences` owns this one rather than
     // `validateDatasetReferences`, because the reference is an object NAME and