Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .changeset/analytics-like-escape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
"@objectstack/service-analytics": patch
---

fix(service-analytics): the three SQL compilers compare LIKE values literally (#5567)

`$contains` / `$notContains` / `$startsWith` / `$endsWith` build a `LIKE` pattern
around the comparand the author wrote. All three of this package's SQL compilers
concatenated that comparand straight into a wildcard position — no escaping, no
`ESCAPE` clause — so `_` (LIKE's single-character wildcard) and `%` (its
multi-character one) stopped being literals. Measured on real SQLite, over the
rows `x_admin` / `xyadmin` / `off 50% now` / `off 5012 now`:

| `where` | returned | correct |
|----------------------------------|---------------|---------|
| `{name: {$contains: '_admin'}}` | `['1','2']` | `['1']` |
| `{name: {$contains: '50%'}}` | `['3','4']` | `['3']` |
| `{name: {$startsWith: 'x_'}}` | `['1','2']` | `['1']` |
| `{name: {$endsWith: '0% now'}}` | `['3','4']` | `['3']` |

Every row is a **widening** — rows the author excluded came back — and
`$notContains` is the mirror image, excluding rows the author kept. One of the
three call sites is the ADR-0021 D-C read-scope (tenant + RLS) lowering, where a
wider predicate is over-reach rather than a loose filter (the #5347 / #5324
ruling on that same file). Prime Directive #3 forces machine names to
`snake_case`, so essentially every machine-name comparand carries a `_` and hit
this silently.

All three compilers now escape the comparand and bind an explicit
`ESCAPE` argument, matching what `driver-sql`'s `applyLike` has always done — so
the same filter selects the same rows whichever strategy answers, and the
`/analytics/sql` echo describes the statement that ran instead of a wider one.

**No authoring change.** A comparand with no `_`, `%` or `\` binds exactly the
pattern it bound before; only its meaning when it *does* carry one changes, from
wildcard to literal. If you were relying on a comparand acting as a wildcard,
that was never a declared capability of these operators — the spec describes them
as substring / prefix / suffix matches — and `driver-sql` already read it
literally, so the reading you got depended on which strategy served the query.
10 changes: 10 additions & 0 deletions packages/plugins/driver-sql/src/sql-driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6272,6 +6272,16 @@ export class SqlDriver implements IDataDriver {
* character (MySQL/Postgres do, but the explicit clause is correct for all
* three). `shape` positions the wildcard: `contains` → `%v%`, `starts` → `v%`,
* `ends` → `%v`.
*
* **Second implementation, deliberately** (#5567):
* `packages/services/service-analytics/src/like-pattern.ts` carries the same
* transform — same escaped character class, same three shapes, same bound
* `ESCAPE` — because `service-analytics` depends on no driver and this is a
* private method taking a knex builder, so there is nothing for it to import.
* That file's header explains the choice; it is held to THIS expression, character
* for character, by `service-analytics`'s `like-metacharacter-escape.test.ts`.
* A third hand-copy is the thing to refuse: import from one of the two, or add
* a consumer to that test.
*/
private applyLike(
builder: any,
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -235,24 +235,33 @@ describe('[#5333] `/analytics/sql` echo — every authorable operator renders a
// ── The issue's measured table, one case per row ────────────────────────────

describe("the issue's measured table", () => {
/*
* [#5567] Each pattern is now followed by a bound `ESCAPE` argument, so the
* echo describes the comparand `driver-sql` actually compares (its
* `applyLike` has always escaped and bound `ESCAPE`). None of these three
* comparands carries a `_` or `%`, so the PATTERN is byte-identical to what
* #5333 pinned — the second bind is the whole delta. The metacharacter cases,
* where the pattern itself changes and the row set with it, are in
* `like-metacharacter-escape.test.ts`.
*/
it('`$startsWith` echoes `LIKE` with the prefix pattern — was no WHERE at all', async () => {
const { sql, params } = await echo({ stage: { $startsWith: 'w' } });
expect(sql).toContain('WHERE');
expect(sql).toContain('stage LIKE $1');
expect(params).toEqual(['w%']);
expect(sql).toContain('stage LIKE $1 ESCAPE $2');
expect(params).toEqual(['w%', '\\']);
});

it('`$endsWith` echoes `LIKE` with the suffix pattern — was no WHERE at all', async () => {
const { sql, params } = await echo({ stage: { $endsWith: 'n' } });
expect(sql).toContain('WHERE');
expect(sql).toContain('stage LIKE $1');
expect(params).toEqual(['%n']);
expect(sql).toContain('stage LIKE $1 ESCAPE $2');
expect(params).toEqual(['%n', '\\']);
});

it('`$contains` still echoes the substring pattern — the row that already worked', async () => {
const { sql, params } = await echo({ stage: { $contains: 'o' } });
expect(sql).toContain('stage LIKE $1');
expect(params).toEqual(['%o%']);
expect(sql).toContain('stage LIKE $1 ESCAPE $2');
expect(params).toEqual(['%o%', '\\']);
});

it('the LIKE patterns are the ones the EXECUTED statement binds', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,13 @@ describe('compileScopedFilterToSql', () => {

it('comparison + string operators', () => {
expect(compileScopedFilterToSql({ amount: { $gte: 100 } }, 't').sql).toBe('"t"."amount" >= ?');
// [#5567] The LIKE family binds its pattern AND the escape character, so the
// comparand compares literally. `'A'` carries no metacharacter, so the
// pattern itself is unchanged — the second bind is the whole delta here.
// Metacharacter coverage (and the row sets) live in
// `like-metacharacter-escape.test.ts`.
expect(compileScopedFilterToSql({ name: { $startsWith: 'A' } }, 't')).toEqual({
sql: '"t"."name" LIKE ?', params: ['A%'],
sql: '"t"."name" LIKE ? ESCAPE ?', params: ['A%', '\\'],
});
});

Expand Down
120 changes: 120 additions & 0 deletions packages/services/service-analytics/src/like-pattern.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* LIKE pattern construction for this package's three SQL compilers (#5567).
*
* A `$contains` / `$notContains` / `$startsWith` / `$endsWith` comparand is a
* LITERAL the author typed. Concatenating it straight into a wildcard position
* silently reinterprets it as a pattern, because `_` is LIKE's single-character
* wildcard and `%` its multi-character one:
*
* - `{name: {$contains: '_admin'}}` matched `xyadmin` as well as `x_admin`;
* - `{name: {$contains: '50%'}}` matched `off 5012 now` as well as `off 50% now`.
*
* Both directions are WIDENING, and one of the three call sites is
* `read-scope-sql.ts` — the ADR-0021 D-C read-scope (tenant + RLS) lowering,
* where a wider predicate is over-reach rather than a loose filter (#5347 /
* #5324, on that same file). Prime Directive #3 forces machine names to
* `snake_case`, so essentially every machine-name comparand carries a `_` and
* hits this silently.
*
* ## The two halves are one fix
*
* Escaping the value and declaring the escape character are not independent
* steps — either alone is a different bug:
*
* - Escaping alone: `%\_admin%` with no escape character in force is a search
* for a literal backslash. SQLite has NO default escape character, so this
* would return zero rows there.
* - The clause alone: nothing in the pattern is escaped, so nothing changes.
*
* Hence {@link likePattern} always produces a pattern escaped for
* {@link LIKE_ESCAPE_CHAR}, and every emitter pairs it with an `ESCAPE` binding.
*
* ## Why the escape character is BOUND, never written as a literal
*
* Every emitter here binds it as an ordinary placeholder (`LIKE ? ESCAPE ?`)
* rather than writing `ESCAPE '\'` into the SQL text. Two reasons, both load-bearing:
*
* 1. **The literal spelling is not portable.** MySQL applies C escape syntax
* inside string literals — "If you want a LIKE string to contain a literal
* `\`, you must double it" — so the backslash escape character is spelled
* `'\\'` there and `'\'` on SQLite/Postgres. These compilers do not know
* which dialect will run their output. A bound value is escaped by the
* driver for its own dialect, so there is exactly one spelling here.
* 2. **It rides the existing placeholder plumbing.** `read-scope-sql.ts` emits
* `?` and BOTH of its consumers (`NativeSQLStrategy.applyReadScope`,
* `ObjectQLStrategy.generateSql`) renumber `?` → `$N` while pushing the
* matching value. Because the escape character is a bound value it is
* carried by that rewrite with no change at the upper layer — which answers
* the "which layer does the ESCAPE clause belong to" question in the issue:
* the predicate layer, entirely, because nothing above it has to know.
*
* Dialect support for the clause itself, confirmed against the vendors' own
* reference manuals (quoted in PR for #5567): Postgres defaults to backslash and
* accepts `ESCAPE`; MySQL assumes `\` unless `NO_BACKSLASH_ESCAPES` is set and
* accepts `ESCAPE` with an argument that "must evaluate as a constant at
* execution time" (a bound placeholder is); SQLite honours NO default escape
* character at all, which is the reason the explicit clause is required rather
* than merely tidy.
*
* ## Relationship to `driver-sql`'s `applyLike`
*
* This is deliberately the same transform `SqlDriver.applyLike`
* (`packages/plugins/driver-sql/src/sql-driver.ts`) applies — same escaped
* character class, same three wildcard shapes, same bound `ESCAPE` — and its
* TSDoc points back here. It is a SECOND implementation on purpose, not an
* oversight:
*
* - `service-analytics` depends on no driver (see its `package.json`: only
* `@objectstack/core` and `@objectstack/spec`), and `applyLike` is a private
* method on a knex builder — it takes a builder and a field, not a string,
* so there is nothing importable even if the dependency existed.
* - Promoting it to a shared package would add a new public surface to
* `@objectstack/core` for three call sites inside one package. Not worth a
* new export until a fourth consumer outside this package needs it.
*
* What keeps the two from drifting is not these comments: it is
* `__tests__/like-metacharacter-escape.test.ts`, which asserts
* {@link escapeLikePattern} against `applyLike`'s expression character for
* character. A third hand-copy of this logic anywhere is the thing to refuse —
* import from here, or add a consumer to that test.
*/

/**
* Where the wildcard sits relative to the comparand. Named exactly as
* `driver-sql`'s `applyLike` names its `shape` parameter, so the two read alike:
* `contains` → `%v%`, `starts` → `v%`, `ends` → `%v`.
*/
export type LikeShape = 'contains' | 'starts' | 'ends';

/**
* The escape character every emitter in this package binds into its `ESCAPE`
* clause. A single backslash — the value `driver-sql` binds, and the default
* Postgres and MySQL already assume.
*/
export const LIKE_ESCAPE_CHAR = '\\';

/**
* Escape the LIKE metacharacters (`%`, `_`) and the escape character itself
* (`\`) so a comparand matches literally.
*
* Character for character the expression `driver-sql`'s `applyLike` uses; the
* shared test holds them to each other.
*/
export function escapeLikePattern(value: unknown): string {
return String(value).replace(/[\\%_]/g, '\\$&');
}

/**
* Build the LIKE pattern for one comparand: escaped, then wrapped in the
* wildcards `shape` calls for.
*
* The result MUST be bound together with {@link LIKE_ESCAPE_CHAR} as the
* predicate's `ESCAPE` argument — see the escaping-alone note in this file's
* header for what happens on SQLite otherwise.
*/
export function likePattern(shape: LikeShape, value: unknown): string {
const escaped = escapeLikePattern(value);
return shape === 'starts' ? `${escaped}%` : shape === 'ends' ? `%${escaped}` : `%${escaped}%`;
}
48 changes: 44 additions & 4 deletions packages/services/service-analytics/src/read-scope-sql.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

import type { FilterCondition } from '@objectstack/spec/data';
import { likePattern, LIKE_ESCAPE_CHAR } from './like-pattern.js';

/**
* Compile an RLS / tenant read-scope `FilterCondition` into a parameterized,
Expand Down Expand Up @@ -66,6 +67,22 @@ import type { FilterCondition } from '@objectstack/spec/data';
* canonical; {@link nullSafeNegationOperand} here is the same rewrite
* `sql-driver.ts` applies, so an analytics query and an ordinary `find()` scope
* the same rows.
*
* ## The LIKE family compares LITERALS (#5567)
*
* `_` is LIKE's single-character wildcard and `%` its multi-character one, so a
* comparand concatenated straight into a pattern position stops meaning what the
* author wrote: `{owner_name: {$contains: '_admin'}}` also admitted `xyadmin`,
* and `{$contains: '50%'}` also admitted `off 5012 now`. Every LIKE arm below
* therefore binds an ESCAPED pattern plus its escape character — see
* `like-pattern.ts` for the transform, for why the escape character is a bound
* value rather than a SQL literal, and for its correspondence with `driver-sql`'s
* `applyLike`.
*
* On THIS compiler that widening was the #5347 / #5324 shape again: a read scope
* admitting rows the policy did not is over-reach, not a degraded filter. Note
* the file was fail-closed everywhere else — the LIKE family was the one place an
* author's literal was silently reinterpreted rather than refused.
*/

const IDENT = /^[a-z_][a-z0-9_]*$/i;
Expand Down Expand Up @@ -212,6 +229,27 @@ function bind(params: unknown[], v: unknown): string {
return '?';
}

/**
* [#5567] Bind a LIKE pattern together with its escape character: `? ESCAPE ?`.
*
* Both are ordinary bound values, so this whole concern stays inside the
* predicate: `applyReadScope` (`native-sql-strategy.ts`) and `generateSql`
* (`objectql-strategy.ts`) rewrite `?` → `$N` while pushing the matching value
* from `params`, and they carry the escape character for free — neither consumer
* needed a change. A SQL literal `ESCAPE '\'` would have pushed the problem up a
* layer AND been unportable: MySQL strips one backslash inside a string literal,
* so the literal spelling differs per dialect while a bound value does not.
*
* The clause is not optional decoration. SQLite honours no default escape
* character, so the escaped pattern alone would search for a literal backslash
* there and match nothing — the two halves are one fix (see `like-pattern.ts`).
*/
function bindLike(params: unknown[], pattern: string): string {
// Left-to-right evaluation of the template puts the pattern in `params` before
// the escape character, which is the order the `?` appear.
return `${bind(params, pattern)} ESCAPE ${bind(params, LIKE_ESCAPE_CHAR)}`;
}

function compileOperator(col: string, op: string, val: unknown, field: string, params: unknown[]): string {
switch (op) {
case '$eq': return val === null ? `${col} IS NULL` : `${col} = ${bind(params, val)}`;
Expand All @@ -234,10 +272,12 @@ function compileOperator(col: string, op: string, val: unknown, field: string, p
if (!Array.isArray(val) || val.length !== 2) throw new Error(`[read-scope-sql] $between for "${field}" needs [min,max] (fail-closed).`);
return `${col} BETWEEN ${bind(params, val[0])} AND ${bind(params, val[1])}`;
}
case '$contains': return `${col} LIKE ${bind(params, `%${String(val)}%`)}`;
case '$notContains': return `${col} NOT LIKE ${bind(params, `%${String(val)}%`)}`;
case '$startsWith': return `${col} LIKE ${bind(params, `${String(val)}%`)}`;
case '$endsWith': return `${col} LIKE ${bind(params, `%${String(val)}`)}`;
// [#5567] The comparand is a LITERAL, so it is escaped and the escape
// character is bound with it. See {@link bindLike}.
case '$contains': return `${col} LIKE ${bindLike(params, likePattern('contains', val))}`;
case '$notContains': return `${col} NOT LIKE ${bindLike(params, likePattern('contains', val))}`;
case '$startsWith': return `${col} LIKE ${bindLike(params, likePattern('starts', val))}`;
case '$endsWith': return `${col} LIKE ${bindLike(params, likePattern('ends', val))}`;
case '$null': return val ? `${col} IS NULL` : `${col} IS NOT NULL`;
case '$exists': return val ? `${col} IS NOT NULL` : `${col} IS NULL`;
default:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
type NormalizedFilterNode,
} from './filter-normalizer.js';
import { compileScopedFilterToSql } from '../read-scope-sql.js';
import { likePattern, LIKE_ESCAPE_CHAR, type LikeShape } from '../like-pattern.js';
import { nextUtcCalendarDay } from '@objectstack/core';

/**
Expand Down Expand Up @@ -662,12 +663,17 @@ export class NativeSQLStrategy implements AnalyticsStrategy {
contains: 'LIKE', notContains: 'NOT LIKE',
startsWith: 'LIKE', endsWith: 'LIKE',
};
/** The LIKE pattern each string operator wraps its comparand in. */
const likePattern: Record<string, (v: string) => string> = {
contains: (v) => `%${v}%`,
notContains: (v) => `%${v}%`,
startsWith: (v) => `${v}%`,
endsWith: (v) => `%${v}`,
/**
* Where each string operator puts the wildcard. [#5567] The pattern itself is
* built by the shared `likePattern`, which ESCAPES the comparand — `_` and
* `%` are LIKE wildcards, so the old inline table quietly turned an author's
* literal into a pattern (`$contains: '_admin'` also matched `xyadmin`).
* `objectql-strategy.ts`'s `LIKE_SQL_OPS` carries the same table for the
* `/analytics/sql` echo of this statement; they move together.
*/
const likeShape: Record<string, LikeShape> = {
contains: 'contains', notContains: 'contains',
startsWith: 'starts', endsWith: 'ends',
};

// Null predicates and the LIKE family read the column as stored — the former
Expand All @@ -690,10 +696,15 @@ export class NativeSQLStrategy implements AnalyticsStrategy {

// The LIKE family reads the column as stored — a substring/prefix/suffix
// match is on the raw text — so it keeps the un-normalised reference.
const pattern = likePattern[operator];
if (pattern) {
params.push(pattern(values[0]));
return `${rawCol} ${sqlOp} $${params.length}`;
const shape = likeShape[operator];
if (shape) {
// [#5567] Escaped pattern AND an explicit `ESCAPE`, bound together: the
// escaping alone would search for a literal backslash on SQLite (no
// default escape character there), the clause alone would change nothing.
params.push(likePattern(shape, values[0]));
const patternRef = `$${params.length}`;
params.push(LIKE_ESCAPE_CHAR);
return `${rawCol} ${sqlOp} ${patternRef} ESCAPE $${params.length}`;
}

// A bare-day `lte` bound means "through that whole day" (#3777): compile
Expand Down
Loading
Loading