fix: parse DateTime to filter by timestamp on search page

[karl-power]

Summary

On the search page, including or excluding a value from a DateTime column (e.g. Timestamp, type DateTime64(9)) generated SQL that ClickHouse rejects:

Timestamp NOT IN ('2026-06-16T15:35:16.731000000Z')
-- DB::Exception: Cannot convert string '2026-06-16T15:35:16.731000000Z' to type DateTime64(9)

The filter value was emitted as a bare string literal, but ClickHouse will not implicitly cast an ISO-8601 string to DateTime64, so the entire query failed and the filter was unusable. This is reachable from the grid cell popover (the "Include"/"Exclude" actions in DBRowTableFieldWithPopover) — both polarities were broken.

filtersToQuery now wraps DateTime column values in parseDateTime64BestEffort('<value>', 9) instead of emitting a plain literal, reusing the exact pattern the codebase already uses for DateTime values in useRowWhere.tsx. The result for the case above is Timestamp NOT IN (parseDateTime64BestEffort('2026-06-16T15:35:16.731000000Z', 9)), which is valid SQL.

Screenshots or video

Before	After
before.mov	after.mov

How to test on Vercel preview

Preview routes: /search

Steps:

1. Go the search page and apply timestamp filtering (exclude and include).

References

• Linear Issue: Closes HDX-4506
• Related PRs:

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	Jun 18, 2026 8:08am
hyperdx-storybook	Ready	Preview, Comment	Jun 18, 2026 8:08am

[changeset-bot]

🦋 Changeset detected

Latest commit: e5df625

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@hyperdx/common-utils	Patch
@hyperdx/app	Patch
@hyperdx/api	Patch
@hyperdx/otel-collector	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

• Cross-layer change: touches frontend (packages/app) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats

• Production files changed: 6
• Production lines changed: 161 (+ 404 in test files, excluded from tier calculation)
• Branch: karl/filter-search-on-timestamp
• Author: karl-power

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[greptile-apps]

Greptile Summary

This PR fixes a ClickHouse query failure when including or excluding DateTime/DateTime64 column values on the search page. Previously, filtersToQuery emitted bare string literals inside IN/NOT IN clauses (e.g. Timestamp NOT IN ('2026-06-16T...')), which ClickHouse rejects with a type-cast error; now it wraps each value in the appropriate parse function (parseDateTime64BestEffort, parseDateTimeBestEffort, toDate32, or toDate) based on the column's ClickHouse type.

• filtersToQuery (in common-utils) gains a dateTimeColumns option; when provided, each string value for a DateTime/Date column is wrapped in a type-matching ClickHouse expression instead of a bare quoted literal.
• extractInClauses (in searchFilters.tsx) now strips those wrapper expressions back off before comma-splitting, preserving the parse → serialize → re-parse round-trip.
• useDateTimeColumns is extracted as a shared hook in useMetadata.tsx, replacing the previously duplicated useMemo derivation in DBSearchPage.tsx and DBSearchPageFilters.tsx; ActiveFilterPills also receives the map to display DateTime values in formatted locale/timezone form.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to DateTime column filter generation and its inverse parse step, both covered by new unit tests including adversarial round-trip cases.

The fix correctly wraps DateTime values in type-matched ClickHouse expressions across all four date-type variants (DateTime64, DateTime, Date32, Date) including Nullable/LowCardinality wrappers. The symmetric unwrapping regex in extractInClauses preserves the full serialize → URL → re-hydrate round-trip, and the adversarial test value confirms SQL-escaped embedded quotes survive intact. The useDateTimeColumns hook eliminates the previously duplicated derivation, and the dateTimeColumns dependency is correctly threaded through all call sites and useCallback dependency arrays.

No files require special attention.

Important Files Changed

Filename	Overview
packages/common-utils/src/filters.ts	Adds dateTimeValueExpr to produce type-correct ClickHouse expressions for Date/DateTime column values in filtersToQuery; regex ordering and fallback are correct for all known ClickHouse date variants including Nullable/LowCardinality wrappers.
packages/app/src/searchFilters.tsx	Adds unwrapping regex in extractInClauses to strip parseDateTime64BestEffort/parseDateTimeBestEffort/toDate32/toDate wrappers before comma-splitting, correctly preserving the round-trip; useSearchPageFilterState now accepts and forwards dateTimeColumns.
packages/app/src/hooks/useMetadata.tsx	Adds useDateTimeColumns hook that derives a Map<columnName, chType> from ColumnMeta[] using the existing filterColumnMetaByType utility, eliminating the previously duplicated inline useMemo.
packages/app/src/components/ActiveFilterPills.tsx	Adds optional dateTimeColumns prop; pill display values for DateTime columns are formatted via useFormatTime while raw values are preserved for SQL generation, editing, and copy. A stable empty map constant prevents unnecessary useMemo invalidation when the prop is omitted.
packages/app/src/components/DBSearchPageFilters.tsx	Replaces the inline useMemo for DateTime column detection with the new useDateTimeColumns hook and passes the map to filtersToQuery and its dependency array.
packages/app/src/DBSearchPage.tsx	Reorders initialization so dateTimeColumns (via useDateTimeColumns) is derived before useSearchPageFilterState is called, then threads the map into both the filter state hook and ActiveFilterPills.
packages/common-utils/src/tests/filters.test.ts	Comprehensive new test suite covering all four date-type wrappers, multi-value lists, stringifyKeys bypass, boolean passthrough, Nullable-type column, and the no-op baseline (absent dateTimeColumns).
packages/app/src/tests/searchFilters.test.ts	Round-trip tests for the full serialize→deserialize cycle, including the adversarial value that looks like a wrapper suffix — demonstrates the regex correctly handles SQL-escaped embedded quotes.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant User
    participant DBRowTable
    participant filterState as useSearchPageFilterState
    participant filtersToQuery
    participant ClickHouse
    participant parseQuery

    User->>DBRowTable: Click Exclude on DateTime value
    DBRowTable->>filterState: setFilterValue('Timestamp', '2026-06-16T15:35:16Z', 'exclude')
    filterState->>filtersToQuery: "filtersToQuery(filters, { dateTimeColumns })"
    Note over filtersToQuery: dateTimeValueExpr('DateTime64(9)', ...)
    filtersToQuery-->>filterState: Timestamp NOT IN (parseDateTime64BestEffort('2026-06-16T...', 9))
    filterState->>ClickHouse: Execute query with wrapped DateTime literal
    ClickHouse-->>filterState: Results (no cast error)

    Note over filterState,parseQuery: URL round-trip / re-hydration
    filterState->>parseQuery: parseQuery(searchQuery)
    Note over parseQuery: extractInClauses unwraps wrapper back to plain quoted string
    parseQuery-->>filterState: FilterState with bare string values

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant User
    participant DBRowTable
    participant filterState as useSearchPageFilterState
    participant filtersToQuery
    participant ClickHouse
    participant parseQuery

    User->>DBRowTable: Click Exclude on DateTime value
    DBRowTable->>filterState: setFilterValue('Timestamp', '2026-06-16T15:35:16Z', 'exclude')
    filterState->>filtersToQuery: "filtersToQuery(filters, { dateTimeColumns })"
    Note over filtersToQuery: dateTimeValueExpr('DateTime64(9)', ...)
    filtersToQuery-->>filterState: Timestamp NOT IN (parseDateTime64BestEffort('2026-06-16T...', 9))
    filterState->>ClickHouse: Execute query with wrapped DateTime literal
    ClickHouse-->>filterState: Results (no cast error)

    Note over filterState,parseQuery: URL round-trip / re-hydration
    filterState->>parseQuery: parseQuery(searchQuery)
    Note over parseQuery: extractInClauses unwraps wrapper back to plain quoted string
    parseQuery-->>filterState: FilterState with bare string values

Loading

_{Reviews (4): Last reviewed commit: "handle DateTime columns" | Re-trigger Greptile}

[github-actions]

E2E Test Results

✅ All tests passed • 200 passed • 3 skipped • 1348s

Status	Count
✅ Passed	200
❌ Failed	0
⚠️ Flaky	4
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

[pulpdrew]

It's working well with the Timestamp column on the default schema, but not the TimestampTime column:

[karl-power]

It's working well with the Timestamp column on the default schema, but not the TimestampTime column:

@pulpdrew It should be working on TimestampTime now too.

[pulpdrew]

Current implementation is a good improvement!

However, is there a way to use the types of the columns being returned by the query, rather than the types of the columns on the table? That way aliases and non-column expressions expressions can also be filters. Otherwise maybe we should consider trying not to show the add-to-filters button for cells that we likely can't filter on (could be future work):