Skip to content

Reduce time spent on copy and hard delete and validation tests#1706

Draft
shangyian wants to merge 3349 commits intoDataJunction:mainfrom
shangyian:speedup
Draft

Reduce time spent on copy and hard delete and validation tests#1706
shangyian wants to merge 3349 commits intoDataJunction:mainfrom
shangyian:speedup

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

Summary

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

agorajek and others added 30 commits May 22, 2025 15:26
Co-authored-by: GitHub Actions Bot <>
…on#1390)

* Add graphql schema generation script and seed with generated schema

* Include schema.graphql in packaging
* Add /cubes endpoint to get metadata on all cubes

* Change query param to catalog
…tion#1388)

* Fix the derived expression of count distinct metrics to use the count distinct argument

* Add separate treatment of measures for distinct quantifier aggregations

* Add support to complex expressions inside distinct-agg metrics

* Refactor names and add tests for resolving metric components

* Cleanup

* More naming cleanup

* Add graphql naming refactor

* Update comments and update GraphQL schema
* Add pagination to /cubes endpoint

* don't pull inactive nodes for /cubes/... endpoints
* Add endpoint to list available dialects

* Add support for GraphQL querying of dialects
Co-authored-by: GitHub Actions Bot <>
* Fix an issue where renaming dimensions for measures SQL creates should include both columns

* Fix tests

* Refactor to make it cleaner
…1396)

* Add option to delete materializations by node version.

* Lint and commments addressed.

* Comment addressed plus added node_revision_id to Materialization output.
* Replaces use of Depends(get_session) in get_context() with session middleware, since the former doesn't support teardown in Strawberry.

* Add test for db session middleware

* Fix tests
…ataJunction#1402)

* When a node is deactivated, its materializations should be as well

* Fix tests
Co-authored-by: GitHub Actions Bot <>
* Add filtering on columns for node revisions in GQL

* Add tests for resolvers
* Add docker compose with superset

* Remove comments

* add password init

* Add secret key

* docker compose file that works with superset

* Fix

* Fix names

* Fix

---------

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: GitHub Actions Bot <>
Fix the ignore files for unit test workflow.
* Fix test action.

* Add UI code filter.
* Skip deactivated nodes in namespace export.

* Adjust unit test.
* Add support for reader clusters

* Add ability to switch between reader and writer cluster across REST and GraphQL calls

* Fix tests

* Add test for reader/writer session

* Add example .env

* Fix

* Fix tests

* Set database connection max overflow to 100

* Clean up

* Fix graphql tests

* Fix

* Fix

* Fix
Co-authored-by: GitHub Actions Bot <>
* Add additional database indexes for performance

* Update db migration

* lint

* Fix tests

* Remove extraneous

* Remove unnecessary indexes

* Comments

* Remove and fix some unnecessary indexes

* Fix
* Make the dimensions dag query more efficient

* Make the dimensions dag query more efficient
shangyian and others added 26 commits January 18, 2026 22:00
Co-authored-by: GitHub Actions Bot <actions@github.com>
Co-authored-by: GitHub Actions Bot <actions@github.com>
Co-authored-by: GitHub Actions Bot <actions@github.com>
Co-authored-by: GitHub Actions Bot <actions@github.com>
Co-authored-by: GitHub Actions Bot <actions@github.com>
* Fix npm release by upgrading node

* Fix package.json url to point to the correct repository

* Fix release workflow to fetch previous version tag and generate release notes manually from git log
Co-authored-by: GitHub Actions Bot <actions@github.com>
…ion (DataJunction#1685)

* Support derived metrics that reference derived metrics: including finding available dimensions for such metrics and generating SQL (metrics, measures) for multi-level derived metrics.

* Fix
Co-authored-by: GitHub Actions Bot <actions@github.com>
Co-authored-by: GitHub Actions Bot <actions@github.com>
* Add support for ORDER BY and LIMIT to v3 SQL gen

* Fix
* Add toggle for compile SQL

* For metric nodes, remove Columns and Materializations tabs and add type info

* Fix required dimensions display in he UI

* Added Preview tab for metrics where tab click redirects to query planner

* Fix display of SQL to wrap rather than extending the width of the page

* Speed up with add'l caching

* Speed up with add'l caching

* Fix

* Fix
Co-authored-by: GitHub Actions Bot <actions@github.com>
* Add query runner UI

* Add more tests

* Fix test coverage

* Lint

* Remove availability population for SQL gen

* Fix all tests

* Fix all tests

* Fix coverage

* Fix engine determining process

* Fix
Co-authored-by: GitHub Actions Bot <actions@github.com>
* Switch python client to use v3 SQL builder and add plan() option for someone trying to view measures SQL via cli

* Fix tests

* Fix tests

* fix

* fix

* Fix tests

* Fix
Co-authored-by: GitHub Actions Bot <actions@github.com>
* Remove forwarding of query service client headers

* Fix an issue where the selected engine/dialect should be comaptible with the availability state's engine/dialect

* Remove filtered headers from service client header forward

* Resolve dialect and engine in a single function for SQL gen

* Fix tests
* Add dj data command to the cli

* Add docs and fix tests

* Add tests for push dryrun
@netlify
Copy link
Copy Markdown

netlify bot commented Jan 24, 2026

Deploy Preview for thriving-cassata-78ae72 ready!

Name Link
🔨 Latest commit 682768c
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6975125694e16c00083b4249
😎 Deploy Preview https://deploy-preview-1706--thriving-cassata-78ae72.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants