feat(event-stats): add named-query endpoint for runs/jobs detail - #228
tmikula-dev wants to merge 6 commits into
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
When the logic from infra PR: https://github.com/absa-group/cps-eventbus-gateway/pull/175 is deployed with this one. The endpoint can be reached via Postman and the output seems to be correct (data + pagination). |
oto-macenauer-absa
left a comment
oto-macenauer-absa
left a comment
There was a problem hiding this comment.
lgtm, I'd just add those additional queries for some flexibility
| return ReaderQueries( | ||
| get_stats=queries.get_stats.sql, # pylint: disable=no-member | ||
| get_stats_with_cursor=queries.get_stats_with_cursor.sql, # pylint: disable=no-member | ||
| get_runs_jobs_detail=queries.get_runs_jobs_detail.sql, # pylint: disable=no-member |
There was a problem hiding this comment.
Just making sure ... the name is correct and the dynamic name does not trigger mypy?
| ORDER BY j.internal_id DESC | ||
| LIMIT :lim; | ||
|
|
||
| -- name: get_runs_jobs_detail_with_cursor(ts_start, ts_end, cursor_id, lim) |
There was a problem hiding this comment.
Please add a aggregated query for groups of jobs by the status - "succeeded", "failed", "killed", "skipped" - status, count
Use case is: how many jobs failed for last month?
| LIMIT :lim; | ||
|
|
||
| -- name: get_runs_jobs_detail_with_cursor(ts_start, ts_end, cursor_id, lim) | ||
| SELECT r.event_id, r.job_ref, r.tenant_id, r.source_app, |
There was a problem hiding this comment.
would it make sense to add a simple generic aggregate query by one column?
use cases:
- count of jobs grouped by a source app
- count of jobs grouped by a tenant
the output would always be two columns, one is the key, the other is the count
2 participants
Overview
Adds a new
/stats/{topic_name}/query/{query_name}endpoint that executes predefined, registry-driven named queries against PostgreSQL (starting withruns_jobs_detail, reproducing the existing Qlik runs/jobs dashboard feed) with keyset pagination, alongside supporting unit/integration tests, shared response-building utilities, and several pylint/typing cleanups (duplicate-code removal, tighterPaginationtyping, unused-import and config fixes).Release Notes
/stats/{topic_name}/query/{query_name}endpoint (HandlerNamedQuery) supporting predefined named queries with keyset paginationruns_jobs_detailnamed query reproducing the Qlik runs/jobs dashboard feedRelated
Closes #116
Infra Issue: https://github.com/absa-group/cps-eventbus-gateway/issues/174