Feature Description
There is Atum Service: https://github.com/AbsaOSS/atum-service we use within our ecosystem to perform various metrics calculations against the processed data, such as record counts.
The EventGate is common integration point between our applications and can hold references to Atum measurements in the message; for this, we talked that the schema can be adjusted by creating a new field under the jobs, maybe atum_measurements or something like that.
This should be an optional field, basically a JSON, similar to additional_info which we did not selected for this because it can contain much more, and application-specific info. But the atum_measurements is relatively directed and well-formatted (the format is outside of the scope of this ticket and even EventGate).
Problem / Opportunity
Being informed about the record counts / measurements on the jobs' messages side can provide insight for our customers and convenient, consistent usage.
Acceptance Criteria
The job runs schema contains one more optional attribute for capturing atum measurements in some format.
Consider adding this into the PG DB for reference, same as additional_info attribute.
Feature Description
There is Atum Service: https://github.com/AbsaOSS/atum-service we use within our ecosystem to perform various metrics calculations against the processed data, such as record counts.
The EventGate is common integration point between our applications and can hold references to Atum measurements in the message; for this, we talked that the schema can be adjusted by creating a new field under the jobs, maybe
atum_measurementsor something like that.This should be an optional field, basically a JSON, similar to
additional_infowhich we did not selected for this because it can contain much more, and application-specific info. But theatum_measurementsis relatively directed and well-formatted (the format is outside of the scope of this ticket and even EventGate).Problem / Opportunity
Being informed about the record counts / measurements on the jobs' messages side can provide insight for our customers and convenient, consistent usage.
Acceptance Criteria
The job runs schema contains one more optional attribute for capturing atum measurements in some format.
Consider adding this into the PG DB for reference, same as
additional_infoattribute.