Skip to content

Issue #3003 : Add Output transform metrics transform - #8178

Merged
hansva merged 1 commit into
apache:mainfrom
mattcasters:issue-3003
Aug 31, 2026
Merged

Issue #3003 : Add Output transform metrics transform#8178
hansva merged 1 commit into
apache:mainfrom
mattcasters:issue-3003

Conversation

@mattcasters

Copy link
Copy Markdown
Contributor

Port Kettle Output steps metrics as a Hop-native Output transform metrics transform.

Fixes #3003

What

A local-engine generator that waits for listed sibling transform copies to finish, then emits one metrics row per copy (name, plugin id, line counters, rejected rows, duration). Incoming hops are not consumed and are treated as an error because they can deadlock the pipeline.

Why

Pipeline Logging / Execution Information cover project-wide monitoring, but they do not replace this in-pipeline transform for Kettle/PDI migrations.

How

  • New plugin plugins/transforms/transformsmetrics, plugin id TransformsMetrics (deprecated extra id StepsMetrics).
  • Metadata is @HopMetadataProperty with Hop transform* XML keys. The dialog uses grouped GuiCompositeWidgets tabs plus a TableView extra group.
  • Runtime follows BlockUntilTransformsFinish wait semantics, then emits metrics instead of passing rows through.
  • Kettle import maps StepsMetricsTransformsMetrics and step* tags → transform* (including the Kettle steplineswrittentfield typo). Stored output field values are left unchanged.

Tests

  • Unit: XML round-trip, getFields, check() (TransformsMetricsMetaTest)
  • Import: Kettle snippet rewrite (KettleImportTransformsMetricsTest)
  • Integration: 0110-output-transform-metrics with a golden dataset

Docs

User-manual page, sample pipeline, Transforms and Fields tab screenshots.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

@hansva
hansva merged commit a4f2c12 into apache:main Aug 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: Add the "Output steps metrics" transform to HOP

2 participants