feat(scorecard): add catalog module to validate fields (starting with a required attributes provider) by fullsend-ai-coder[bot] · Pull Request #4258 · redhat-developer/rhdh-plugins

fullsend-ai-coder · 2026-08-11T21:45:49Z

Add new scorecard backend module that checks catalog entity metadata attributes against configurable rules. The CatalogMetadataMetricProvider evaluates entity fields (e.g., metadata.title, spec.lifecycle) and maps field states to configurable status strings via a three-tier status mapping merge (check-level > options-level > hardcoded defaults).

The module supports:

Required attribute checks (field exists, non-empty)
Value whitelist checks (field must be one of accepted values)
Per-check entity filters (e.g., only Components, only Templates)
Configurable status mapping for field states (exists, empty,
emptyString, emptyArray, missed) and specific values
Automatic threshold rule generation from status mappings

New files:

CatalogMetadataConfig.ts: config parsing and status mapping merge
CatalogMetadataMetricProvider.ts: MetricProvider<'number'> impl
CatalogMetadataMetricProvider.test.ts: 53 tests covering config
parsing, field resolution, status evaluation, filter matching,
threshold generation, and calculateMetrics scenarios

Closes #4256

Post-script verification

Branch is not main/master (agent/4256-catalog-metadata-module)
Secret scan passed (gitleaks — 9ff73957b1e59ceca89032ba15af079e17c47087..HEAD)
Pre-commit hooks passed (authoritative run on runner)
Tests ran inside sandbox

rhdh-gh-app · 2026-08-11T21:46:21Z

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-catalog	workspaces/scorecard/plugins/scorecard-backend-module-catalog	patch	`v0.0.0`

codecov · 2026-08-11T21:47:07Z

Codecov Report

❌ Patch coverage is 94.69027% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.17%. Comparing base (71104e8) to head (cb86a43).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4258      +/-   ##
==========================================
+ Coverage   59.09%   59.17%   +0.08%     
==========================================
  Files        2458     2461       +3     
  Lines       98057    98283     +226     
  Branches    27341    27428      +87     
==========================================
+ Hits        57945    58159     +214     
- Misses      39830    39842      +12     
  Partials      282      282

Flag	Coverage Δ		*Carryforward flag
adoption-insights	`84.55% <ø> (ø)`		Carriedforward from 71104e8
ai-integrations	`68.29% <ø> (ø)`		Carriedforward from 71104e8
app-defaults	`69.79% <ø> (ø)`		Carriedforward from 71104e8
augment	`46.67% <ø> (ø)`		Carriedforward from 71104e8
boost	`77.63% <ø> (ø)`		Carriedforward from 71104e8
bulk-import	`72.79% <ø> (ø)`		Carriedforward from 71104e8
cost-management	`13.55% <ø> (ø)`		Carriedforward from 71104e8
dcm	`67.21% <ø> (ø)`		Carriedforward from 71104e8
e2e-adoption-insights	`60.00% <ø> (ø)`		Carriedforward from 71104e8
e2e-intelligent-assistant	`46.74% <ø> (ø)`		Carriedforward from 71104e8
extensions	`56.59% <ø> (ø)`		Carriedforward from 71104e8
global-floating-action-button	`71.18% <ø> (ø)`		Carriedforward from 71104e8
global-header	`66.50% <ø> (ø)`		Carriedforward from 71104e8
homepage	`47.50% <ø> (ø)`		Carriedforward from 71104e8
install-dynamic-plugins	`59.95% <ø> (ø)`		Carriedforward from 71104e8
intelligent-assistant	`75.42% <ø> (ø)`		Carriedforward from 71104e8
konflux	`91.98% <ø> (ø)`		Carriedforward from 71104e8
lightspeed	`69.02% <ø> (ø)`		Carriedforward from 71104e8
mcp-integrations	`83.40% <ø> (ø)`		Carriedforward from 71104e8
orchestrator	`71.31% <ø> (ø)`		Carriedforward from 71104e8
quickstart	`63.74% <ø> (ø)`		Carriedforward from 71104e8
sandbox	`79.56% <ø> (ø)`		Carriedforward from 71104e8
scorecard	`86.70% <94.69%> (+0.45%)`	⬆️
theme	`88.14% <ø> (ø)`		Carriedforward from 71104e8
translations	`5.12% <ø> (ø)`		Carriedforward from 71104e8
x2a	`79.20% <ø> (ø)`		Carriedforward from 71104e8

*This pull request uses carry forward flags. Click here to find out more.

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71104e8...cb86a43. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

christoph-jerolimov · 2026-08-11T22:05:28Z

/fs-fix

change new package.json version to 0.0.0
add a changeset for the new package (as patch to release a 0.0.1)
update api reports with yarn tsc && yarn build:api-reports:only

fullsend-ai-coder · 2026-08-11T22:07:08Z

🤖 Finished Fix · ✅ Success · Started 10:07 PM UTC · Completed 10:20 PM UTC

Commit: 323bf69 · View workflow run →

- Change package version from 1.0.0 to 0.0.0 for initial release - Add patch changeset to release as 0.0.1 - Generate API report via yarn build:api-reports:only Addresses review feedback on #4258

fullsend-ai-coder · 2026-08-11T22:20:53Z

🔧 Fix agent — iteration 1 (human-triggered)

Addressed all 3 human-requested changes: set package version to 0.0.0, added patch changeset, and generated API report.

Fixed (3):

package.json version should be 0.0.0 (workspaces/scorecard/plugins/scorecard-backend-module-catalog-metadata/package.json): Changed version from 1.0.0 to 0.0.0 for initial release of the new package
add changeset for new package as patch (workspaces/scorecard/.changeset/add-catalog-metadata-module.md): Added patch changeset for @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-catalog-metadata to release as 0.0.1
update API reports (workspaces/scorecard/plugins/scorecard-backend-module-catalog-metadata/report.api.md): Generated API report by running yarn tsc && yarn build:api-reports:only from the scorecard workspace

Tests: passed

_{Updated by fullsend fix agent}

christoph-jerolimov · 2026-08-11T22:26:43Z

/fs-fix

add a README.md to the new module. It should explain the feature and include multiple app-config configuration examples

fullsend-ai-coder · 2026-08-11T22:28:17Z

🤖 Finished Fix · ✅ Success · Started 10:28 PM UTC · Completed 10:34 PM UTC

Commit: f278fa0 · View workflow run →

fullsend-ai-coder · 2026-08-11T22:34:23Z

🔧 Fix agent — iteration 2 (human-triggered)

Added README.md to the catalog-metadata module with feature explanation and 6 app-config configuration examples as requested.

Fixed (1):

add a README.md to the new module with feature explanation and configuration examples: Created comprehensive README.md for the catalog-metadata module with feature overview, prerequisites, installation instructions, and six app-config configuration examples: (1) basic required attribute check, (2) value whitelist check, (3) multiple checks with different entity kinds, (4) options-level status mapping defaults, (5) multi-field entity filter, (6) full configuration with schedule and per-metric thresholds. Also documented field resolution, status evaluation, three-tier merge, automatic threshold generation, available metrics, and schedule configuration.

Tests: passed

_{Updated by fullsend fix agent}

Add comprehensive README documenting the catalog-metadata backend module with feature overview, installation instructions, and six app-config configuration examples covering required attribute checks, value whitelist checks, multi-kind entity filters, options-level status mapping defaults, per-check overrides, and full configuration with schedule and per-metric thresholds. Addresses review feedback on #4258

christoph-jerolimov · 2026-08-12T04:56:18Z

/fs-fix align code style, run yarn prettier:fix

fullsend-ai-coder · 2026-08-12T04:57:29Z

🤖 Finished Fix · ✅ Success · Started 4:57 AM UTC · Completed 5:10 AM UTC

Commit: 7f9c480 · View workflow run →

Run `yarn prettier:fix` to align README markdown table columns per project code style conventions. Addresses review feedback on #4258

fullsend-ai-coder · 2026-08-12T05:10:30Z

🔧 Fix agent — iteration 3 (human-triggered)

Ran yarn prettier:fix as requested. One file was reformatted: plugins/scorecard-backend-module-catalog-metadata/README.md. Three markdown tables had their column separators aligned to consistent widths. All prettier checks now pass.

Fixed (1):

align code style with prettier: Ran yarn prettier:fix in the scorecard workspace. Prettier aligned markdown table columns in the catalog-metadata module README.md (3 tables reformatted with consistent column widths).

Tests: passed

_{Updated by fullsend fix agent}

christoph-jerolimov · 2026-08-12T05:51:44Z

/fs-fix

Add an example "catalog-metadata-scorecard-without-title" without metadata title and "catalog-metadata-scorecard-with-title" with title to the catalog entity. Both should have an annotation scorecard.example: catalog-metadata. And then a configuration for the new catalog-metadata module to the app-config.yaml with a filter for that annotation.

fullsend-ai-coder · 2026-08-12T05:52:59Z

🤖 Fix · ⚠️ Cancelled · Started 5:52 AM UTC · Ended 6:00 AM UTC

Commit: 6820bab · View workflow run →

christoph-jerolimov · 2026-08-12T06:00:11Z

/fs-fix

Rename the module to just "catalog". The folder needs to be changed from workspaces/scorecard/plugins/scorecard-backend-module-catalog-metadata to workspaces/scorecard/plugins/scorecard-backend-module-catalog
Rename CatalogMetadataConfig to CatalogRequiredAttributesConfig and CatalogMetadataMetricProvider to CatalogRequiredAttributesMetricProvider.
getProviderDatasourceId should be catalog and getProviderId prefix should be also just catalog.
All examples and readmes needs to use the catalog instead of catalogMetadata reference as well.
Add or update the an example to "catalog-scorecard-without-title" without metadata title and "catalog-metadata-scorecard-with-title" with title to the catalog entity. Both should have an annotation scorecard.example: catalog-metadata. And then a configuration for the new catalog-metadata module to the app-config.yaml with a filter for that annotation.

fullsend-ai-coder · 2026-08-12T06:01:53Z

🤖 Finished Fix · ✅ Success · Started 6:01 AM UTC · Completed 6:21 AM UTC

Commit: 6820bab · View workflow run →

fullsend-ai-coder · 2026-08-12T06:21:19Z

🔧 Fix agent — iteration 4 (human-triggered)

Addressed all 5 human review findings: renamed module folder and package from catalog-metadata to catalog, renamed classes to CatalogRequiredAttributesConfig and CatalogRequiredAttributesMetricProvider, updated all provider/datasource/metric IDs to use catalog prefix, updated all README and config.d.ts references, and added example entities with catalog module configuration. All 53 tests pass.

Fixed (5):

Rename module folder from catalog-metadata to catalog (workspaces/scorecard/plugins/scorecard-backend-module-catalog): Renamed folder from scorecard-backend-module-catalog-metadata to scorecard-backend-module-catalog, updated package.json name to @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-catalog
Rename CatalogMetadataConfig and CatalogMetadataMetricProvider (workspaces/scorecard/plugins/scorecard-backend-module-catalog/src/metricProviders): Renamed CatalogMetadataConfig.ts to CatalogRequiredAttributesConfig.ts, CatalogMetadataMetricProvider.ts to CatalogRequiredAttributesMetricProvider.ts, renamed types and classes accordingly, updated all imports
Update provider datasource and provider IDs to use catalog prefix (workspaces/scorecard/plugins/scorecard-backend-module-catalog/src/metricProviders/CatalogRequiredAttributesMetricProvider.ts): Changed getProviderDatasourceId from catalogMetadata to catalog, getProviderId from catalogMetadata.requiredAttributes to catalog.requiredAttributes, metric ID prefix from catalogMetadata to catalog
Update all examples and READMEs to use catalog instead of catalogMetadata (workspaces/scorecard/plugins/scorecard-backend-module-catalog/README.md): Updated README title, installation instructions, all YAML config examples, metric ID references, provider ID and datasource references. Updated config.d.ts config path from catalogMetadata to catalog
Add example entities and catalog module config to app-config.yaml (workspaces/scorecard/examples/components): Added catalog-scorecard-without-title.yaml (no metadata.title) and catalog-metadata-scorecard-with-title.yaml (with metadata.title), both with scorecard.example: catalog-metadata annotation. Added catalog.requiredAttributes config to app-config.yaml with a title check for Component kind. Updated all-scorecards-location.yaml to include new entities

Tests: passed

_{Updated by fullsend fix agent}

- Rename folder from scorecard-backend-module-catalog-metadata to scorecard-backend-module-catalog - Rename CatalogMetadataConfig to CatalogRequiredAttributesConfig - Rename CatalogMetadataMetricProvider to CatalogRequiredAttributesMetricProvider - Update getProviderDatasourceId to 'catalog' and getProviderId to 'catalog.requiredAttributes' - Update metric ID prefix from catalogMetadata to catalog - Update config path from catalogMetadata to catalog - Update all README examples and config.d.ts references - Add example entities catalog-scorecard-without-title and catalog-metadata-scorecard-with-title with scorecard.example annotation - Add catalog module configuration to app-config.yaml Addresses review feedback on #4258

christoph-jerolimov · 2026-08-12T06:34:41Z

/fs-fix

rebase and resolve conflicts

fullsend-ai-coder · 2026-08-12T06:36:08Z

🤖 Finished Fix · ✅ Success · Started 6:36 AM UTC · Completed 6:50 AM UTC

Commit: 0493669 · View workflow run →

Add new scorecard backend module that checks catalog entity metadata attributes against configurable rules. The CatalogMetadataMetricProvider evaluates entity fields (e.g., metadata.title, spec.lifecycle) and maps field states to configurable status strings via a three-tier status mapping merge (check-level > options-level > hardcoded defaults). The module supports: - Required attribute checks (field exists, non-empty) - Value whitelist checks (field must be one of accepted values) - Per-check entity filters (e.g., only Components, only Templates) - Configurable status mapping for field states (exists, empty, emptyString, emptyArray, missed) and specific values - Automatic threshold rule generation from status mappings New files: - CatalogMetadataConfig.ts: config parsing and status mapping merge - CatalogMetadataMetricProvider.ts: MetricProvider<'number'> impl - CatalogMetadataMetricProvider.test.ts: 53 tests covering config parsing, field resolution, status evaluation, filter matching, threshold generation, and calculateMetrics scenarios Closes #4256

- Change package version from 1.0.0 to 0.0.0 for initial release - Add patch changeset to release as 0.0.1 - Generate API report via yarn build:api-reports:only Addresses review feedback on #4258

Add comprehensive README documenting the catalog-metadata backend module with feature overview, installation instructions, and six app-config configuration examples covering required attribute checks, value whitelist checks, multi-kind entity filters, options-level status mapping defaults, per-check overrides, and full configuration with schedule and per-metric thresholds. Addresses review feedback on #4258

Run `yarn prettier:fix` to align README markdown table columns per project code style conventions. Addresses review feedback on #4258

- Rename folder from scorecard-backend-module-catalog-metadata to scorecard-backend-module-catalog - Rename CatalogMetadataConfig to CatalogRequiredAttributesConfig - Rename CatalogMetadataMetricProvider to CatalogRequiredAttributesMetricProvider - Update getProviderDatasourceId to 'catalog' and getProviderId to 'catalog.requiredAttributes' - Update metric ID prefix from catalogMetadata to catalog - Update config path from catalogMetadata to catalog - Update all README examples and config.d.ts references - Add example entities catalog-scorecard-without-title and catalog-metadata-scorecard-with-title with scorecard.example annotation - Add catalog module configuration to app-config.yaml Addresses review feedback on #4258

Rebased onto main to incorporate the code-coverage module merge (PR #3476) and other recent changes. Resolved conflicts in yarn.lock (regenerated via yarn install) and all-scorecards-location.yaml (kept both code-coverage and catalog example entries). Addresses rebase request on #4258

fullsend-ai-coder · 2026-08-12T06:50:45Z

🔧 Fix agent — iteration 5 (human-triggered)

Rebased onto main, resolved conflicts in yarn.lock and all-scorecards-location.yaml. Regenerated yarn.lock via yarn install. Tests pass (53/53), build succeeds.

Fixed (1):

rebase and resolve conflicts: Rebased 5 PR commits onto origin/main. Resolved merge conflicts in workspaces/scorecard/yarn.lock (regenerated via yarn install to include both the new code-coverage module from main and the catalog module from this PR) and workspaces/scorecard/examples/all-scorecards-location.yaml (kept both code-coverage and catalog example entries in alphabetical order). All 53 tests pass, build succeeds, secret scan clean.

Tests: passed

_{Updated by fullsend fix agent}

sonarqubecloud · 2026-08-12T06:52:13Z

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

christoph-jerolimov

Lgtm! Will create follow ups for e2e tests.

fullsend-ai-coder Bot requested review from a team, Eswaraiahsapram, christoph-jerolimov and its-mitesh-kumar as code owners August 11, 2026 21:45

fullsend-ai-coder Bot added the ready-for-review Agent PR ready for human review label Aug 11, 2026

fullsend-ai-coder Bot requested a review from a team as a code owner August 11, 2026 21:45

github-actions Bot added the workspace/scorecard label Aug 11, 2026

christoph-jerolimov changed the title ~~feat(#4256): add catalog-metadata module for scorecard~~ feat(scorecard): add catalog-metadata to validate fields Aug 11, 2026

fullsend-ai-coder Bot added a commit that referenced this pull request Aug 12, 2026

style(#4256): align markdown table formatting with prettier

6820bab

Run `yarn prettier:fix` to align README markdown table columns per project code style conventions. Addresses review feedback on #4258

christoph-jerolimov changed the title ~~feat(scorecard): add catalog-metadata to validate fields~~ feat(scorecard): add catalog module to validate fields (starting with a required attributes provider) Aug 12, 2026

fullsend-ai-coder Bot added 6 commits August 12, 2026 06:39

fix(#4256): set initial version to 0.0.0, add changeset and API report

8687831

- Change package version from 1.0.0 to 0.0.0 for initial release - Add patch changeset to release as 0.0.1 - Generate API report via yarn build:api-reports:only Addresses review feedback on #4258

style(#4256): align markdown table formatting with prettier

f38f2ac

Run `yarn prettier:fix` to align README markdown table columns per project code style conventions. Addresses review feedback on #4258

fullsend-ai-coder Bot force-pushed the agent/4256-catalog-metadata-module branch from 0493669 to cb86a43 Compare August 12, 2026 06:50

christoph-jerolimov approved these changes Aug 12, 2026

View reviewed changes

fullsend-ai-triage Bot mentioned this pull request Aug 12, 2026

Create new catalog module for scorecard (for custom entity integrity checks) #4256

Open

Uh oh!

Conversation

fullsend-ai-coder Bot commented Aug 11, 2026

Post-script verification

Uh oh!

rhdh-gh-app Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changed Packages

Uh oh!

codecov Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

christoph-jerolimov commented Aug 11, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 11, 2026

🔧 Fix agent — iteration 1 (human-triggered)

Uh oh!

christoph-jerolimov commented Aug 11, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 11, 2026

🔧 Fix agent — iteration 2 (human-triggered)

Uh oh!

christoph-jerolimov commented Aug 12, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026

🔧 Fix agent — iteration 3 (human-triggered)

Uh oh!

christoph-jerolimov commented Aug 12, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christoph-jerolimov commented Aug 12, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026

🔧 Fix agent — iteration 4 (human-triggered)

Uh oh!

christoph-jerolimov commented Aug 12, 2026

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Aug 12, 2026

🔧 Fix agent — iteration 5 (human-triggered)

Uh oh!

sonarqubecloud Bot commented Aug 12, 2026

Quality Gate passed

Uh oh!

christoph-jerolimov left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rhdh-gh-app Bot commented Aug 11, 2026 •

edited

Loading

codecov Bot commented Aug 11, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 11, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 11, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 12, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 12, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 12, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Aug 12, 2026 •

edited

Loading