Fixes 30868: stop Auto Classification tagging plain code columns as PII

[pmbrull]

Describe your changes:

Fixes #30868

A column named scenario_code holding SCN-125, SCN-113, … was auto-classified as PII.Sensitive at confidence 1.00. Two independent defects combine to cause it, and either one alone is enough to tag the column.

1. The cvv_pattern regex was not anchored. \b\d{3,4}\b matches a 3-4 digit run inside a value, so the 125 in SCN-125 matched. Changed to \A\d{3,4}\Z.

Not ^\d{3,4}$ — the recognizer sets regexFlags.multiline, under which ^..$ still matches a single line of a multi-line value. \A..\Z was the only anchor that closed that case in testing.

2. Context words were matched as substrings, not tokens. presidio_utils.enhance_using_context tested each context word with ctx_word.lower() in " ".join(context).lower(), where context is the column name split into parts. "cid" in "acid level" is true, so acid_level, incident_count and decoder_ring all matched the CVV context list. Extracted the comparison into context_matches(), which compares whole tokens; multi-word entries ("indian passport") keep substring semantics since they have no single token to compare against.

These stack because OpenMetadata's enhance_using_context sets a matching result's score straight to MAX_SCORE instead of Presidio's additive +0.35. That turns the weakest pattern in the set (0.5, "any 3-4 digits") into a certain match — past the 0.6 classification minimum and past the default workflow confidence of 80. I left that behaviour alone: changing it affects every recognizer, and both defects above are fixable without it.

Type of change:

• Bug fix

High-level design:

N/A — small change.

Tests:

Use cases covered

• A scenario_code / error_code column of SCN-125-style values is no longer tagged PII
• An acid_level / incident_count / decoder_ring column of bare 3-digit values is no longer tagged PII
• A cvv, security_code or card_verification_code column of bare 3-digit values is still tagged PII
• A credit_card_number column is still tagged PII
• A multi-line value whose second line is 3 digits is no longer tagged PII

Unit tests

• I added unit tests for the new/changed logic.
• Files updated: ingestion/tests/unit/pii/algorithms/test_presidio_utils.py
  • new TestContextMatches — whole-token matches, substring non-matches (acid/incident/decoder), multi-word entries, case-insensitivity
  • TestEnhanceUsingContext::test_context_word_that_is_only_a_substring_does_not_boost
  • TestEnhanceUsingContext::test_score_below_minimum_is_not_boosted — first coverage of the MIN_SCORE_FOR_ENHANCEMENT branch, which had none
• ingestion/tests/unit/pii + ingestion/tests/unit/metadata/pii: 235 passed (222 before, +13 new), 0 failures.

Backend integration tests

• Not applicable (no backend API changes).

Ingestion integration tests

• Not applicable — covered by unit tests above plus the end-to-end scoring run below.

Playwright (UI) tests

• Not applicable (no UI changes).

Manual testing performed

Replayed the real scoring path (TagAnalyzer → ScoreTagsForColumnService → ConflictResolver, Presidio + en_core_web_md) offline against the shipped recognizer definitions:

column	values	before	after
scenario_code	SCN-100 … SCN-125	1.00 ❌	none ✅
error_code	ERR-200 …	1.00 ❌	none ✅
acid_level	bare 3-digit	1.00 ❌	none ✅
decoder_ring	bare 3-digit	1.00 ❌	none ✅
incident_count	bare 3-digit	1.00 ❌	none ✅
note_code	"line one\n125"	1.00 ❌	none ✅
cvv	bare 3-digit	1.00 ✅	1.00 ✅
security_code	bare 3-digit	1.00 ✅	1.00 ✅
card_verification_code	bare 3-digit	1.00 ✅	1.00 ✅
credit_card_number	Luhn-valid PANs	0.90 ✅	0.90 ✅

Also ran a 20-column regression table covering every auto-classifiable tag (Person, Email, PhoneNumber, Address, Location, NRP, Gender, BirthDate, DateTime, CreditCardNumber, BankNumber, IBANCode, Crypto, IPAddress, URL, NationalID, VATCode, DriverLicense, MedicalLicense) at the default confidence of 80: 0 regressions.

Migration verified against a real tag row on PostgreSQL 15 and MySQL 8:

• updates exactly 1 row, only the intended tag
• recognizer count, recognizer order and the context list are preserved
• re-running updates 0 rows

Note on the MySQL migration: locating the recognizer with EXISTS (SELECT ... FROM JSON_TABLE(json, ...)) in the WHERE clause of an UPDATE does not correlate reliably — it evaluated correctly in a SELECT but silently matched 0 rows in the UPDATE as soon as the table had more than one row, which would have shipped a no-op migration. It uses a join instead, which is where the idempotency gate lives.

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• My PR is linked to a GitHub issue via Fixes #<issue-number> above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
• For UI changes: I attached a screen recording and/or screenshots above.
• I have added tests (unit / integration / Playwright as applicable) and listed them above.

Bug fix

• I have added a test that covers the exact scenario we are fixing.

Migration re-verified by running the committed statement against a live OpenMetadata database (the
dev PostgreSQL 15.18 instance, connected with the conf/openmetadata.yaml default credentials) rather
than only a restored fixture:

• UPDATE 1 on the first run, UPDATE 0 on a second run
• recognizer count, an md5 of the recognizer-name ordering, and the context-list length are all
  byte-identical before and after
• tag confirmed as the only table holding recognizer config; tag_usage stores applied labels and
  classification stores only classification-level autoClassificationConfig
• exactly two rows in the whole database carry a CvvRecognizer: PII.Sensitive (this PR) and
  General.CreditCardNumber (the Collate PR), so between the two migrations there is no overlap and
  no row is missed

After migrating, re-scoring a live scenario_code column of SCN-100 … SCN-125 against the
database's own tag definitions returns no tag, while cvv and security_code still score 1.00.

🤖 Generated with Claude Code

[github-actions]

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

[Copilot]

Pull request overview

This PR fixes a false-positive PII auto-classification path in ingestion by (1) anchoring the CVV regex so it only matches whole values and (2) tightening context-word matching so context hits are based on whole tokens (avoiding substring matches like cid in acid_level). It also includes SQL data migrations to update existing PII.Sensitive tag recognizer config in both Postgres and MySQL, plus unit tests covering the new context-matching behavior.

Changes:

• Anchor the CVV recognizer pattern from \b\d{3,4}\b to \A\d{3,4}\Z in the shipped tag/recognizer JSON and via DB migrations.
• Replace substring-based context-word matching with token-based matching via a new context_matches() helper.
• Add unit tests for context_matches() and additional enhance_using_context branches.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
openmetadata-service/src/main/resources/json/data/tags/piiTagsWithRecognizers.json	Anchors CVV regex in the default shipped recognizer configuration.
ingestion/src/metadata/pii/algorithms/presidio_utils.py	Adds context_matches() and switches context boosting to token-based matching.
ingestion/tests/unit/pii/algorithms/test_presidio_utils.py	Adds targeted unit tests for token-vs-substring context matching and enhancement edge cases.
bootstrap/sql/migrations/native/2.0.0/postgres/postDataMigrationSQLScript.sql	Updates existing Postgres tag JSON to the anchored CVV regex while preserving recognizer order.
bootstrap/sql/migrations/native/2.0.0/mysql/postDataMigrationSQLScript.sql	Updates existing MySQL tag JSON to the anchored CVV regex using JSON_TABLE + join for reliability.

Suppressed comments (1)

ingestion/src/metadata/pii/algorithms/presidio_utils.py:483

• In the enhance_using_context decorator wrapper, context_matches(rec.context, context) is recomputed once per result even though it depends only on the column context and recognizer context. Compute it once before the loop and reuse it to avoid repeated tokenization work.

            if context_matches(rec.context, context):

[github-actions]

🔴 Playwright Results — workflow failed

Validated commit 2d9423669a1dee29ef792d6af30260a43fbd620b in Playwright run 30828769541, attempt 2.

✅ 608 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 3 skipped · 🧰 0 lifecycle flaky

Pipeline and setup failures (1)

• Playwright performance gate Maximum shard-job elapsed before upload failed (target ≤ 1800 s) — exceeded on 1 shard(s): chromium-03 1884 s.

Performance

Blocking targets: ❌ unmet · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 1h 56m 16s

⏱️ Max setup 3m 15s · max shard execution 17m 47s · max shard-job elapsed before upload 31m 24s · reporting 5s

🌐 208.23 requests/attempt · 2.76 app boots/UI scenario · 8.25% common-shard skew

Optimization targets still in progress:

• Browser traffic was 208.23 requests per attempt (convergence target: fewer than 200).
• Application boot ratio was 2.76 per UI scenario (1760 boots / 638 scenarios; convergence target: at most 1).

Shard	Passed	Failed	Flaky	Skipped	Lifecycle failed	Lifecycle flaky
✅ Shard chromium-01	120	0	0	0	0	0
✅ Shard chromium-02	139	0	0	0	0	0
✅ Shard chromium-03	140	0	0	3	0	0
✅ Shard data-asset-rules-01	61	0	0	0	0	0
✅ Shard domain-isolation-01	14	0	0	0	0	0
✅ Shard global-state-01	34	0	0	0	0	0
✅ Shard ingestion-01	25	0	0	0	0	0
✅ Shard ingestion-02	34	0	0	0	0	0
✅ Shard reindex-01	2	0	0	0	0	0
✅ Shard search-01	10	0	0	0	0	0
✅ Shard search-rbac-01	29	0	0	0	0	0

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

[pmbrull]

Thanks — half of this is right, and I've pushed a fix for that half in 2d94236.

The crash risk is not real, on either line.

Line 483 (the enhance_using_context decorator) already returned early on a falsy rec.context, and that guard predates this PR:

if not rec.context or not context:
    return results

So rec.context was never reached in an empty state there.

Line 349 (ContextAwareUsBankRecognizer) has no such guard, but self.context cannot be None or empty. Presidio's UsBankRecognizer.__init__ coerces it:

context = context if context else self.CONTEXT

Verified against the actual factory:

factory default (context omitted)  -> self.context=['check', 'account', 'account#', 'acct', 'bank', 'save', 'debit']
factory context=None               -> self.context=['check', 'account', 'account#', 'acct', 'bank', 'save', 'debit']
direct  context=None               -> self.context=['check', 'account', 'account#', 'acct', 'bank', 'save', 'debit']
direct  context=[]                 -> self.context=['check', 'account', 'account#', 'acct', 'bank', 'save', 'debit']

Even hypothetically it would not be a regression from this PR: the code being replaced iterated the same attribute — any(ctx_word.lower() in context_lower for ctx_word in self.context).

"Compute the match once per call" is a fair point, and I've applied it.

context_matches() was called inside the per-result loop, re-joining and re-tokenizing the column name for every result. The match depends only on the recognizer's context list and the column name, neither of which changes inside the loop. Both sites now resolve it once and bail out early:

if not rec.context or not context or not context_matches(rec.context, context):
    return results

That also removes a level of nesting from the loop body. I added the explicit self.context guard on the UsBank path while I was there — it cannot trigger, but it makes the precondition local rather than inherited from a base class two libraries away.

No behaviour change: 235 unit tests still pass, and the false-positive set (scenario_code, error_code, acid_level, decoder_ring, incident_count, multi-line note_code), the true-positive set (cvv, security_code, card_verification_code, credit_card_number) and the 19-tag regression table are all identical before and after.

[gitar-bot]

Code Review ✅ Approved

Anchors the CVV pattern regex and refines context word matching to whole tokens, stopping plain code columns like scenario_code from being incorrectly tagged as PII. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source}

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud