Skip to content

[controller] Sanitize table configuration representations - #19236

Draft
xiangfu0 wants to merge 1 commit into
apache:masterfrom
xiangfu0:xiangfu0/table-config-redaction
Draft

[controller] Sanitize table configuration representations#19236
xiangfu0 wants to merge 1 commit into
apache:masterfrom
xiangfu0:xiangfu0/table-config-redaction

Conversation

@xiangfu0

Copy link
Copy Markdown
Contributor

Summary

  • Apply one masking policy to table-config GET APIs, materialized-view details, and SHOW CREATE output.
  • Cover sensitive ingestion, storage, provider, custom-map, JAAS, key, and credential-bearing URI values while preserving unresolved environment placeholders.
  • Preserve GET-edit-PUT behavior by restoring unchanged masked values only when they map unambiguously to the same stored property and consumer identity.
  • Require explicit get_znode permission for raw ZooKeeper reads and avoid including configuration values in related diagnostics.

Root cause

Table configuration representations were produced through several independent paths without a shared masking and update policy. Some paths also used getters that apply variable replacement before rendering, while update handling had no safe interpretation for values returned as masks.

Update semantics

  • ***** retains the corresponding unresolved stored value only when the property and consumer identity are unchanged and unambiguous.
  • Literal replacements remain intentional updates.
  • New, moved, ambiguous, or reassociated masks are rejected instead of being persisted.
  • Environment placeholders remain placeholders and are not resolved for display or masked-value restoration.

How to reproduce

  1. Create a test table with synthetic sensitive values in representative ingestion, storage, JAAS, key, and URI properties, plus an environment placeholder and a benign property.
  2. Fetch it through either table-config GET API as a table READ principal and render it with SHOW CREATE.
  3. Verify sensitive values are represented as *****, the placeholder and benign property are unchanged, and raw ZooKeeper retrieval is denied without get_znode.
  4. Edit only the benign property in the returned configuration and PUT it back.
  5. Verify the stored sensitive values are retained; then verify an orphaned or reassociated mask is rejected.

Validation

  • 224 targeted tests passed across SPI masking and restoration, controller APIs, READ-only authorization, raw-read permission checks, SHOW CREATE, materialized-view metadata and scheduling, DDL emission, authentication, and task diagnostics.
  • Post-rebase smoke runs passed: SPI 28/28 and controller/API/SHOW/MV 85/85.
  • spotless:apply, license:format, checkstyle:check, and license:check passed for all affected modules.
  • Direct affected-module test-compile passed on JDK 25 with -Xlint:all and no warnings on added lines.
  • git diff --check passed.

The full reactor test-compile -am gate remains blocked in unchanged pinot-segment-local by a pre-existing zstd-jni annotation classpath issue. Direct compilation of every modified module passes.

@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.00365% with 351 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.79%. Comparing base (cb89c0e) to head (ca39cb8).

Files with missing lines Patch % Lines
...ot/spi/config/table/TableConfigRedactionUtils.java 67.85% 188 Missing and 163 partials ⚠️

❗ There is a different number of reports uploaded between BASE (cb89c0e) and HEAD (ca39cb8). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (cb89c0e) HEAD (ca39cb8)
unittests 2 1
unittests2 1 0
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19236      +/-   ##
============================================
- Coverage     66.97%   57.79%   -9.19%     
+ Complexity     1417        7    -1410     
============================================
  Files          3453     2660     -793     
  Lines        218858   160237   -58621     
  Branches      34787    26463    -8324     
============================================
- Hits         146591    92605   -53986     
+ Misses        60556    59675     -881     
+ Partials      11711     7957    -3754     
Flag Coverage Δ
integration 100.00% <ø> (+100.00%) ⬆️
integration1 100.00% <ø> (?)
integration2 0.00% <ø> (ø)
java-25 57.79% <68.00%> (-9.19%) ⬇️
lane-a 100.00% <ø> (+100.00%) ⬆️
lane-b 0.00% <ø> (ø)
temurin 57.79% <68.00%> (-9.19%) ⬇️
unittests 57.79% <68.00%> (-9.19%) ⬇️
unittests1 57.79% <68.00%> (+0.07%) ⬆️
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiangfu0
xiangfu0 force-pushed the xiangfu0/table-config-redaction branch from 9d9d140 to ca39cb8 Compare August 13, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants