Skip to content

[GLUTEN][VL] Re-enable the ignored "Velox Parquet Write" test in ORC suite - #12977

Open
yikf wants to merge 2 commits into
apache:mainfrom
yikf:reenable-velox-orc-native-parquet-write-test
Open

[GLUTEN][VL] Re-enable the ignored "Velox Parquet Write" test in ORC suite#12977
yikf wants to merge 2 commits into
apache:mainfrom
yikf:reenable-velox-orc-native-parquet-write-test

Conversation

@yikf

@yikf yikf commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

The "Velox Parquet Write" test in VeloxOrcDataTypeValidationSuite has been ignored since the Spark 3.4 test framework was introduced.

The identical test in VeloxParquetDataTypeValidationSuite is enabled and passing, the only reason the ORC-suite copy failed was that it did not drop the complex-type columns that Velox native write does not support.

How was this patch tested?

re-enable test.

Was this patch authored or co-authored using generative AI tooling?

Yes, Generated-by: Claude claude-opus-4-8

Copilot AI lite review requested due to automatic review settings September 8, 2026 03:39
@github-actions github-actions Bot added the VELOX label Sep 8, 2026
@yikf
yikf force-pushed the reenable-velox-orc-native-parquet-write-test branch from baa89d2 to f42609a Compare September 8, 2026 03:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change re-enables the test by applying the same passing logic used in the Parquet suite, with only a minor comment wording nit noted.

Pull request overview

Re-enables the previously ignored “Velox Parquet Write” test in VeloxOrcDataTypeValidationSuite by aligning it with the already-enabled Parquet-suite version, ensuring the test no longer includes complex-type columns that Velox native write doesn’t support.

Changes:

  • Switches the test from ignore to test so it runs in the ORC suite again.
  • Drops complex-type columns (array, struct, map) before writing Parquet with native writer enabled.
  • Adds a read-back + checkAnswer verification to validate write correctness.
File summaries
File Description
backends-velox/src/test/scala/org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite.scala Re-enables the Parquet write test and adjusts the input DataFrame to avoid unsupported complex types; verifies write/read correctness.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 8, 2026 06:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped to re-enabling a test with a straightforward fix and added validation, with only minor naming cleanup suggested.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

backends-velox/src/test/scala/org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite.scala:474

  • New local variables use snake_case (write_path, data_path), which is inconsistent with the rest of this suite’s Scala naming (e.g., executedPlan) and typical camelCase conventions. Renaming improves readability and avoids introducing a one-off style exception in this file.
  • Files reviewed: 1/1 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants