Skip to content

HIVE-29430: Extract get partitions from HMSHandler#6311

Open
dengzhhu653 wants to merge 16 commits intoapache:masterfrom
dengzhhu653:HIVE-29430
Open

HIVE-29430: Extract get partitions from HMSHandler#6311
dengzhhu653 wants to merge 16 commits intoapache:masterfrom
dengzhhu653:HIVE-29430

Conversation

@dengzhhu653
Copy link
Member

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors Hive Metastore Server partition-related logic by extracting multiple “get partitions” code paths out of HMSHandler into a dedicated GetPartitionsHandler, and similarly extracting append-partition logic into AppendPartitionHandler. It also updates various tests to reflect the new exception/message behavior triggered by the refactor.

Changes:

  • Introduce GetPartitionsHandler to consolidate partition-fetching APIs and rewire multiple HMSHandler endpoints to use it.
  • Introduce AppendPartitionHandler and route append-partition requests through the handler framework.
  • Update tests and request argument plumbing (e.g., GetPartitionsArgs.order) to match the refactored behavior.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/GetPartitionsHandler.java New consolidated handler for partition retrieval logic across multiple APIs.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/AppendPartitionHandler.java New handler to encapsulate append-partition logic previously in HMSHandler.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/TAbstractBase.java Adds a minimal TBase implementation used for handler request bodies.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/AbstractRequestHandler.java Adds class-level suppression for rawtypes warnings used by handler registry.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/client/builder/GetPartitionsArgs.java Extends args with order to support ordered partition-name queries.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java Adds getMetaFilterHook() for handler use.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java Rewires many partition APIs to delegate to the new handlers; adjusts exception flow and logging.
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java Updates expected exception type(s) after refactor.
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java Updates expected listener input table name formatting.
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreMethods.java Updates expected exception type for partitions-by-names negative test.
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java Updates expected exception type and error message substring.
Comments suppressed due to low confidence (1)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3370

  • In get_partitions_pspec(), the catch block rethrows, but the finally block calls endFunction(..., null, tbl_name) with a null exception. This means endFunction listeners/audit won’t receive the failure cause for this method. Please capture the exception (similar to other HMSHandler methods) and pass it into endFunction.
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (1)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3370

  • In get_partitions_pspec(), the finally block always passes a null exception to endFunction. If an exception is thrown (and rethrown in the catch), endFunction listeners/metrics won't see the failure cause. Track the caught exception in a local variable (like other methods in this class) and pass it to endFunction.
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3375

  • get_partitions_pspec() now catches and rethrows exceptions, but the endFunction(...) call in the finally block always passes a null exception. This prevents MetaStoreEndFunctionListener(s) from seeing failures for this API. Capture the caught exception into a local variable and pass it through to endFunction to keep end-function auditing/metrics consistent with other HMS APIs.
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3375

  • In get_partitions_pspec(), endFunction is always invoked with a null exception argument. This means MetaStoreEndFunctionListener (and any other end-function consumers) cannot observe the actual failure cause when an exception is thrown inside the try block. Capture the thrown exception in a local variable and pass it to endFunction in the finally block (similar to the pattern used in most other HMSHandler methods).
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3328

  • In get_partitions_pspec, the finally block always calls endFunction(..., null, ...) even when an exception was thrown. This loses exception details for end-function listeners/metrics and can break tests that assert on listener context. Capture the thrown exception in a local Exception ex and pass it to endFunction.
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:3331

  • In get_partitions_pspec, exceptions are caught and rethrown, but endFunction(...) in the finally block always receives null for the exception argument. This drops exception details from audit/metrics/logging for failures. Track the caught exception in a local Exception ex variable and pass it to endFunction (and keep the success flag aligned with whether the call actually succeeded).
    } catch (Exception e) {
      throw handleException(e).throwIfInstance(NoSuchObjectException.class, MetaException.class).defaultMetaException();
    }
    finally {
      endFunction("get_partitions_pspec", partitionSpecs != null && !partitionSpecs.isEmpty(), null, tbl_name);
    }

standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/AbstractRequestHandler.java:224

  • AbstractRequestHandler now uses toString() as the log/status prefix (e.g., in getRequestStatus/cancelRequest/getResult). Since many handlers override toString() using fields initialized in beforeExecute(), toString() can be invoked before those fields are set (or while initialization is in-flight), which can throw NPE and break status polling / cancellation paths. Consider using a non-overridable prefix method (e.g., based only on class name + id) for status/log messages, or ensure handler toString() implementations are null-safe and do not dereference fields that are only set in beforeExecute().
  public RequestStatus getRequestStatus() throws TException {
    String logMsgPrefix = toString();
    if (future == null) {
      throw new IllegalStateException(logMsgPrefix + " hasn't started yet");
    }

    RequestStatus resp = new RequestStatus(id);
    if (future.isDone()) {
      resp.setFinished(true);
      resp.setMessage(logMsgPrefix + (future.isCancelled() ? " Canceled" : " Done"));
    } else {
      resp.setMessage(logMsgPrefix + " In-progress, state - " + getRequestProgress());
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants