HIVE-29705: Remediate critical CVEs in hive-druid-handler (HIVE-25013)#6589
Open
saihemanth-cloudera wants to merge 1 commit into
Open
HIVE-29705: Remediate critical CVEs in hive-druid-handler (HIVE-25013)#6589saihemanth-cloudera wants to merge 1 commit into
saihemanth-cloudera wants to merge 1 commit into
Conversation
asf-ci-hive
added
tests pending
tests failed
and removed
tests pending
tests failed
labels
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR remediates critical CVEs in hive-druid-handler by removing the legacy Netty 3-based Druid HTTP client stack, updating/shading dependency versions (e.g., Netty 4, Jackson, async-http-client), and making the Druid handler optional in default distributions.
Changes:
- Replaced Druid’s Netty 3 HTTP client usage with a new Apache HttpClient-based wrapper (
HiveDruidHttpClient+ request/response wrappers). - Updated platform dependency versions (Netty 4.1.133.Final, Jackson 2.18.8) and pinned
async-http-client2.15.0 for shading. - Made
hive-druid-handleropt-in in distribution packaging via a newdruid-handlerMaven profile.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| storage-api/pom.xml | Bumps shared Netty 4 version used by the build. |
| standalone-metastore/pom.xml | Updates Jackson and Netty versions for standalone metastore. |
| pom.xml | Updates Jackson/Netty versions and adds async-http-client version/dependency management. |
| packaging/pom.xml | Moves hive-druid-handler into an opt-in packaging profile. |
| druid-handler/pom.xml | Updates dependencies/exclusions/relocations to remove Netty 3 and shade patched deps. |
| druid-handler/src/test/org/apache/hadoop/hive/druid/serde/TestDruidSerDe.java | Updates tests/mocks to use the new HTTP client interface. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidQueryRecordReader.java | Switches query execution from Druid Netty client to HiveDruidHttpClient. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/security/RetryIfUnauthorizedResponseHandler.java | Removes Netty 3 response retry handler (obsolete with new client). |
| druid-handler/src/java/org/apache/hadoop/hive/druid/security/ResponseCookieHandler.java | Removes Netty 3 cookie handler (replaced by CookieManager usage). |
| druid-handler/src/java/org/apache/hadoop/hive/druid/security/KerberosHttpClient.java | Removes Netty 3 Kerberos HTTP client wrapper (replaced by new client). |
| druid-handler/src/java/org/apache/hadoop/hive/druid/security/DruidKerberosUtil.java | Makes Kerberos helper methods public for use by new HTTP client package. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/io/DruidQueryBasedInputFormat.java | Updates request creation to use HiveDruidHttpRequest. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/http/HiveDruidHttpResponse.java | Adds new response wrapper replacing Druid Netty 3 holders. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/http/HiveDruidHttpRequest.java | Adds new request wrapper replacing Druid Netty 3 Request. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/http/HiveDruidHttpClient.java | Introduces new Apache HttpClient-based client (async wrapper + Kerberos/cookie logic). |
| druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandlerUtils.java | Updates request submission/redirect handling to use new client + wrappers. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java | Replaces lifecycle-based Netty client setup with new client + shutdown hook. |
| druid-handler/src/java/org/apache/hadoop/hive/druid/DruidKafkaUtils.java | Updates Druid supervisor calls to use new client + wrappers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+104
to
+108
| try (CloseableHttpResponse response = httpClient.execute(buildHttpRequest(current))) { | ||
| int statusCode = response.getStatusLine().getStatusCode(); | ||
| storeCookies(current.getUrl().toURI(), response); | ||
| byte[] body = EntityUtils.toByteArray(response.getEntity()); | ||
|
|
Comment on lines
+39
to
+43
| import java.io.InputStream; | ||
| import java.net.CookieManager; | ||
| import java.net.HttpCookie; | ||
| import java.net.URI; | ||
| import java.security.PrivilegedExceptionAction; |
| .equals(HttpResponseStatus.BAD_REQUEST)) { | ||
| } else if (response.getStatusCode() == HiveDruidHttpResponse.SC_NOT_FOUND | ||
| || response.getStatusCode() == HiveDruidHttpResponse.SC_BAD_REQUEST) { | ||
| LOG.debug("No Kafka Supervisor found for datasource[%s]", dataSourceName); |
| .equals(HttpResponseStatus.BAD_REQUEST)) { | ||
| } else if (response.getStatusCode() == HiveDruidHttpResponse.SC_NOT_FOUND | ||
| || response.getStatusCode() == HiveDruidHttpResponse.SC_BAD_REQUEST) { | ||
| LOG.info("No Kafka Supervisor found for datasource[%s]", dataSourceName); |
Comment on lines
+90
to
+93
| public InputStream executeStream(HiveDruidHttpRequest request) throws IOException { | ||
| HiveDruidHttpResponse response = innerExecute(request); | ||
| return new java.io.ByteArrayInputStream(response.getBody()); | ||
| } |
Comment on lines
+68
to
+72
| this.httpClient = HttpClients.custom() | ||
| .setDefaultRequestConfig(RequestConfig.custom() | ||
| .setConnectTimeout(readTimeoutMs) | ||
| .setSocketTimeout(readTimeoutMs) | ||
| .setConnectionRequestTimeout(readTimeoutMs) |
Comment on lines
+76
to
+80
| this.executor = Executors.newCachedThreadPool(r -> { | ||
| Thread t = new Thread(r, "HiveDruidHttpClient"); | ||
| t.setDaemon(true); | ||
| return t; | ||
| }); |
Comment on lines
+921
to
+923
| LOG.info("Creating HiveDruidHttpClient with {}ms read timeout, kerberos={}", | ||
| readTimeout.toStandardDuration().getMillis(), kerberosEnabled); | ||
| return new HiveDruidHttpClient((int) readTimeout.toStandardDuration().getMillis(), kerberosEnabled); |
Comment on lines
33
to
36
| <dependency> | ||
| <groupId>com.fasterxml.jackson.dataformat</groupId> | ||
| <artifactId>jackson-dataformat-smile</artifactId> | ||
| </dependency> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Replace Druid's Netty 3 HTTP client with Apache HttpClient to remove
io.netty:netty 3.x from the shaded druid-handler JAR (CVE-2019-20444).
Force patched async-http-client 2.15.0 and Jackson 2.18.8 into the shade,
bump platform Netty 4 to 4.1.133.Final, and make hive-druid-handler opt-in
via the druid-handler packaging profile so default distributions avoid
legacy transitive CVEs.
Why are the changes needed?
Avoid CVEs
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit tests