Core: Upgrade Jetty to 12.1.5 by nastra · Pull Request #10837 · apache/iceberg

nastra · 2024-08-01T13:49:12Z

Now that we fixed #10338 and moved to JDK 17 we're finally able to upgrade Jetty and the Servlet API to the latest version

build.gradle

core/src/test/java/org/apache/iceberg/rest/RESTCatalogServlet.java

gradle/libs.versions.toml

github-actions · 2024-11-10T00:16:21Z

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

nastra · 2026-01-13T15:55:06Z

Looks like Spark tests are failing because Spark internally still depends on an older Jetty version:


jakarta/servlet/SingleThreadModel
java.lang.NoClassDefFoundError: jakarta/servlet/SingleThreadModel
	at org.sparkproject.jetty.servlet.ServletHolder.setServlet(ServletHolder.java:173)
	at org.sparkproject.jetty.servlet.ServletHolder.<init>(ServletHolder.java:120)
	at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:121)
	at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:69)
	at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:253)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:521)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076)
	at org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015)
	at scala.Option.getOrElse(Option.scala:201)
	at org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923)
	at org.apache.iceberg.spark.TestBase.startMetastoreAndSpark(TestBase.java:83)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.ClassNotFoundException: jakarta.servlet.SingleThreadModel
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	... 16 more


jakarta.servlet.SingleThreadModel
java.lang.ClassNotFoundException: jakarta.servlet.SingleThreadModel
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	at org.sparkproject.jetty.servlet.ServletHolder.setServlet(ServletHolder.java:173)
	at org.sparkproject.jetty.servlet.ServletHolder.<init>(ServletHolder.java:120)
	at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:121)
	at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:69)
	at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:253)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:521)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076)
	at org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015)
	at scala.Option.getOrElse(Option.scala:201)
	at org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923)
	at org.apache.iceberg.spark.TestBase.startMetastoreAndSpark(TestBase.java:83)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

manuzhang · 2026-01-21T04:34:22Z

Can the jetty depended by Iceberg tests be separated from that by Spark? I don't think iceberg-spark-runtime needs to include jetty dependency from Iceberg.

manuzhang · 2026-02-03T15:00:24Z

@nastra Are you still working on this?

nastra · 2026-02-03T15:31:46Z

@nastra Are you still working on this?

I'm not actively working on this right now

manuzhang · 2026-02-04T02:38:08Z

@nastra I can pick this up if you don't have time currently.

nastra · 2026-02-04T05:56:34Z

@manuzhang sure, just go ahead and open a separate PR

manuzhang · 2026-02-13T06:11:37Z

This work is continued in #15232, please help review @nastra

nastra · 2026-02-13T15:39:19Z

@manuzhang #15232 contained too many unrelated changes and was also missing a bunch of other places where the UI config had to be disabled. However, the idea of disabling the UI is correct and I've added those here and added you as the author.

@singhpk234 @amogh-jahagirdar we should merge the PR using "Rebase and merge" so that the individual commits get merged separately

nastra · 2026-02-13T15:44:16Z

aws/src/integration/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java

        CreateMultipartUploadRequest.builder().bucket(BUCKET).key("random/multipart-key").build());
  }

+  @SuppressWarnings("removal")


this is needed because GzipHandler is now deprecated. We'll be switching to Jetty's new Compression API in #15043

nastra · 2026-02-13T15:45:50Z

...4/spark-extensions/src/jmh/java/org/apache/iceberg/spark/MergeCardinalityCheckBenchmark.java

    this.spark =
        SparkSession.builder()
-            .config("spark.ui.enabled", false)
+            .config(TestBase.DISABLE_UI)


The main issue with Spark is that Spark brings its own (older) Jetty version for the UI, which conflicts with the Jetty version we're using. The downside is that we need to disable the UI pretty much everywhere

Spark only upgraded to jetty 12 after 4.2, so I believe it's worthwhile. As a follow-up, I plan to create a util method to start SparkSession such that we don't need to copy the same config everywhere.

nastra · 2026-02-16T07:28:51Z

@singhpk234 @amogh-jahagirdar can you guys please review this one?

…Jetty version

nastra · 2026-02-16T16:23:06Z

spark/v3.4/build.gradle

    testImplementation project(path: ':iceberg-hive-metastore', configuration: 'testArtifacts')
    testImplementation project(path: ":iceberg-spark:iceberg-spark-${sparkMajorVersion}_${scalaVersion}", configuration: 'testArtifacts')
-    testImplementation (project(path: ':iceberg-open-api', configuration: 'testFixturesRuntimeElements')) {
-      transitive = false


this was initially done because there were conflicts. However, given that we don't have any conflicts anymore we can remove this

manuzhang · 2026-02-17T06:28:04Z

@nastra I prefer to merge this in one commit since the first commit doesn't pass test and we can't revert the commits separately.

github-actions bot added core build labels Aug 1, 2024

nastra commented Aug 1, 2024

View reviewed changes

build.gradle Show resolved Hide resolved

nastra commented Aug 1, 2024

View reviewed changes

core/src/test/java/org/apache/iceberg/rest/RESTCatalogServlet.java Show resolved Hide resolved

nastra commented Aug 1, 2024

View reviewed changes

gradle/libs.versions.toml Outdated Show resolved Hide resolved

nastra changed the title ~~Core: Upgrade Jetty and Servlet API~~ Core: Upgrade Jetty to 12.0.12 and Servlet API Aug 2, 2024

nastra changed the title ~~Core: Upgrade Jetty to 12.0.12 and Servlet API~~ Core: Upgrade Jetty to 12.0.12 Aug 2, 2024

nastra force-pushed the bump-jetty-and-servlet-api branch from b0cf998 to 0f7e32c Compare August 2, 2024 12:19

github-actions bot added the stale label Nov 10, 2024

nastra added not-stale and removed stale labels Nov 10, 2024

nastra force-pushed the bump-jetty-and-servlet-api branch from 0f7e32c to 1a902a0 Compare January 13, 2026 12:36

nastra changed the title ~~Core: Upgrade Jetty to 12.0.12~~ Core: Upgrade Jetty to 12.1.5 Jan 13, 2026

nastra requested review from amogh-jahagirdar and singhpk234 January 13, 2026 12:38

nastra force-pushed the bump-jetty-and-servlet-api branch from 1a902a0 to eb3162e Compare January 13, 2026 12:42

github-actions bot added AWS OPENAPI labels Jan 13, 2026

nastra mentioned this pull request Jan 13, 2026

Build: Switch Jetty to use new Compression API for GZIP #15043

Draft

nastra force-pushed the bump-jetty-and-servlet-api branch from eb3162e to 24c1805 Compare January 13, 2026 13:54

github-actions bot added the spark label Jan 13, 2026

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from f33bd66 to 24c1805 Compare January 13, 2026 15:54

nastra marked this pull request as draft January 13, 2026 15:58

manuzhang mentioned this pull request Feb 12, 2026

Core: Upgrade Jetty to 12.1.5 #15232

Closed

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from 6b15c1a to c9d14a6 Compare February 13, 2026 15:35

nastra marked this pull request as ready for review February 13, 2026 15:41

nastra commented Feb 13, 2026

View reviewed changes

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from c9a3108 to 49cdaf1 Compare February 13, 2026 17:45

nastra and others added 2 commits February 16, 2026 17:17

Build: Bump Jetty to 12.1.5

cf1a8a4

Spark: Disable UI configs to avoid dependency conflicts with Spark's …

b4e9819

…Jetty version

nastra force-pushed the bump-jetty-and-servlet-api branch from 49cdaf1 to b4e9819 Compare February 16, 2026 16:20

nastra commented Feb 16, 2026

View reviewed changes

manuzhang approved these changes Feb 17, 2026

View reviewed changes

Conversation

nastra commented Aug 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 10, 2024

Uh oh!

nastra commented Jan 13, 2026

Uh oh!

manuzhang commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manuzhang commented Feb 3, 2026

Uh oh!

nastra commented Feb 3, 2026

Uh oh!

manuzhang commented Feb 4, 2026

Uh oh!

nastra commented Feb 4, 2026

Uh oh!

manuzhang commented Feb 13, 2026

Uh oh!

nastra commented Feb 13, 2026

Uh oh!

nastra Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

nastra Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

manuzhang Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

nastra commented Feb 16, 2026

Uh oh!

nastra Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

manuzhang commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

nastra commented Aug 1, 2024 •

edited

Loading

manuzhang commented Jan 21, 2026 •

edited

Loading