Update dependency org.apache.tika:tika-core to v4 by solrbot · Pull Request #4853 · apache/solr

solrbot · 2026-09-01T01:02:59Z

This PR contains the following updates:

Package	Type	Update	Change
org.apache.tika:tika-core (source)	dependencies	major	`3.3.1` → `4.0.0`

Release Notes

apache/tika (org.apache.tika:tika-core)

`v4.0.0`

Compare Source

`v3.3.2`

Compare Source

Configuration

📅 Schedule: (UTC)

Branch creation
- "before 9am on the first day of the month"
Automerge
- At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot

epugh · 2026-09-01T21:02:13Z

Seems early enough that we can move to tika 4?

janhoy · 2026-09-01T21:27:14Z

Commonmark for Markdown parsing is now part of tika-core. Our extraction handler currently does not rely on MD, but explicitly requests xml from tika-server. So we could attempt excluding these three new jar deps and see if it will fly. There may be some part of core depending on these classes even if we don't use them explicitly so maybe not worth the risk?

epugh · 2026-09-01T21:30:08Z

@tballison somebody moved my cheese ;-).

solrbot · 2026-09-01T23:14:40Z

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

⚠️ Warning: custom changes will be lost.

…xml. This is mostly a "hey, look, I got us to tika 4", but it feels like its still trying to push Tika 4 into a tika 3 approach, when Tika wants to be different.

janhoy · 2026-09-03T20:35:38Z

The 3.3.x line remains supported and continues to receive bug fixes and dependency upgrades

I'd say we stay on 3.x for Solr 10, and work on exploring how to best utilize Tika 4 on main only, i.e. no backport. to 10.x. Of course, if we get to a stable back-compat impl of Tika 4 on main, it can be back-ported without risk. Even if Tika gains features like AI stuff, we are not obliged to use those :-)

tballison · 2026-09-03T20:42:15Z

On phone with @epugh now, lol. Let me know if I can help, or if I've done enough damage. 🤣

epugh · 2026-09-03T21:52:31Z

Umm... at least locally the bats test and unit tests passed!!! So maybe this isn't the lift I thought it was?

epugh · 2026-09-03T21:53:25Z

The 3.3.x line remains supported and continues to receive bug fixes and dependency upgrades

I'd say we stay on 3.x for Solr 10, and work on exploring how to best utilize Tika 4 on main only, i.e. no backport. to 10.x. Of course, if we get to a stable back-compat impl of Tika 4 on main, it can be back-ported without risk. Even if Tika gains features like AI stuff, we are not obliged to use those :-)

That works for me... As @tballison pointed out, it's literally a .0.0 version!

Copilot

🟡 Changes recommended

License validation will fail on a stale checksum, while error and partial-extraction handling can produce incorrect request outcomes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Upgrades Solr Cell to Tika 4 and adapts its server integration to Tika’s new endpoints, metadata keys, and per-request configuration.

Changes:

Updates Tika and Docker images to 4.0.0, including dependencies and licenses.
Adds multipart parser configuration and password handling.
Updates extraction tests and documentation for Tika 4 behavior.

File summaries

File	Description
`gradle/libs.versions.toml`	Updates Tika to 4.0.0.
`solr/modules/extraction/gradle.lockfile`	Locks Tika and CommonMark dependencies.
`TikaServerExtractionBackend.java`	Implements Tika 4 request and response handling.
`TikaServerParser.java`	Supports renamed recursive metadata content keys.
`ExtractionRequest.java`	Adds per-request Tika configuration.
`ExtractingParams.java`	Defines the new configuration parameter.
`ExtractingDocumentLoader.java`	Passes configuration from request parameters.
`TikaServerExtractionBackendTest.java`	Tests configuration, passwords, and Tika 4 behavior.
`TikaServerContainerRule.java`	Runs the Tika 4 container with optional configuration.
`ExtractingRequestHandlerTikaServerTest.java`	Enables per-request configuration in integration tests.
`ExtractingRequestHandlerTestAbstract.java`	Updates expected Tika metadata keys.
`tika-server-config.json`	Enables test-time per-request configuration.
`test_extraction.bats`	Updates the packaging test container.
`indexing-with-tika.adoc`	Documents Tika 4 and parser configuration.
`tutorial-diy.adoc`	Updates the tutorial container version.
`tika-core-4.0.0.jar.sha1`	Adds the Tika checksum.
`commonmark-LICENSE-BSD.txt`	Adds the CommonMark license.
`commonmark-NOTICE.txt`	Adds the CommonMark notice file.
`commonmark-0.30.0.jar.sha1`	Adds the CommonMark checksum.
`commonmark-ext-gfm-tables-0.30.0.jar.sha1`	Adds the tables extension checksum.
`commonmark-ext-gfm-strikethrough-0.30.0.jar.sha1`	Adds the strikethrough extension checksum.
`PR#4853-update-org-apache-tika-tika-core.yml`	Records the dependency update.

Review details

Files reviewed: 21/22 changed files
Comments generated: 5
Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@@ -0,0 +1 @@
+f370d8619de9e19329b2cc117943c83468c2ff9f


+    if (code == 422 && !request.tikaServerRecursive) {
+      PushbackInputStream peekable = new PushbackInputStream(responseStream, 1);
+      int firstByte = peekable.read();
+      if (firstByte == -1) {
+        throw new SolrException(


              .tikaServerRecursive(tikaserverRecursive)
              .tikaServerTimeoutSeconds(tikaTimeoutSecs)
              .tikaServerRequestHeaders(Map.of())
+              .tikaServerConfigJson(params.get(ExtractingParams.TIKASERVER_CONFIG_JSON))


+   * have {@code allowPerRequestConfig=true} set, or the request is rejected with 403. Ignored for
+   * recursive (tikaserver.recursive) requests, since TikaServer has no XML-output variant of
+   * /rmeta/config.


+          // Tika 4.x renamed its metadata keys under a single lowercase tk: prefix (TIKA-4816)
+          "fmap.tk:parsed-by",
          "ignored_parser",
-          "fmap.X-TIKA:Parsed-By",
+          "fmap.tk:parsed-by-full-set",
          "ignored_parser",


tballison · 2026-09-04T20:00:32Z

+    // that). A request that extracted nothing at all (e.g. a wrong password) also gets 422, but
+    // with an empty body -- treat that case as the failure it is instead of a silent empty
+    // "success". Peek the first byte to tell the two apart.
+    if (code == 422 && !request.tikaServerRecursive) {


We moved to default single file per-fork processing. We now have 429 as a backpressure signal, if a request can't find an active worker in 30s(?), you'll get 429.

tballison · 2026-09-04T20:09:29Z

+
+[IMPORTANT]
+====
+`tikaserver.config` requires your Tika Server to be started with `allowPerRequestConfig: true` in its own JSON configuration (under the `server` section).


This sentence by itself suggests that you have to have this. I'd recommend editing so there's an "if you want per file config, you'll need this". Ignore this comment if tikaserver.config is saying exactly that. :/

tballison · 2026-09-04T20:10:03Z

+====

-NOTE: In earlier versions of Solr Cell you could supply Tika configuration directly to Solr. This is no longer possible.
+`tikaserver.config` is combined with any password resolved from `resource.password` or `passwordsFile` (see <<Indexing Encrypted Documents>>) into a single request to Tika Server, and is only supported for non-recursive extraction (`tikaserver.recursive=false`, the default); Tika Server has no way to accept per-request configuration for recursive extraction while also returning the XHTML content Solr Cell needs.


POST /rmeta/config part "file": <document> part "config": {"basic-content-handler-factory":{"type":"XML"}, "pdf-parser":{"ocr":{"strategy":"NO_OCR"}}, "simple-password-provider":{"password":"..."}}

should work. This is less than ideal. My gut says that we should go with symmetry between /rmeta/config/xml and /tika/config/xml; but I now remember why we chose this path. I need to chat with the team to figure out whether to fix this for 4.1.0. Apologies for the noise.

https://issues.apache.org/jira/browse/TIKA-4881

tballison · 2026-09-04T20:11:12Z

-        assertTrue(c.contains("embedded:image0.jpg"));
-        assertEquals(
-            "org.apache.tika.parser.DefaultParser", md.getFirst("X-TIKA:Parsed-By-Full-Set"));
+        // Tika 4.x renamed its metadata keys under a single lowercase tk: prefix (TIKA-4816)


these 🤖 comments are useful for PR review, but there's no need for history, is there?

janhoy · 2026-09-04T20:19:21Z

+A raw JSON object sent as the per-request parser configuration for Tika Server (e.g., `{"pdf-parser":{"ocr":{"strategy":"NO_OCR"}}}`).
+See <<Parser-Specific Properties>> below for details and an important security note: this requires `allowPerRequestConfig=true` on the Tika Server, which is off by default.
+
+Example: `tikaserver.config={"pdf-parser":{"ocr":{"strategy":"NO_OCR"}}}`


What triggered you to enable this? Can we not just tell users to configure their Tika Server?

I view extraction handler as a low-end solution for folks who have very simple extraction needs and just want it embedded in solr. If you hare in the business of really heavy extraction you're probably running a custom Tika cluster on the application side and inject already extracted text in plain solr documents?

There is a cost associated with every additional config we need to document, support, deprecate if tika changes format etc. If you give a real-life example of its utility, something we believe will be truly useful, then I want to hear.

The above allows fine-grained control over ocr on PDFs. I agree with your point about the user base for the extraction handler.

If you don't want to allow configs or deal with documentation, you can cut it all out.

For the intermediate user, as long as you are loading an initialization config, they can figure this out on their own.

Out of the box, no config, and off you go.

tballison

LGTM. Couple of documentation issues. Fix incoming in 4.1.0.

On documentation, my bot recommends adding:

Sizing / provisioning
  https://tika.apache.org/docs/4.0.x/pipes/cpu-sizing.html
  The formula: numClients × 2 + 2 ≤ hostCores, and why forks get -XX:ActiveProcessorCount auto-injected.

  What numClients actually controls (the one to lead with)
  https://tika.apache.org/docs/4.0.x/using-tika/server/index.html#_endpoints_and_forked_process_groups
  "Endpoints and Forked-Process Groups" — it does two separate jobs: bounds concurrent requests per group and sizes each fork's CPU view. Also says /tika + /rmeta + /unpack + /meta + /detect + /pipes all share
  one group, which matters for Solr since they use /tika/xml and /rmeta/xml.

  Backpressure
  https://tika.apache.org/docs/4.0.x/using-tika/server/index.html#_backpressure_the_server_can_tell_you_it_is_busy
  The 429 + Retry-After contract.

Update dependency org.apache.tika:tika-core to v4

f019a75

solrbot added the exempt-stale Prevent a PR from going stale label Sep 1, 2026

solrbot added this to the 10.x milestone Sep 1, 2026

github-actions Bot added dependencies Dependency upgrades module:extraction tool:build admin-ui labels Sep 1, 2026

solrbot and others added 2 commits September 1, 2026 01:03

Add changelog entry for PR#4853

17bcad6

Add missing license for tika 4

fe73492

epugh self-assigned this Sep 1, 2026

barebones implmentation that doesn't not use markdown, it stays with …

d9daf06

…xml. This is mostly a "hey, look, I got us to tika 4", but it feels like its still trying to push Tika 4 into a tika 3 approach, when Tika wants to be different.

github-actions Bot added documentation Improvements or additions to documentation tests labels Sep 3, 2026

Merge remote-tracking branch 'upstream/main' into pr/4853

d8643fa

github-actions Bot removed the admin-ui label Sep 3, 2026

epugh added 2 commits September 3, 2026 17:42

Support Tika 4 method of passing in configuraiton.

5ccf463

Unused...

c631731

epugh requested review from janhoy and a balanced review from Copilot September 3, 2026 21:55

Copilot started reviewing on behalf of epugh September 3, 2026 21:56 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

janhoy removed this from the 10.x milestone Sep 3, 2026

tballison reviewed Sep 4, 2026

View reviewed changes

janhoy reviewed Sep 4, 2026

View reviewed changes

tballison reviewed Sep 4, 2026

View reviewed changes

		@@ -0,0 +1 @@
		f370d8619de9e19329b2cc117943c83468c2ff9f

Uh oh!

Conversation

solrbot commented Sep 1, 2026

Release Notes

Configuration

Uh oh!

epugh commented Sep 1, 2026

Uh oh!

janhoy commented Sep 1, 2026

Uh oh!

epugh commented Sep 1, 2026

Uh oh!

solrbot commented Sep 1, 2026

Edited/Blocked Notification

Uh oh!

janhoy commented Sep 3, 2026

Uh oh!

tballison commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

epugh commented Sep 3, 2026

Uh oh!

epugh commented Sep 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tballison Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tballison left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tballison commented Sep 3, 2026 •

edited

Loading

tballison Sep 4, 2026 •

edited

Loading