[nalinigans]

This is the initial set of changes to use GenomicsDB/TileDB as the TileDB submodule, and to get GenomicsDB building again on the current GitHub runners.

TileDB submodule

• dependencies/TileDB now points at https://github.com/GenomicsDB/TileDB.git, at its latest main (0d3095a, up from 8f497fc).

Build fixes

• Maven: CMakeLists.txt passes -Dgenomicsdb.version to mvn versions:set too. Maven 3.9.10 and later reject the unresolved ${genomicsdb.version} in pom.xml's <version>; older Maven only warned.
• CMake 4: the macOS runners now ship CMake 4, which rejects cmake_minimum_required older than 3.5. Old dependencies that TileDB builds, starting with aws-c-common, still declare one, so the workflow sets CMAKE_POLICY_VERSION_MINIMUM: 3.5. CMake 3.x on the Ubuntu runners ignores it. The proper fix belongs in TileDB's dependency builds.
• lcov on macOS: Homebrew no longer installs a formula from a local file, so install_macos_prereqs.sh installs the pinned lcov 1.16 formula through a local tap.

CI (basic.yml and friends)

• Runners: retired runners removed. The matrix is ubuntu-22.04 and macos-15-intel, and CodeQL runs on ubuntu-latest.
• HDFS setup:
  • Hadoop and Spark are downloaded from dlcdn.apache.org first, falling back to archive.apache.org, with retries and wget --timeout=60 so a stalled download fails instead of hanging.
  • Spark is now cached along with Hadoop. The cache is saved right after the install step, and is actually used: the Hadoop check looked for an env file that is never cached, so it re-downloaded every time.
  • The install step has a 45-minute timeout instead of the 6-hour job limit.
• Codecov: statuses are held (manual_trigger) until a new notify-codecov job runs after all the build jobs have uploaded, so they no longer report on partial coverage.
• Actions: moved to their Node 24 versions (checkout@v7, setup-python@v7, setup-java@v6, cache@v6, codecov-action@v7).
• Maven cache: skipped on macOS runs that don't build Java, which removes the "Path Validation Error" warning.

master → main

The default branch has been renamed, so references to it now say main: workflow branch checks and triggers (basic.yml, codeql.yml), the release_test.yml checkout, tag_for_docker_release.sh, the README badges, the docs/apis.rst source link and the license URL in pom.xml.

Follow-ups

• Make TileDB's bundled dependencies build with CMake 4, so CMAKE_POLICY_VERSION_MINIMUM can be dropped.
• Upgrade Spark from 3.4.0 (end of life, and only on the slow Apache archive) to 3.5.x.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.32%. Comparing base (d35a419) to head (85594a3).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #359      +/-   ##
===========================================
+ Coverage    75.99%   77.32%   +1.32%     
===========================================
  Files          146      144       -2     
  Lines        24184    23061    -1123     
  Branches      3016     3018       +2     
===========================================
- Hits         18379    17832     -547     
+ Misses        5591     5015     -576     
  Partials       214      214              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nalinigans]

This can be removed when we accept all the TileDB changes

[tfenne]

Thanks, this is great to see: CI green on current runners, and the Hadoop/Spark cache fix and download timeouts will save a lot of waiting (we hit the slow archive.apache.org downloads on the TileDB side too).

One thing I think needs fixing before merge: release_test.yml now checks out GenomicsDB/TestGenomicsDBJar at main, but that repository still only has master, so the release test would fail at checkout. Could be fixed either here or renaming master in the other repo.

The rest are notes rather than requests, and I'm happy to pick any of them up in follow-up PRs:

• The TileDB submodule is at 0d3095a; TileDB main has since picked up GenomicsDB/TileDB#8, GenomicsDB/TileDB#13, GenomicsDB/TileDB#14 and GenomicsDB/TileDB#16 (the error-message buffer overflow and the two aarch64 char fixes); we can also bump the pin once we stop merging things into TileDB in a few days.
• Once TileDB GenomicsDB/TileDB#12 is in, TileDB's SDK builds handle CMake 4 themselves and the CMAKE_POLICY_VERSION_MINIMUM workaround can go.
• release.yml and release_test.yml still run on macos-13, which has been retired, so the next release tag would fail on its macOS jobs. I have a macOS release job working in our fork that I can bring across.
• Hadoop 3.3.5 and Spark 3.4.0 aren't on dlcdn.apache.org any more, so uncached downloads still come from the archive; bumping them, as you note for Spark, will let the dlcdn path pay off.
• The README table header still says "Master".

[nalinigans]

Thanks, this is great to see: CI green on current runners, and the Hadoop/Spark cache fix and download timeouts will save a lot of waiting (we hit the slow archive.apache.org downloads on the TileDB side too).

One thing I think needs fixing before merge: release_test.yml now checks out GenomicsDB/TestGenomicsDBJar at main, but that repository still only has master, so the release test would fail at checkout. Could be fixed either here or renaming master in the other repo.

Done!!

The rest are notes rather than requests, and I'm happy to pick any of them up in follow-up PRs:

• The TileDB submodule is at 0d3095a; TileDB main has since picked up GenomicsDB/TileDB#8, GenomicsDB/TileDB#13, GenomicsDB/TileDB#14 and GenomicsDB/TileDB#16 (the error-message buffer overflow and the two aarch64 char fixes); we can also bump the pin once we stop merging things into TileDB in a few days.

Yes, we can bump up the TileDB pin once we merge the remaining TileDB PRs.

• Once TileDB GenomicsDB/TileDB#12 is in, TileDB's SDK builds handle CMake 4 themselves and the CMAKE_POLICY_VERSION_MINIMUM workaround can go.

Yes, I placed a comment for that.

• release.yml and release_test.yml still run on macos-13, which has been retired, so the next release tag would fail on its macOS jobs. I have a macOS release job working in our fork that I can bring across.

I did not change anything there, so feel free to PR your changes. Thanks.

• Hadoop 3.3.5 and Spark 3.4.0 aren't on dlcdn.apache.org any more, so uncached downloads still come from the archive; bumping them, as you note for Spark, will let the dlcdn path pay off.

We need to bump these, but left it for later.

• The README table header still says "Master".

Done!!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud