update n5 component versions

[bogovicj]

Leaving this as a draft for now, because some artifacts depending on n5 will need updating

PRs are open for some of them:

• bigwarp
• bigdataviewer-core
• bigdataviewer-n5

Details

• n5-4.0.0
• n5-aws-s3-5.0.0
• n5-blosc-2.0.0
• n5-google-cloud-6.0.0
• n5-hdf5-3.0.0
• n5-ij-5.0.0
• n5-imglib2-8.0.0
• n5-universe-3.0.0
• n5-zarr-2.0.0
• n5-zstandard-2.0.0

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/ngff-weekly-dev-update-thread/110810/95

[ctrueden]

Thanks, @bogovicj! FYI, I'm in the process of improving the pom-scijava CI—once that work makes it onto the mainline branch, we can rebase this PR onto the latest to see more clearly what else breaks due to this update.

[bogovicj]

Hi @ctrueden ,
Any news about this pom-scijava CI? We're eager to merge this in.

[ctrueden]

@bogovicj Thanks for checking in. I've been working on this a lot in recent days.

There is a new command line tool, pombast, that serves as a more robust and powerful replacement for the previous "mega-melt" shell scripts here in pom-scijava. I've been addressing the pom-scijava version skew issues one by one in conjunction with pombast development. If you visit https://status.scijava.org and click the Smelt column twice to sort it descending, you can see which components are currently validating versus having some version-skew-related issues. As of this writing, here's what's left to address:

• net.imagej:imagej-legacy:2.0.3 - Pain due to the jump from Java 8 to 11 in dependencies, notably SCIFIO. Probably can work around this by configuring pombast to build and run imagej-legacy with Java 11 rather than 8...
• net.imagej:imagej-server:0.2.0 - Same: need to test this with Java 11 now, due to SCIFIO Java version bump
• net.imagej:imagej-tensorflow:1.1.9 - Same: need to test this with Java 11 now, due to SCIFIO Java version bump
• org.janelia.saalfeldlab:n5-universe:2.3.0 - N5IO No container exists at...
• org.janelia.saalfeldlab:n5-zarr:1.5.1 - zarrCacheBehaviorHelper:383 arrays first differed at element [0]; expected:<[a]> but was:<[b]>
• org.janelia.saalfeldlab:n5:3.5.1 - N5IO No container exists at...
• org.morphonets:SNT:5.0.9 - needs adjustment to account for jgrapht API breakage
• org.scijava:scijava-search:3.0.0 - needs adjustment to account for forum.image.sc Discourse REST API changes (?)
• org.scijava:scripting-kotlin:0.2.1 - needs adjustment to match pom-scijava-base's current kotlin version
• sc.fiji:TrackMate-Skeleton:2.0.3 - needs updating to match TrackMate 8.x API breakage
• sc.fiji:TrackMate-StarDist:1.2.1 - needs updating to match TrackMate 8.x API breakage
• sc.fiji:labkit-ui:0.4.0 - one failing test with NPE; probably not a version skew issue (flaky test?) but needs confirmation

@bogovicj Do you know what the N5 test errors mean? Are the tests themselves broken, or did some remote access mechanism change in a backwards-incompatible way? Should I work around it by skipping these tests for the mega-melt CI?

For what it's worth, a few days ago I made an attempt to smelt everything with all the latest N5 component releases across the board, but there were many more problems. If you want to try doing it yourself, you can:

1. Clone pom-scijava repo.
2. Edit all n5<*>.version properties to the values you want to test. (For unreleased changes, you can set the version to 999-test or some such in a local working copy, mvn install it, then set the version in pom-scijava to match.)
3. uv tool install --from git+https://github.com/scijava/pombast pombast
4. cd /path/to/pom-scijava && pombast smelt .

The first time it runs it takes much longer, because it needs to clone repositories for all components being tested, and it has no successes cached from prior runs. But subsequent runs should be much faster, re-testing only those components that have not already succeeded with that particular dependency configuration.

Once I get the rest of the issues above under control (hopefully in the next couple of days—although grant proposal submission deadline also draws nigh... 👻), I'll give another go at smelting all the latest n5 stuff. Or if you do it and want to report back here, that's cool too.

Edit: I forgot to mention: the version of the BOM currently in testing is on the pombast branch here, not master. I could merge it anytime, although the CI build will still be failing if I do that. I was hoping to get everything perfect and then merge it and watch the CI turn green... 😇

[bogovicj]

Awesome, thanks for the run down @ctrueden !

I expect I'll be able to figure out what going on with the n5 artifacts. Will investigate and report back. May have some questions about pombast on the way.