Skip to content

Add rclone check and checksum#12269

Open
atrigila wants to merge 6 commits into
nf-core:masterfrom
atrigila:rclone_check_checksum
Open

Add rclone check and checksum#12269
atrigila wants to merge 6 commits into
nf-core:masterfrom
atrigila:rclone_check_checksum

Conversation

@atrigila

@atrigila atrigila commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

I created two new modules for the RCLONE suite: rclone check and rclone checksum. I used the new functionality in nf-core/tools (dev) to create the container for rclone (which includes conda-lock files) from environment.yml files.

Notes:

  • The software would produce an error if, for example, the files are not available in destination. As we find this is something informative that we want to get a report on, I added the || true, so it won't fail, but we will get the files with the report.

Related to: nf-core/datasync#44

@atrigila atrigila marked this pull request as ready for review July 7, 2026 15:43
then {
assertAll(
{ assert process.success },
{ assert snapshot([match: process.out.match, versions_rclone: process.out.versions_rclone]).match() }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not sanitizeOutput(process.out)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have clarified this in the PR description. I tried that. If you snapshot all results it will produce empty files for the optional output processes (and that would make the linter fail). This is the "cleanest" I could come up with but I am open to hear if you have any other suggestion.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sanitizeOutput(process.out, unstableKeys: ["combined", "differ", "missing_on_dst", "missing_on_src","error"]) ?

Comment thread modules/nf-core/rclone/checksum/main.nf Outdated

@SPPearce SPPearce left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you doing something unusual in the snapshots?

atrigila and others added 4 commits July 8, 2026 10:10
@atrigila atrigila requested a review from SPPearce July 8, 2026 15:54
@atrigila

atrigila commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@SPPearce Do you have any other idea on how to approach this? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants