I want to run a loop in parallel on a SLURM cluster and use progressr to output near-live updates. Here's my setup:
test.R
library("future")
library("furrr")
library("progressr")
options(progressr.enable = TRUE)
handlers(global = TRUE)
handlers(handler_progress(
format = ":current/:total [:bar] :percent in :elapsed, ETA: :eta",
width = 60,
complete = "="
))
plan(cluster)
f <- function(i) {
Sys.sleep(3)
i
}
local({
n <- 100
p <- progressor(n)
result <- future_map(1:n, function(i){
p(class = "sticky")
f(i)
})
})test.sh
#!/bin/bash
#SBATCH --time=0-00:05:00
#SBATCH --ntasks 10
#SBATCH --output=slurm.out
Rscript test.R
The issue is that when the job is running, cat slurm.out always shows
0/100 [--------------------------------------------------] 0% in 0s, ETA: ? and no live progress is reported. How can I get it to show the current progress?
I have read this discussion on getting a progress bar using future.batchtools and that thread on printing messages to the console when they are generated, but I can't get it to work. Is this a bug?
I want to run a loop in parallel on a SLURM cluster and use
progressrto output near-live updates. Here's my setup:test.R
test.sh
The issue is that when the job is running,
cat slurm.outalways shows0/100 [--------------------------------------------------] 0% in 0s, ETA: ?and no live progress is reported. How can I get it to show the current progress?I have read this discussion on getting a progress bar using
future.batchtoolsand that thread on printing messages to the console when they are generated, but I can't get it to work. Is this a bug?