new atime ci, no more old cml container

[tdhock]

The old atime CI uses an old container (old compilers etc.)
I propose switching to this new CI which is definitely more code.
Plus: easier to customize.
Minus: more maintenance burden.

[github-actions]

• HEAD=new-atime-ci much slower for isoweek improved in #7144
• HEAD=new-atime-ci slower P<0.001 for fwrite refactored in #6393
  

Generated via commit 4dc8c82

Download link for the artifact containing the test results: ↓ atime-results.zip

Task	Duration
R setup and installing dependencies	1 minutes and 5 seconds
Installing different package versions	16 minutes and 54 seconds
Running and plotting the test cases	5 minutes and 15 seconds

[MichaelChirico]

IIUC this is designed to supplant the old action which can't be updated? Is there anything stopping us from forking it instead (or, if the current PR is intended as such a fork, could we at least host it in a dedicated repo under Rdatatable or maybe under your tdhock?)

[tdhock]

right, the current PR is a fork. @Anirban166 would you consider putting this new code in your Autocomment-atime-results repo? (I was not sure if Ani is willing / able to continue maintenance, but that would be great)
The basic problem with the old action is that is uses an old cml container, old compilers, and this PR fixes that.

[tdhock]

as I suspected, Ani is not super active these days.
Would be nice to run atime ci for prs like #7907
I would suggest we merge this PR now, and we can maybe move it to a dedicated repo later, Is that Ok?

[tdhock]

hi @MichaelChirico @aitap can you please approve this PR?
There are other current PRs #7917 #7907 which would benefit from the atime CI analysis enabled by this PR.
I can do some revisions in a later PR if you like?

[MichaelChirico]

Thanks @tdhock! I had just peeked yesterday and saw it was marked draft. I'll take a look now.

[MichaelChirico]

Hey Toby, I had a look and was puzzled by the level of complexity in the proposed GHA. I am not super familiar with those tools, so I had a few LLMs review the proposal:

Review the changes in the current branch. Is it concise, minimal, canonical? Is it easy to maintain? Are there easily foreseen edge cases, redundancies, inefficiencies?

Where it came up with a number of concerns:

---

Bugs & Edge Cases

1. JSON encoding, word splitting, and glob expansion (lines 138-139)

   MD=$(sed 's/"/\\"/g' comment.md | sed ':a;N;$!ba;s/\n/\\n/g')
   echo '{"body":"'$MD'"}' > pr-comment.json

   • Because $MD is unquoted in echo '{"body":"'$MD'"}', bash performs word splitting (which collapsed the padding spaces in the markdown table in the comment above) and filename globbing on *, ?, and [...] (several test names in .ci/atime/tests.R contain bracket expressions like DT[by] and DT[,.SD], and HEAD_issues.md lines start with *).
   • Any literal \ in comment.md is not escaped, producing invalid JSON.
   • We can avoid sed and pr-comment.json entirely by letting gh api (or jq -Rs '{body: .}') encode the file directly: gh api "$ENDPOINT" -F body=@comment.md.

2. Unauthenticated API call, pagination, and node step (lines 31-32, 57-84)

   • curl https://api.github.com/repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments on line 64 does not send an Authorization header, so it shares the 60 req/hr unauthenticated rate limit per runner IP. If rate-limited, curl writes the 403 JSON object to comments.json and the node step crashes with prc.filter is not a function.
   • The endpoint only returns the first 30 comments by default (per_page=30), so on PRs with >30 comments it won't find an existing comment created later in the thread and will post duplicates.
   • Since gh and jq are pre-installed on ubuntu-latest (see cran-status-check.yaml), we can drop actions/setup-node@v6, comments.json, comment-number.txt, and pr-comment.json and replace steps comments.json, comment-number.txt, and convert comment to json... with:

     COMMENT_ID=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments" \
       --jq '.[] | select(.body | startswith("<!-- automatic comment for atime -->")) | .id' | head -n 1)
     if [ -z "$COMMENT_ID" ]; then
       gh api "repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments" -F body=@comment.md
     else
       gh api "repos/$GITHUB_REPOSITORY/issues/comments/$COMMENT_ID" -F body=@comment.md
     fi

3. Fork PRs (lines 17, 29-30, 141)

   • On PRs opened from a fork, GITHUB_HEAD_REF is a branch on the fork, not origin, so git switch "${GITHUB_HEAD_REF}" fails with fatal: invalid reference (or worse, if the contributor's fork branch is named master, git switch master silently checks out origin/master and benchmarks master against master).
   • Also on fork PRs, GITHUB_TOKEN is read-only, so posting the comment fails with HTTP 403 (currently hidden because curl on line 141 lacks --fail).
   • Since CONTRIBUTING.md notes that atime only runs on branches in Rdatatable/data.table, the job-level guard should be:

     if: github.event.pull_request.head.repo.full_name == 'Rdatatable/data.table'

     along with explicit job permissions (permissions: { contents: read, pull-requests: write }).

4. Image upload error handling (lines 88-91)

   • curl ... https://picrd.com/api/upload | sed 's/.*image_url":"//' | sed 's/",".*//' has no --fail or retry; if picrd.com returns an HTTP 5xx HTML page, the entire HTML body gets interpolated into PREVIEW_URL / ALL_URL. Using curl --fail --retry 3 -sS ... | jq -r '.image_url' avoids corrupting comment.md.

Redundancies & Canonicality

• R setup (lines 37-46): All 7 of our other workflows in .github/workflows/ (and the old Autocomment-atime-results action) use r-lib/actions/setup-r@v2 + r-lib/actions/setup-r-dependencies@v2. Using eddelbuettel/github-actions/r-ci@master here introduces a second R setup stack, leaves untracked ./run.sh and .Rprofile files in GITHUB_WORKSPACE (which atime copies into every historical build directory), and writing options(repos = c(CRAN = 'https://cloud.r-project.org')) to .Rprofile bypasses binary repositories so atime, ggplot2, and directlabels compile from source.
• Safe directory exception (lines 33-36): git config --global --add safe.directory '*' was only needed inside the old root-owned cml Docker container (and currently runs after git switch anyway).
• Step conditions (lines 58, 70, 125): if: ${{ github.event_name == 'pull_request' }} on individual steps is redundant now that on: only has pull_request.
• concurrency: group: Since a run takes ~23 minutes, adding a concurrency: block (as in R-CMD-check.yaml) will cancel superseded runs when new commits are pushed to a PR.
• Debug output (lines 65, 77-80, 92, 137-140): cat comments.json, console.log(content), and cat pr-comment.json dump the entire PR comment thread into the logs multiple times.
• (Follow-up idea) Caching historical builds: Installing the ~25 fixed historical SHAs in .ci/atime/tests.R takes ~17 minutes per run. Since atime::atime_versions_install() skips data.table.<sha> packages already present in .libPaths()[1], caching data.table.* in the R library keyed on hashFiles('.ci/atime/tests.R') could cut ~15 minutes off every run.

---

And it came up with #7921 as a proposed iteration based on this feedback. I find the switch to use the supported gh tooling particularly nice, as well as the proposed caching of installed SHAs. #7921 also unblocks {atime} on fork PRs -- it's just restricted from posting comments, but the jobs run and we can view the results in the logs, AIUI.