Skip to content

Add hook to include --cpu-target for julia wrappers - #327

Open
Crivella wants to merge 3 commits into
EESSI:mainfrom
Crivella:feature-julia_hook
Open

Add hook to include --cpu-target for julia wrappers#327
Crivella wants to merge 3 commits into
EESSI:mainfrom
Crivella:feature-julia_hook

Conversation

@Crivella

@Crivella Crivella commented Oct 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

The current only version of Julia 1.12.2 on EESSI needs a rebuild to pick up the wrappers introduced by

While testing, it seems that julia and/or the build scripts of some packages makes some assumption on the cpu target it is on that might not be correct. See eg the failures in the detail section of this message EESSI/software-layer#1828 (comment)

I manually tested that rebuilding julia with the wrappers and modifying them to include the --cpu-target pointing to the correct CPU does work.
The chosen target for the map comes from the LLVM version baked into julia and can be seen with julia --cpu-target=help (will only show the x86 or arm target depending on the version being ran)

Manually tested by rebuilding the package from EESSI/software-layer#1828 on:

  • x86_64/generic
  • haswell
  • aarch/generic
  • graviton4

TO CHECK

  • Do not have access to a grace node to test that neoverse-v2 does work properly with it
  • Not sure if icelake-client or icelake-server is more appropriate. I kept it to client as i think it has a couple less feature to make sure we are compatible with all icelake targets
  • Also not sure if the package i am using is the perfect test for this, it is running machine learning and linear algebra stuff, but i do not not if there are other deps/packages that would constitute a better test...

@Crivella Crivella mentioned this pull request Oct 22, 2026
Open
ocaisa
ocaisa previously approved these changes Oct 23, 2026
View reviewed changes

@ocaisa ocaisa left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle looks fine. We only saw issues with this for older architectures, does that mean that we might as well hardcode the ones we know now, but future archs will probably work ok by default?

Comment thread eb_hooks.py Outdated
@Crivella @ocaisa
Update eb_hooks.py
4693d95 
Co-authored-by: ocaisa <alan.ocais@cecam.org>
@Crivella

Crivella commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

i think it depends in the Julia version as well, some time we might need to pass an older arch if the new one is not available.
The other place we need this for sure is for generic builds, since the node we build on might not be truly generic...

For newer archs the script will give a warning but not hard fail. I would go on the assumption that it will work and if we find problems we can add the new arch to the map, but also fine having this hard fail

ocaisa
ocaisa reviewed Oct 23, 2026
View reviewed changes
Comment thread eb_hooks.py Outdated
ocaisa
ocaisa reviewed Oct 23, 2026
View reviewed changes
Comment thread eb_hooks.py
ocaisa
ocaisa approved these changes Oct 23, 2026
View reviewed changes

@ocaisa ocaisa left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me, I should be able to test it

@ocaisa

ocaisa commented Oct 23, 2026

Copy link
Copy Markdown
Member

Tested with Julia 1.13.2 and it worked.

@ocaisa

ocaisa commented Oct 23, 2026

Copy link
Copy Markdown
Member

bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2
bot: build repo:eessi.io-2026.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

@eessi-bot-aws

eessi-bot-aws Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown

New job on instance eessi-bot-mc-aws for repository eessi.io-2023.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.10/pr_327/213627

date job status comment
Oct 23 10:49:42 UTC 2026 submitted job id 213627 awaits release by job manager
Oct 23 10:50:12 UTC 2026 released job awaits launch by Slurm scheduler
Oct 23 10:57:25 UTC 2026 running job 213627 is running
Oct 23 11:01:42 UTC 2026 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-213627.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen2-17927532650.tar.zstsize: 0 MiB (35955 bytes)
entries: 1
modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2023.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
reprod directories under 2023.06/software/linux/x86_64/amd/zen2/reprod
no reprod directories in tarball
other under 2023.06/software/linux/x86_64/amd/zen2
no other files in tarball
Oct 23 11:01:42 UTC 2026 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-213627.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@eessi-bot-aws

eessi-bot-aws Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.10/pr_327/213628

date job status comment
Oct 23 10:49:47 UTC 2026 submitted job id 213628 awaits release by job manager
Oct 23 10:50:09 UTC 2026 released job awaits launch by Slurm scheduler
Oct 23 10:57:22 UTC 2026 running job 213628 is running
Oct 23 11:01:39 UTC 2026 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-213628.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2025.06-software-linux-x86_64-amd-zen2-17927532360.tar.zstsize: 0 MiB (35960 bytes)
entries: 1
modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2025.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod
no reprod directories in tarball
other under 2025.06/software/linux/x86_64/amd/zen2
no other files in tarball
Oct 23 11:01:39 UTC 2026 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-213628.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@eessi-bot-aws

eessi-bot-aws Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown

New job on instance eessi-bot-mc-aws for repository eessi.io-2026.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.10/pr_327/213629

date job status comment
Oct 23 10:49:53 UTC 2026 submitted job id 213629 awaits release by job manager
Oct 23 10:50:07 UTC 2026 released job awaits launch by Slurm scheduler
Oct 23 10:55:15 UTC 2026 running job 213629 is running
Oct 23 10:58:30 UTC 2026 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-213629.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2026.06-software-linux-x86_64-amd-zen2-17927530530.tar.zstsize: 0 MiB (35904 bytes)
entries: 1
modules under 2026.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2026.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
reprod directories under 2026.06/software/linux/x86_64/amd/zen2/reprod
no reprod directories in tarball
other under 2026.06/software/linux/x86_64/amd/zen2
no other files in tarball
Oct 23 10:58:30 UTC 2026 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 expected failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-213629.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ocaisa ocaisa ocaisa approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Crivella @ocaisa