-
Notifications
You must be signed in to change notification settings - Fork 16
Make all Lmod-based shell initialisations more robust, add additional features to facilitate site integration #153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ocaisa
wants to merge
23
commits into
EESSI:main
Choose a base branch
from
ocaisa:better_init
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+481
−82
Open
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
143341c
Make all shell initialisations more robust
ocaisa a428f68
Fix typo and tests
ocaisa 6b057e8
Merge branch 'main' into better_init
ocaisa c3d3159
See where the error is
ocaisa 1466e39
Merge branch 'better_init' of github.com:ocaisa/software-layer-script…
ocaisa 3692346
Add new capabilities and new tests
ocaisa 46c878f
Try to fix csg
ocaisa 3a6c117
csh has very short maximum length for envvar names, stick to <32 chars
ocaisa 785ae13
Some fancy new runners available!
ocaisa 61db941
Actually use the CPU override
ocaisa ec6f4e9
Merge branch 'main' into better_init
ocaisa dcce340
Use specific commit for assert.sh, also run tests on system with Lmod…
ocaisa 5e5ed24
Merge branch 'better_init' of github.com:ocaisa/software-layer-script…
ocaisa a97140a
Don't clobber an existing .cshrc
ocaisa d779b54
Add a sanity check to make sure module command works
ocaisa d91ded8
Set explicit MODULEPATH
ocaisa fc13853
No quotes on ~
ocaisa 7d621f1
Make test init script listen to EESSI_SOFTWARE_SUBDIR_OVERRIDE
ocaisa 78e286b
Remove copy/pasted comment
ocaisa 4d97256
Add licence and more extensive documentation
ocaisa 30a8f7c
Add some additional comments to the bash script
ocaisa 3b30931
Print some more details on first initialisation
ocaisa 43c372f
Add a module purge before doing the initialisation
ocaisa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| setenv("INSIDE_GITHUB_ACTIONS", "true") | ||
| -- Interfere with PATH so Lmod keeps a record | ||
| prepend_path("PATH", "/snap/bin") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| setenv("INSIDE_GITHUB_ACTIONS", "true") | ||
| -- Interfere with PATH so Lmod keeps a record | ||
| prepend_path("PATH", "/snap/bin") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,19 +1,110 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # SPDX-License-Identifier: GPL-2.0-only | ||
| # Copyright (C) 2020-2026 EESSI contributors | ||
| # | ||
| # EESSI - European Environment for Scientific Software Installations | ||
| # | ||
| # This file is a template for initialising EESSI via Lmod for the environment indicated by the shebang. | ||
| # | ||
| # Assumptions: | ||
| # - EESSI accessible under $EESSI_CVMFS_REPO or /cvmfs/software.eessi.io | ||
| # - expected to be true as this file is also meant to be shipped under that location | ||
| # - Lmod must the shipped in the repository in the correct location | ||
| # | ||
| # Options: | ||
| # - The script listens to environment variables so that it can be silently configured by a site | ||
| # - EESSI_CVMFS_REPO: perform the initialisation from a respository other than /cvmfs/software.eessi.io | ||
| # - EESSI_VERSION: loads a specific EESSI version ignoring the default in the file | ||
| # - EESSI_DEFAULT_MODULES_PREPEND: environment variable allows you to prepend modules to the defaults (loaded last, : separated) | ||
| # - EESSI_DEFAULT_MODULES_APPEND: environment variable allows you to append modules to the defaults (loaded first, : separated) | ||
| # - EESSI_EXTRA_MODULEPATH: environment variable allows a site to append to MODULEPATH (: separated, lower priority than EESSI MODULEPATH) | ||
| # - EESSI_NO_MODULE_PURGE_ON_INIT: environment variable that disables purging modules before initialisation | ||
| # | ||
| # Other options that can be set to influence the end result: | ||
| # - The EESSI module also listens to environment variables so that it can be silently configured by a site | ||
| # - EESSI_MODULE_FAMILY_NAME: use the value of the environment variable to set an Lmod family for the EESSI module | ||
| # - EESSI_MODULE_STICKY: make the EESSI module sticky | ||
| # - EESSI_MODULE_UPDATE_PS1: have the EESSI module update PS1 to give a prompt that is prepended with "{EESSI/...} " | ||
| # (- EESSI_MODULE_DEBUG_INIT: enable debug print statements when loading the EESSI module) | ||
| # | ||
| # Effects: | ||
| # - Should always succeed | ||
| # - Initialises Lmod from specific version of EESSI | ||
| # - Clobbers any existing Lmod configuration | ||
| # - Some special environment variables that are internal to Lmod (__LMOD_REF_COUNT_MODULEPATH and | ||
| # _ModuleTable001_) are used to force a hard reset of an existing Lmod installation. This | ||
| # approach may be brittle. | ||
| # - Then loads module EESSI/... to initialise EESSI | ||
| # - use `module show EESSI/...` to see the environment variables set by the EESSI module | ||
| # | ||
| # Reverting the effects: | ||
| # - EESSI initialisation via `module unload EESSI/...` | ||
| # - Lmod initialisation cannot be undone | ||
|
|
||
| # Choose an EESSI CVMFS repository | ||
| EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO:-/cvmfs/software.eessi.io}" | ||
| # Choose an EESSI version | ||
| EESSI_VERSION_DEFAULT="__EESSI_VERSION_DEFAULT__" | ||
|
|
||
| # Choose an EESSI version (the default is only used if the EESSI_VERSION environment variable is not provided) | ||
| # (Note: in the repository which is home to this file a template value __EESSI_VERSION_DEFAULT__ is present in | ||
| # the line below which is replaced within our deployment pipeline.) | ||
| EESSI_VERSION_DEFAULT="${__EESSI_VERSION_USED_FOR_INIT:-__EESSI_VERSION_DEFAULT__}" | ||
| EESSI_VERSION="${EESSI_VERSION:-${EESSI_VERSION_DEFAULT}}" | ||
| # Path to top-level module tree | ||
| export MODULEPATH="${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/modules" | ||
| . "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/$(uname -m)/usr/share/Lmod/init/bash" | ||
| # On the first run we want to record the EESSI version used for init as an environment variable so that if a different | ||
| # version of this script is called (e.g, for a a different EESSI version) it retains a memory which EESSI | ||
| # version was actually used in the initialisation. This is useful as __Init_EESSI_Default_Modules used below will | ||
| # be defined on the first call and Lmod initialisation will not happen twice. | ||
| # This sets the value only on first execution, if the variable already exists in the environment | ||
| # the original value is retained. | ||
| export __EESSI_VERSION_USED_FOR_INIT="${__EESSI_VERSION_USED_FOR_INIT:-${EESSI_VERSION}}" | ||
|
|
||
| # LMOD_SYSTEM_DEFAULT_MODULES are used by Lmod to load a default set of modules in the scenario | ||
| # - where we initialise Lmod (the if part of the clause below) | ||
| # - where we reset Lmod (the else part of the clause below) | ||
| # This means that if we call this script twice we will get the same end result: an Lmod installation | ||
| # with a set of default modules loaded. | ||
| # | ||
| # We also allow the ability to predefine elsewhere the default list of modules to load (colon separated): | ||
| # - EESSI_DEFAULT_MODULES_PREPEND environment variable allows you to prepend modules (loaded last) | ||
| # - EESSI_DEFAULT_MODULES_APPEND environment variable allows you to append modules (loaded first) | ||
| LMOD_SYSTEM_DEFAULT_MODULES="${EESSI_DEFAULT_MODULES_PREPEND:+$EESSI_DEFAULT_MODULES_PREPEND:}EESSI/$EESSI_VERSION${EESSI_DEFAULT_MODULES_APPEND:+:$EESSI_DEFAULT_MODULES_APPEND}" | ||
| export LMOD_SYSTEM_DEFAULT_MODULES | ||
|
|
||
| if [ -z "$__Init_EESSI_Default_Modules" ]; then | ||
| export __Init_EESSI_Default_Modules=1; | ||
|
|
||
| # Lmod version in 2023.06 has a problem with newer Lmod caches, so let's stick to more recent Lmod | ||
| # (has no effect except on Lmod itself, and compatible caches are still created/supported by EESSI) | ||
| LMOD_EESSI_VERSION=${EESSI_VERSION/2023.06/2025.06} | ||
|
|
||
| # Let's attempt a purge of any loaded modules as any environment variables currently set will survive EESSI initialisation | ||
| # (it's ok if the module command does not exist) | ||
| if [ -z "$EESSI_NO_MODULE_PURGE_ON_INIT" ]; then | ||
| module purge >/dev/null 2>&1 && echo "Modules purged before initialising EESSI" | ||
| fi | ||
|
|
||
| # Lmod may have been initialised so we need to clear some internal variables to allow for a full reset | ||
| # - make it forget about the system set MODULEPATH | ||
| unset __LMOD_REF_COUNT_MODULEPATH | ||
| # - and clear out any memory Lmod might have | ||
| unset _ModuleTable001_ | ||
|
|
||
| if [ -z "$__Init_Default_Modules" ]; then | ||
| export __Init_Default_Modules=1; | ||
| # Path to top-level module tree | ||
| # - EESSI_EXTRA_MODULEPATH environment variable allows a site to append to MODULEPATH (lower priority than EESSI MODULEPATH) | ||
| export MODULEPATH="${EESSI_CVMFS_REPO}/init/modules${EESSI_EXTRA_MODULEPATH:+:$EESSI_EXTRA_MODULEPATH}" | ||
| . "${EESSI_CVMFS_REPO}/versions/${LMOD_EESSI_VERSION}/compat/linux/$(uname -m)/usr/share/Lmod/init/bash" | ||
| module --initial_load --no_redirect restore | ||
ocaisa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## ability to predefine elsewhere the default list | ||
| LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"EESSI/$EESSI_VERSION"} | ||
| export LMOD_SYSTEM_DEFAULT_MODULES | ||
| module --initial_load --no_redirect restore | ||
| # After initialising, we now know the architecture(s) that was/were selected so let's report them | ||
| echo "EESSI has selected ${EESSI_SOFTWARE_SUBDIR} as the compatible CPU target for EESSI/${EESSI_VERSION}" | ||
| if [ -n "$EESSI_ACCEL_SUBDIR" ]; then | ||
| echo "EESSI has selected ${EESSI_ACCEL_SUBDIR} as the compatible accelerator target for EESSI/${EESSI_VERSION}" | ||
| else | ||
| echo "EESSI did not identify an accelerator on the system" | ||
| fi | ||
| # If people want more detailed information about what EESSI is doing, they can also set an environment variable | ||
| # for additional information. | ||
| echo "(for debug information when loading the EESSI module, set the environment variable EESSI_MODULE_DEBUG_INIT)" | ||
| else | ||
| module refresh | ||
| module reset | ||
| fi | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.