Add conda v1 recipes for compass and otps packages
#933
Conversation
xylar
added
documentation
|
@matthewhoffman and @trhille, could you follow the quick start for users on Perlmutter or Chrysalis or some other linux machine? I just want to make sure the packages are actually working as expected. It would be nice to run some tests but even just setting something up would be a good starting point. |
trhille
left a comment
trhille
left a comment
There was a problem hiding this comment.
@xylar, this is great! I followed the quick-start guide on Perlmutter and was able to create the compass env and successfully run the landice/kangerlussuaq/mesh_gen test case.
I have one small comment to do with a potential tripping point in conda config settings that we might as well make a note of, but I'm approving this because it works as-is and that is really not a compass issue.
I'd be happy to do any other testing you think is necessary.
docs/users_guide/quick_start.rst
Outdated
| .. code-block:: bash | ||
|
|
||
| conda create -n compass -c conda-forge -c e3sm/label/compass python=3.10 \ | ||
| conda create -n compass -c conda-forge -c e3sm/label/compass python=3.13 \ |
There was a problem hiding this comment.
When I first tried this, I got an error:
Solving environment: failed
LibMambaUnsatisfiableError: Encountered problems while solving:
- package compass-1.8.0-mpi_mpich_haa344d4_0 is excluded by strict repo priority
I was able to solve this by changing my conda channel priority settings: conda config --set channel_priority flexible. It might be worth adding a note here.
There was a problem hiding this comment.
Oh, dear! I always feared we'd get scooped. Someone made a compass package on conda-forge:
https://github.com/conda-forge/compass-feedstock
So I'll need to switch the channel order so -c e3sm/label/compass comes first. But I need to make sure we're not serving packages on that label that we don't want. I don't think so.
|
@xylar , thanks for putting this together so quickly. I tried to follow these steps in the Ubuntu Docker container for MALI that Luca has set up. I was able to install the compass package without incident but hit an internal compiler error while trying to compile jigsaw. Are you familiar at all with this error? |
|
@matthewhoffman, no idea. I'd have to ask ChatGPT, which I'm happy to do. |
xylar
force-pushed
the
add-conda-v1-recipe
branch
2 times, most recently
from
95a5d35 to
cad5279
Compare
|
@xylar , with the help of ChatGPT, I was able to get past the error and get jigsaw compiled successfully by disabling link-time optimization with these flags: It thought it might be related to messy architecture (compiling AMD in a container on ARM architecture) and/or compiler version. This should not be necessary generally, but seems like a fine workaround for this situation. |
|
I ran into trouble creating the load script with I retested and I get some error messages that are confusing: but not worth generating yet another set of packages to fix. I just fixed it in this branch so it will go away the next time we build packages. |
xylar
force-pushed
the
add-conda-v1-recipe
branch
from
cad5279 to
685f304
Compare
matthewhoffman
left a comment
matthewhoffman
left a comment
There was a problem hiding this comment.
@xylar , after resolving the debugging issue with jigsaw, I was able to use the conda package to successfully install compass in the linux container. I confirmed that compass list worked. I tried setting up a test case and realized that I need to deal with specifying a config file or machine for the container. I think that is beyond the scope of this PR, so I'm approving this now.
|
I think some issues remain for The main thing is it's assuming PIO comes from the conda environment, which is never going to be true. I'm not sure how to make it find PIO, given that we're basically making it up to our users to figure that out. One option is to simply remove In any case, it seems likely that the load script produced in the docker container will need to be more sophisticated than this, including setting various other environment variable that MALI needs. Presumably, we just won't call @matthewhoffman, what are your thoughts here? |
|
@xylar , I agree something more sophisticated needs to happen for the container situation (which may or may not directly bear on this PR). I just made the discovery that multi processor runs with landice_model are not working in the container because I'm picking up mpirun from the conda environment, which I think is a similar situation to what you are describing. I'll have to think about this some more and test to understand the situation better and have a suggestion. |
|
I was able to get around my mpirun issue by specifying the full path to mpirun in |
You need to do the equivalent of loading the MPI module after you load the conda environment. This is something we always do in the compass load scripts for supported machines. You can probably use one of them as an example to work from. |
That's also an okay workaround. But I would think just making sure the path to the executable gets added to your |
|
Okay, I think we'll live with a somewhat imperfect load script produced by |
3 participants
This merge also fixes two issues with the compass package exposed by the conda builds:
__init_.pythat should have been__init__.pyHere, I have also updated the quick start for users to be consistent with the latest releases I have uploaded: compass 1.8.0.
Checklist
Testingin this PR) any testing that was used to verify the changes