update script for building for software.eessi.io (version 2023.06) #383
Conversation
|
Instance
|
There was a problem hiding this comment.
Huh, lots of changes:
- we probably need to change the container images (build containers)
$\rightarrow$ different repo though - the GHA workflows need to be changes accordingly
- we should double-check if the bot has any mentioning of "pilot", pilot.eessi-hpc.org and the like; if we create new container images, the repo config for the bot has to be changed
$\rightarrow$ just noted here, changes may need to be done in other repositories
eessi_container.sh
Outdated
| @@ -79,7 +79,7 @@ display_help() { | |||
| echo " -m | --mode MODE - with MODE==shell (launch interactive shell) or" | |||
| echo " MODE==run (run a script or command) [default: shell]" | |||
| echo " -r | --repository CFG - configuration file or identifier defining the" | |||
| echo " repository to use [default: EESSI-pilot via" | |||
| echo " repository to use [default: EESSI via" | |||
| echo " default container, see --container]" | |||
There was a problem hiding this comment.
The container images we use should probably be updated with the new configuration too.
There was a problem hiding this comment.
Well, yes and no.
Once cvmfs-contrib/config-repo#206 is merged, software.eessi.io should become automatically available with the current build (and client) container, I think.
So I prefer to try and see if that's the case first before we update the build container (which may be wise anyway, to sync up with the latest CernVM-FS for example).
There was a problem hiding this comment.
If we want to make use of the configuration provided by cvmfs-config-default, we do have to make sure to also mount /cvmfs/cvmfs-config.cern.ch in the container, as that provides the configuration files. So, we need to add a --fusemount container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch to the apptainer command.
There was a problem hiding this comment.
When trying interactively, for some reason accessing /cvmfs/cvmfs-config.cern.ch just works even without the extra --fusemount option, and hence so does accessing /cvmfs/software.eessi.io:
Apptainer> find /etc/cvmfs | grep eessi
/etc/cvmfs/domain.d/eessi-hpc.org.conf
/etc/cvmfs/keys/eessi-hpc.org
/etc/cvmfs/keys/eessi-hpc.org/pilot.eessi-hpc.org.pub
/etc/cvmfs/keys/eessi-hpc.org/cvmfs-config.eessi-hpc.org.pub
/etc/cvmfs/keys/eessi-hpc.org/ci.eessi-hpc.org.pub
Apptainer> find /etc/cvmfs | grep eessi.io
Apptainer>
Apptainer> ls /cvmfs/cvmfs-config.cern.ch
etc
Apptainer> find /cvmfs/cvmfs-config.cern.ch -type f grep eessi.io | xargs sha256sum
find: paths must precede expression: `grep'
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -
Apptainer> find /cvmfs/cvmfs-config.cern.ch -type f | grep eessi.io | xargs sha256sum
7d484216c59c00b81c9eb89e5ac511930d0e601a36cc2bdb106f51d148b19f14 /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/domain.d/eessi.io.conf
d86b2190859f5124e78e0ff711c5a042be10d569350130a0f10a537eaacbc86b /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/keys/eessi.io/eessi.io.pub
Apptainer>
Apptainer> ls /cvmfs/software.eessi.io
host_injections versions
Apptainer> attr -g revision /cvmfs/software.eessi.io
Attribute "revision" had a 2 byte value for /cvmfs/software.eessi.io:
25
Maybe loading one CernVM-FS repo is enough to also trigger a mount of the cvmfs-config.cern.ch?
There was a problem hiding this comment.
To clarify: accessing /cvmfs/software.eessi.io was working for me because we have CernVM-FS installed system-wide as well, so I guess the mount picks up on /cvmfs/cvmfs-config.cern.ch from the host OS.
If CernVM-FS is not installed in the host OS, then it doesn't work, so we do indeed need to explicitly mount /cvmfs/cvmfs-config.cern.ch via --fusemount.
…nfig' command to where eb command is available
|
bot: build repo:eessi.io-2023.06-software arch:x86_64/generic |
Updates by the bot instance
|
|
New job on instance
|
…s for eessi_container.sh script
boegel
changed the title
|
bot: build repo:eessi.io-2023.06-software arch:x86_64/intel/haswell |
Updates by the bot instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
This PR should be ready to go. The tests will keep failing until the configuration for So we should probably wait with deploying + merging until that's the case (should be on Mon 20 Nov'23). We may also need to make a change before the deploy works correctly (to the right S3 bucket). |
|
First smaller PR that works towards getting these changes in place in the |
|
closing this, superseded by list of PRs mentioned in description |
…-gfbf/2022b
{2023.06}[gfbf/2022b] ASE V3.22.1
/cvmfs/software.eessi.ioas default target repoEESSI_PILOT_VERSIONtoEESSI_VERSION, etc.)marked as WIP because i) untested, ii) maybe we want to introduce a new
2023.06-eessi.iobranch, or fork off2023.06-pilotfirst to retain possibility for building for EESSI pilot repoedit: as requested, I will break up this PR into smaller ones that are easier to review & merge:
software.eessi.io(+ drop 'pilot' terminology) #387eessi_container.shscript forsoftware.eessi.io#390pilot.eessi-hpc.org#392software.eessi.ioto check on EasyBuild v4.8.2 installation #393/software-layer#398Extra PRs beyond what's changed in this one:
$EESSI_SOFTWARE_SUBDIR_OVERRIDEinarchdetect#397