meeting 2023 09 07

Notes for 2023-09-07 meeting

• date & time: Thu 7 Sept 2023 - 14:00 CEST (12:00 UTC)
  • (every first Thursday of the month)
• venue: (online, see mail for meeting link, or ask in Slack)
• agenda:
  • Quick introduction by new people
  • EESSI-related meetings in last month
  • Progress update per EESSI layer (incl. build-and-deploy bot + test suite)
  • EESSI pilot repository (2023.06)
  • Outlook to new EESSI repository (*.eessi.io)
  • EESSI support portal
  • AWS/Azure sponsorship update
  • Update on MultiXscale EuroHPC project
  • Upcoming events
  • Q&A

Slides

• https://raw.githubusercontent.com/EESSI/meetings/main/meetings/EESSI_meeting_20230907.pdf

Meeting notes

(by Bob/Kenneth)

Quick introduction by new people

• Tim Kok (SURF, SKA)
  • radio-astronomy software (C++, CMake), usually provided via containers
• Pedro Santos Neves (HPC team, University of Groningen)

EESSI-related meetings in last month

(see slides)

• CernVM-FS tutorial on Best Practices in HPC
  • under development, see
    • https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices
    • https://github.com/multixscale/cvmfs-tutorial-hpc-best-practices
  • tentative date set for online tutorial: Mon 4 Dec'23
    • to be confirmed
    • will evaluate early Oct'23 to see if that's feasible

Progress update per EESSI layer

Filesystem layer

(see slides)

• ingestion slowing down, partially because we're hitting GitHub API rate limit because of increasing number of tarballs in S3 bucket
  • there are some ideas on how to improve the current approach
  • clean up S3 bucket more actively (after 1 month)
  • clone GitHub repo rather than using GitHub API
  • using GitHub token?

Compatibility layer

(see slides)

• 2023.04 will be removed on Monday, Sept 11, unless someone objects
  • Bob will send an announcement on Slack
  • This version does only have a compatibility layer, no software or init scripts

Software layer

(see slides)

• It would be nice to have a tool that can tell for which architecture / instruction sets a binary was built
  • This probably does not exist (and may be hard to create)

Build-and-deploy bot

(see slides)

• ...

EESSI pilot repository

(see slides)

• For some applications we've been seeing (too many) failing tests on neoverse_v1, but we have a good procedure in place now to deal with this (see slide 3/3)
• The Lmod cache can be updated by the Stratum 0 upon ingestion of a tarball, once the new server is in place

EESSI Contribution Policy

(see slides)

• ...

EESSI test suite

(see slides)

• ...

EESSI Infrastructure

(see slides)

• The monitoring scraper does not do anything with the actual contents of the CVMFS repo, e.g. comparing which modules are available for different architecture
  • Ghent does have a tool to generate such pages, this can be very useful for EESSI as well

EESSI Support Portal

(see slides)

• ...

AWS/Azure sponsored credits

(see slides)

• The AWS cluster will probably be rebuilt, in order to upgrade Slurm

MultiXscale EU project

(see slides)

• ...

Events

• Caspar will have a presentation called "Software distribution made EESSI" at the CompBioMed 2023 conference (organized by the CompBioMed CoE) next week in Munich, see https://www.compbiomed-conference.org

Q&A

• ...