diff --git a/.github/workflows/update_datasets.yml b/.github/workflows/update_datasets.yml index 3d58d2fc..cd0f4d31 100644 --- a/.github/workflows/update_datasets.yml +++ b/.github/workflows/update_datasets.yml @@ -46,7 +46,7 @@ jobs: if: steps.git-check.outputs.changes_detected == 'true' uses: peter-evans/create-pull-request@v4 with: - token: ${{ secrets.GITHUB_TOKEN }} + token: ${{ secrets.PERSONAL_ACCESS_TOKEN }} commit-message: "GitHub Action: Update datasets" title: "[AUTO] Update Datasets" body: "This is an auto-generated PR with dataset updates." diff --git a/docs/blog/posts/2024-07-02_switching_to_Alps.md b/docs/blog/posts/2024-07-02_switching_to_Alps.md index 0256774d..75459ccf 100644 --- a/docs/blog/posts/2024-07-02_switching_to_Alps.md +++ b/docs/blog/posts/2024-07-02_switching_to_Alps.md @@ -7,7 +7,7 @@ date: As you probably all know the new [Alps system :material-open-in-new:](https://www.cscs.ch/computers/alps){:target="_blank"} is currently being set up at CSCS. With this machine, CSCS introduced major changes, of course in the hardware, but also in the user experience. -Piz Daint being scheduled to be decommissioned at the end of August, we are organizing a workshop to guide our users in this transition on **August, 12th from 10:00 to 12:00** at **ETHZ in CHN-L17.1** and over zoom at **https://ethz.zoom.us/j/63402117251.** +Piz Daint being scheduled to be decommissioned at the end of August, we are organizing a workshop to guide our users in this transition on **August, 12th from 10:00 to 12:00** at **ETHZ in CHN-L17.1** and over zoom at **https://ethz.zoom.us/j/63402117251**. The event will be recorded and the password for online participation will be sent the day before. We will cover the following: diff --git a/docs/datasets/climate_model_data.md b/docs/datasets/climate_model_data.md index 6da9ae19..9e083749 100644 --- a/docs/datasets/climate_model_data.md +++ b/docs/datasets/climate_model_data.md @@ -10,7 +10,11 @@ ### CMIP2 -- Location: IAC +=== "IAC" + ```console + /net/atmos/data/cmip2 + ``` + - Size: 0.01 TB - Number of files: 500 - Access: direct / rsync @@ -19,7 +23,11 @@ ### CMIP3 -- Location: IAC +=== "IAC" + ```console + /net/atmos/data/cmip3 + ``` + - Size: 6 TB - Number of files: 40.000 - Access: direct / rsync @@ -28,7 +36,11 @@ ### CMIP5 -- Location: IAC +=== "IAC" + ```console + /net/atmos/data/cmip5 + ``` + - Size: 130 TB - Number of files: 700.000 - Access: direct / rsync @@ -39,11 +51,11 @@ === "IAC" ```console - /nfs/atmos/c2sm + /net/atmos/data/cmip6 ``` === "Euler" ```console - /nfs/atmos/c2sm + /nfs/atmos/c2sm/cmip6 ``` - Size: 520 TB @@ -62,7 +74,11 @@ ### CMIP3-ng -- Location: IAC +=== "IAC" + ```console + /net/atmos/data/cmip3-ng + ``` + - Size: 0.5 TB - Number of files: 5.000 - Access: direct / rsync @@ -72,7 +88,11 @@ ### CMIP5-ng -- Location: IAC +=== "IAC" + ```console + /net/atmos/data/cmip5-ng + ``` + - Size: 33 TB - Number of files: 100.000 - Access: direct / rsync @@ -84,15 +104,15 @@ === "IAC" ```console - /nfs/atmos/c2sm + /net/atmos/data/cmip6-ng ``` === "Euler" ```console - /nfs/atmos/c2sm + /nfs/atmos/c2sm/cmip6-ng ``` -- Size: 200.07 TB :material-information-outline:{ title="last updated: 2024-07-05 01:46:11" } -- Number of files: 663,456 :material-information-outline:{ title="last updated: 2024-07-05 01:46:11" } +- Size: 200.55 TB :material-information-outline:{ title="last updated: 2024-07-10 01:45:44" } +- Number of files: 663,772 :material-information-outline:{ title="last updated: 2024-07-10 01:45:44" } - Access: direct / rsync - Status: frozen (2019-03) - Variables: @@ -164,28 +184,35 @@ === "IAC" ```console - /nfs/atmos/c2sm + /net/atmos/data/cordex ``` === "Euler" ```console - /nfs/atmos/c2sm + /nfs/atmos/c2sm/cordex ``` === "CSCS" ```console - /store/c2sm/c2sme + /store/c2sm/c2sme/cordex ``` -- Size: 356.97 TB :material-information-outline:{ title="last updated: 2024-07-05 01:46:37" } -- Number of files: 552,595 :material-information-outline:{ title="last updated: 2024-07-05 01:46:37" } +- Size: 356.97 TB :material-information-outline:{ title="last updated: 2024-07-10 01:45:53" } +- Number of files: 552,595 :material-information-outline:{ title="last updated: 2024-07-10 01:45:53" } - Access: direct / rsync - Status: monthly updated - Resolution: 0.44° and 0.11° ### CORDEX-ReKliEs -- Location: IAC, Euler -- Size: 22.41 TB :material-information-outline:{ title="last updated: 2024-07-05 01:44:05" } -- Number of files: 94,936 :material-information-outline:{ title="last updated: 2024-07-05 01:44:05" } +=== "IAC" + ```console + /net/atmos/data/cordex-reklies + ``` +=== "Euler" + ```console + /nfs/atmos/c2sm/cordex-reklies + ``` +- Size: 22.41 TB :material-information-outline:{ title="last updated: 2024-07-10 01:43:06" } +- Number of files: 94,936 :material-information-outline:{ title="last updated: 2024-07-10 01:43:06" } - Access: direct - Status: monthly updated - Resolution: 0.11° @@ -197,7 +224,14 @@ ### CORDEX.ch2018 -- Location: IAC, CSCS +=== "IAC" + ```console + /net/ch4/data/cordex.ch2018-freeze-2.1 + ``` +=== "CSCS" + ```console + /store/c2sm/c2sme/cordex.ch2018-freeze-2.1 + ``` - Size: 11 TB - Number of files: 1.800 - Access: direct diff --git a/docs/datasets/index.md b/docs/datasets/index.md index a651c608..2e09969e 100644 --- a/docs/datasets/index.md +++ b/docs/datasets/index.md @@ -3,4 +3,6 @@ C2SM provides different climate datasets such as CMIP5, CMIP6 or CORDEX, as downloaded from the original sources, as well as processed datasets such as the so-called CMIP6-NextGeneration (CMIP6-ng) archive. -If you need access to other climate datasets contact us at [support@c2sm.ethz.ch](mailto:support@c2sm.ethz.ch) or add an [issue :material-open-in-new:](https://github.com/C2SM/Tasks-Support/issues){:target="_blank"} and we will support you with your request. +Since July 2024, C2SM offers access to climate model and reanalysis data via our web-based climate data extraction tool [Zephyr](../tools/zephyr.md), available at [https://zephyr.ethz.ch :material-open-in-new:](https://zephyr.ethz.ch){:target="_blank"}. Zephyr allows you to request climate data from C2SM and simultaneously extract specific regions or individual grid points directly from your browser. Once your request has been submitted and successfully processed, you will receive a download link to access your requested data directly from your web browser. + +If you need access to other climate datasets, contact us at [support@c2sm.ethz.ch](mailto:support@c2sm.ethz.ch) or add an [issue :material-open-in-new:](https://github.com/C2SM/Tasks-Support/issues){:target="_blank"} and we will support you with your request. diff --git a/docs/datasets/obs_reanalysis_data.md b/docs/datasets/obs_reanalysis_data.md index 3723d0e0..a30686db 100644 --- a/docs/datasets/obs_reanalysis_data.md +++ b/docs/datasets/obs_reanalysis_data.md @@ -156,8 +156,8 @@ /net/atmos/data/cerra/processed/v1/ ``` -- Size: 7.72 TB :material-information-outline:{ title="last updated: 2024-07-05 01:43:36" } -- Number of files: 1,824 :material-information-outline:{ title="last updated: 2024-07-05 01:43:36" } +- Size: 7.72 TB :material-information-outline:{ title="last updated: 2024-07-10 01:42:37" } +- Number of files: 1,824 :material-information-outline:{ title="last updated: 2024-07-10 01:42:37" } - Access: direct - Status: updated - Time period: 1985-2020 @@ -198,8 +198,8 @@ /net/atmos/data/cerra-land/processed/v1/ ``` -- Size: 1.78 TB :material-information-outline:{ title="last updated: 2024-07-05 01:43:36" } -- Number of files: 1,016 :material-information-outline:{ title="last updated: 2024-07-05 01:43:36" } +- Size: 1.78 TB :material-information-outline:{ title="last updated: 2024-07-10 01:42:37" } +- Number of files: 1,016 :material-information-outline:{ title="last updated: 2024-07-10 01:42:37" } - Access: direct - Status: updated - Time period: 1985-2020 @@ -249,11 +249,11 @@ ``` === "Euler" ```console - /net/atmos/data/era5_cds/processed/ + /nfs/atmos/c2sm/era5/processed/v2 ``` -- Size: 45.76 TB :material-information-outline:{ title="last updated: 2024-07-05 01:44:33" } -- Number of files: 138,400 :material-information-outline:{ title="last updated: 2024-07-05 01:44:33" } +- Size: 45.76 TB :material-information-outline:{ title="last updated: 2024-07-10 01:44:26" } +- Number of files: 138,400 :material-information-outline:{ title="last updated: 2024-07-10 01:44:26" } - Access: direct - Status: updated - Time period: v1: 1940-present, v2: 1980-present @@ -293,8 +293,8 @@ /net/atmos/data/era5-land_cds/processed/v1/ ``` -- Size: 6.64 TB :material-information-outline:{ title="last updated: 2024-07-05 01:43:38" } -- Number of files: 1,504 :material-information-outline:{ title="last updated: 2024-07-05 01:43:38" } +- Size: 6.64 TB :material-information-outline:{ title="last updated: 2024-07-10 01:42:40" } +- Number of files: 1,504 :material-information-outline:{ title="last updated: 2024-07-10 01:42:40" } - Access: direct - Status: updated - Time period: 1950-present diff --git a/docs/models/icon.md b/docs/models/icon.md index eb269409..f208eedb 100644 --- a/docs/models/icon.md +++ b/docs/models/icon.md @@ -62,11 +62,6 @@ srun -N 1 -c 12 --mem-per-cpu=20G spack install -v -j 12 - The resulting executable is not tested. - Anything can go wrong -- Install latest `uenv` tool - ```shell - git clone git@github.com:eth-cscs/uenv.git && ./uenv/install --local - ``` - - Get `spack-c2sm` on the `uenv_ci` branch ```shell git clone --depth 1 --recurse-submodules --shallow-submodules -b uenv_ci https://github.com/C2SM/spack-c2sm.git diff --git a/docs/tools/zephyr.md b/docs/tools/zephyr.md index 554e602a..8f620758 100644 --- a/docs/tools/zephyr.md +++ b/docs/tools/zephyr.md @@ -11,16 +11,16 @@ Scans of the datasets listed below are performed daily, generating file trees in ## Features -Requests are made via the [Zephyr website :material-open-in-new:](https://zephyr.ethz.ch){:target="_blank"} and submitted using the [Zephyr data request template :material-open-in-new:](https://github.com/C2SM/zephyr-request/issues/new/choose){:target="_blank"}. Zephyr supports a range of climate models and reanalysis datasets, including: - -- CMIP5 -- CMIP6 -- CORDEX -- CORDEX ReKliEs -- ERA5 -- ERA5 land -- CERRA -- CERRA land +Requests are made via the [Zephyr website :material-open-in-new:](https://zephyr.ethz.ch){:target="_blank"} and submitted using the [Zephyr data request template :material-open-in-new:](https://github.com/C2SM/zephyr-request/issues/new/choose){:target="_blank"}. Zephyr supports a range of [climate models and reanalysis datasets](../datasets/index.md), including: + +- [CMIP5](../datasets/climate_model_data.md#cmip5) +- [CMIP6](../datasets/climate_model_data.md#cmip6) +- [CORDEX](../datasets/climate_model_data.md#cordex) +- [CORDEX ReKliEs](../datasets/climate_model_data.md#cordex-reklies) +- [ERA5](../datasets/obs_reanalysis_data.md#era5_1) +- [ERA5 land](../datasets/obs_reanalysis_data.md#era5-land_1) +- [CERRA](../datasets/obs_reanalysis_data.md#cerra_1) +- [CERRA land](../datasets/obs_reanalysis_data.md#cerra-land_1) Zephyr efficiently extracts both regional and global climate datasets, with options to retrieve data at individual nearest grid points or within a user-defined rectangular area. Once a request has been successfully submitted and processed, a download link will be posted to the GitHub issue thread where the request was submitted. The download link will be available for up to seven days.