Merge branch 'literate-nav' of github.com:C2SM/c2sm.github.io into li…

…terate-nav

docs/best_practices/coding.md

@@ -1,8 +1,9 @@
 # Coding
 
-Whether you're new to coding or already working on it, there are two important things to remember: always **use version control** and **add automatic testing**. These steps will make things easier for you and everyone you work with. Also, try out tools that can make coding easier for you.
+Whether you are new to coding or already working on it, there are two important things to remember: always **use version control** and **add automatic testing**. These steps will make things easier for you and everyone you work with. Also, try out tools that can make coding easier for you.
 
 ## Version Control with Git
+
 **Git** is a popular tool for managing source code. C2SM repositories are hosted on GitHub, which uses Git as its version control system.
 
 If you're new to Git or want to improve your Git skills, we recommend attending our annual **Git for Beginners** and/or **Git for Advanced** courses.
@@ -28,29 +29,42 @@ When working together with Git, follow these simple steps for a smooth collabora
 6. **Merge Time**: Once the revisions are done and the tests are green, merge your feature branch into the main branch.
 
 ## Automatic Testing
+
 Code testing is a critical step in software development. It's about finding and fixing problems to make sure the software works well, is of high quality and reliable.
 Testing involves checking different parts of the code to make sure the software is strong and free of bugs.
 The specific tests you need will depend on your project and its requirements. Here is a list of tests that are usually very useful.
+
 ### Unit Tests
+
 These tests are for testing individual components or functions of your code to ensure they work correctly in isolation.
-Find an [example for unit tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/unit_test.py){:target="_blank"} in our spack-c2sm repository.
+Find an [example for unit tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/unit_test.py){:target="_blank"} in our spack-c2sm repository. 
+
 ### Integrations Tests
+
 These tests are to check how different parts of your code work together and communicate with each other.
 Find an [example for integration tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/integration_test.py){:target="_blank"} in our spack-c2sm repository.
+
 ### System Tests
+
 These tests are performed to ensure that all the components and modules of a software system work together as intended and that the system meets its specified requirements and functions correctly in its operational environment.
 Find an [example for system tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/system_test.py){:target="_blank"} in our spack-c2sm repository.
+
 ### Tolerance tests
+
 These tests are used in the development of ICON, specifically when code is ported from CPU to GPU. The results when running on CPU and GPU are not bit identical, therefore a tolerance range is accepted when comparing a test case to the CPU reference. The accepted tolerance range is created by running an ensemble of the same test case with different perturbations. MeteoSwiss has developed [probtest :material-open-in-new:](https://github.com/MeteoSwiss/probtest){:target="_blank"} for handling everything related to tolerance tests with ICON. If you have a DKRZ account and are working with ICON-NWP, you can also check out the manual for [Generating tolerances for non-standard tests :material-open-in-new:](https://gitlab.dkrz.de/icon/wiki/-/wikis/GPU-development/Validating-with-probtest-without-buildbot-references-(Generating-tolerances-for-non-standard-tests){:target="_blank"}).
+
 ### Git Hooks & GitHub Actions
+
 Git Hooks are local scripts in Git that make sure things get done right when you work on your code. GitHub Actions, on the other hand, are integrated with GitHub and allow you to automate code management workflows. They can be run automatically on GitHub whenever something is committed.
 
 Check out our Git course for examples of [Custom Git Hooks :material-open-in-new:](https://github.com/C2SM/git-course/blob/main/advanced/Exercise_7_git-hooks.md){:target="_blank"} hooks, and read GitHub's [documentation on GitHub Actions :material-open-in-new:](https://docs.github.com/en/actions){:target="_blank"}.
 
-
 ## Useful tools for coding
+
 Two popular tools for coding are [Visual Studio (VS) Code :material-open-in-new:](https://code.visualstudio.com){:target="_blank"} and [PyCharm :material-open-in-new:](https://www.jetbrains.com/pycharm/){:target="_blank"}. Here are instructions for setting up and using Visual Studio Code with SSH on a CSCS machine.
+
 ### VS Code
+
 1. [Download :material-open-in-new:](https://code.visualstudio.com/download){:target="_blank"} and install VS Code on your computer.
 2. Install extensions:
     - Open VS Code.

docs/datasets/climate_model_data.md

@@ -111,8 +111,8 @@
     /nfs/atmos/c2sm/cmip6-ng
     ```
 
-- Size: 200.63 TB :material-information-outline:{ title="last updated: 2024-08-11 01:47:43" }
-- Number of files: 663,802 :material-information-outline:{ title="last updated: 2024-08-11 01:47:43" }
+- Size: 200.63 TB :material-information-outline:{ title="last updated: 2024-08-18 01:47:09" }
+- Number of files: 663,802 :material-information-outline:{ title="last updated: 2024-08-18 01:47:09" }
 - Access: direct / rsync
 - Status: frozen (2019-03)
 - Variables: 
@@ -195,8 +195,8 @@
     /store/c2sm/c2sme/cordex
     ```
 
-- Size: 356.97 TB :material-information-outline:{ title="last updated: 2024-08-11 01:48:16" }
-- Number of files: 552,595 :material-information-outline:{ title="last updated: 2024-08-11 01:48:16" }
+- Size: 356.97 TB :material-information-outline:{ title="last updated: 2024-08-18 01:48:54" }
+- Number of files: 552,595 :material-information-outline:{ title="last updated: 2024-08-18 01:48:54" }
 - Access: direct / rsync
 - Status: monthly updated
 - Resolution: 0.44° and 0.11°
@@ -211,8 +211,8 @@
     ```console
     /nfs/atmos/c2sm/cordex-reklies
     ```
-- Size: 22.41 TB :material-information-outline:{ title="last updated: 2024-08-11 01:45:25" }
-- Number of files: 94,936 :material-information-outline:{ title="last updated: 2024-08-11 01:45:25" }
+- Size: 22.41 TB :material-information-outline:{ title="last updated: 2024-08-18 01:45:22" }
+- Number of files: 94,936 :material-information-outline:{ title="last updated: 2024-08-18 01:45:22" }
 - Access: direct
 - Status: monthly updated
 - Resolution: 0.11°

docs/datasets/obs_reanalysis_data.md

@@ -157,8 +157,8 @@
     /net/atmos/data/cerra/processed/v1/
     ```
 
-- Size: 7.72 TB :material-information-outline:{ title="last updated: 2024-08-11 01:44:39" }
-- Number of files: 1,824 :material-information-outline:{ title="last updated: 2024-08-11 01:44:39" }
+- Size: 7.72 TB :material-information-outline:{ title="last updated: 2024-08-18 01:44:42" }
+- Number of files: 1,824 :material-information-outline:{ title="last updated: 2024-08-18 01:44:42" }
 - Access: direct
 - Status: updated
 - Time period: 1985-2020
@@ -199,8 +199,8 @@
     /net/atmos/data/cerra-land/processed/v1/
     ```
 
-- Size: 1.78 TB :material-information-outline:{ title="last updated: 2024-08-11 01:44:39" }
-- Number of files: 1,016 :material-information-outline:{ title="last updated: 2024-08-11 01:44:39" }
+- Size: 1.78 TB :material-information-outline:{ title="last updated: 2024-08-18 01:44:42" }
+- Number of files: 1,016 :material-information-outline:{ title="last updated: 2024-08-18 01:44:42" }
 - Access: direct
 - Status: updated
 - Time period: 1985-2020
@@ -253,8 +253,8 @@
     /nfs/atmos/c2sm/era5/processed/v2
     ```
 
-- Size: 45.76 TB :material-information-outline:{ title="last updated: 2024-08-11 01:45:18" }
-- Number of files: 138,400 :material-information-outline:{ title="last updated: 2024-08-11 01:45:18" }
+- Size: 45.76 TB :material-information-outline:{ title="last updated: 2024-08-18 01:45:24" }
+- Number of files: 138,400 :material-information-outline:{ title="last updated: 2024-08-18 01:45:24" }
 - Access: direct
 - Status: updated
 - Time period: v1: 1940-present, v2: 1980-present
@@ -294,8 +294,8 @@
     /net/atmos/data/era5-land_cds/processed/v1/
     ```
 
-- Size: 6.64 TB :material-information-outline:{ title="last updated: 2024-08-11 01:44:40" }
-- Number of files: 1,504 :material-information-outline:{ title="last updated: 2024-08-11 01:44:40" }
+- Size: 6.64 TB :material-information-outline:{ title="last updated: 2024-08-18 01:44:44" }
+- Number of files: 1,504 :material-information-outline:{ title="last updated: 2024-08-18 01:44:44" }
 - Access: direct
 - Status: updated
 - Time period: 1950-present

scripts/format_external_links.py

@@ -3,27 +3,44 @@
 import argparse
 
 def modify_link(line):
+    replaced = False
+
+    # Define icons and attributes
+    icon_external_link = ':material-open-in-new:'
+    icon_download = ':material-download:'
+    open_new_tab = '{:target="_blank"}'
+
     # Define patterns for general and download links
     general_pattern = r'\[([^\]]+)\]\((http[s]?://[^\s\)]+)\)'
     download_pattern = r'\[([^\]]+)\]\((https://polybox\.ethz\.ch/index\.php/s/[^\s\)]+)\)'
 
     # Define replacements for general and download links
-    general_replacement = r'[\1 :material-open-in-new:](\2){:target="_blank"}'
-    download_replacement = r'[\1 :material-download:](\2){:target="_blank"}'
+    general_replacement = r'[\1 ' + icon_external_link + r'](\2)'
+    download_replacement = r'[\1 ' + icon_download + r'](\2)'
 
-    # Check if the line was already modified or doesn't need modification
-    if ':material-open-in-new:' in line and '{:target="_blank"}' in line:
-        return line, False
-    if ':material-download:' in line and '{:target="_blank"}' in line:
-        return line, False
+    # Check if the general pattern appears in the line
+    if re.search(general_pattern, line):
+        # Check for link icon
+        if icon_external_link not in line and icon_download not in line:
+            new_line = re.sub(download_pattern, download_replacement, line)
+            if new_line != line:
+                line = new_line
+                replaced = True
+            else:
+                new_line = re.sub(general_pattern, general_replacement, line)
+                if new_line != line:
+                    line = new_line
+                    replaced = True
 
-    # Apply the appropriate replacement based on the URL pattern
-    if re.search(download_pattern, line):
-        modified_line, num_subs = re.subn(download_pattern, download_replacement, line)
-    else:
-        modified_line, num_subs = re.subn(general_pattern, general_replacement, line)
+        # Check for new tab attribute
+        if open_new_tab not in line:
+            new_line = re.sub(r'(\[.*?\]\(.*?\))', r'\1' + open_new_tab, line)
+            if new_line != line:
+                line = new_line
+                replaced = True
+
+    return line, replaced
 
-    return modified_line, num_subs > 0
 
 def process_markdown_file(file_path):
     with open(file_path, 'r+', encoding='utf-8') as file:
@@ -34,11 +51,13 @@ def process_markdown_file(file_path):
             if changed:
                 modified = True
                 lines[i] = new_line
+                print(f"Modifying line {i+1} in file: {file_path}")
         if modified:
             file.seek(0)
             file.writelines(lines)
             file.truncate()
-            print(f"Modified: {file_path}")
+            print(f"File modified: {file_path}")
+
 
 def main(start_path):
     for root, dirs, files in os.walk(start_path):
@@ -51,4 +70,4 @@ def main(start_path):
     parser.add_argument('-p', '--path', default=os.getcwd(), help='Base path to search for markdown files. Defaults to current working directory.')
     args = parser.parse_args()
 
-    main(args.path)
+    main(args.path)