Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Link Formatter #187

Merged
merged 28 commits into from
Aug 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
804e137
Check partly formatted links
mjaehn Aug 15, 2024
4780dc1
Push md to trigger action
mjaehn Aug 15, 2024
7c0ffde
Restore argparse
mjaehn Aug 15, 2024
6b1b0bf
Fix argparse
mjaehn Aug 15, 2024
4564422
Push md again
mjaehn Aug 15, 2024
ac52bd8
Fix formatter
mjaehn Aug 15, 2024
41e67dc
Test new formatter
mjaehn Aug 15, 2024
7e97b06
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
a530344
Test again
mjaehn Aug 15, 2024
be5e5c1
Restore function
mjaehn Aug 15, 2024
077d566
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
27ab76c
Fix and test
mjaehn Aug 15, 2024
1893399
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
ba5a9dc
Fix and test
mjaehn Aug 15, 2024
b4e04fd
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
9968f7e
Fix and test
mjaehn Aug 15, 2024
3788ffa
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
79df8e4
Fix formatter
mjaehn Aug 15, 2024
e5876aa
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
64aea0c
Cleanup
mjaehn Aug 15, 2024
c625863
Revert "GitHub Action: Apply external link format"
mjaehn Aug 15, 2024
0652faf
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
113cbbf
Revert "GitHub Action: Apply external link format"
mjaehn Aug 15, 2024
78d5a36
Only check if pattern appears
mjaehn Aug 15, 2024
7f9dc38
Fix pattern check
mjaehn Aug 15, 2024
e25caa2
Trigger test
mjaehn Aug 15, 2024
9624341
GitHub Action: Apply external link format
mjaehn Aug 15, 2024
f9589e7
Update coding.md
mjaehn Aug 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions docs/best_practices/coding.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Coding

Whether you're new to coding or already working on it, there are two important things to remember: always **use version control** and **add automatic testing**. These steps will make things easier for you and everyone you work with. Also, try out tools that can make coding easier for you.
Whether you are new to coding or already working on it, there are two important things to remember: always **use version control** and **add automatic testing**. These steps will make things easier for you and everyone you work with. Also, try out tools that can make coding easier for you.

## Version Control with Git

**Git** is a popular tool for managing source code. C2SM repositories are hosted on GitHub, which uses Git as its version control system.

If you're new to Git or want to improve your Git skills, we recommend attending our annual **Git for Beginners** and/or **Git for Advanced** courses.
Expand All @@ -28,29 +29,42 @@ When working together with Git, follow these simple steps for a smooth collabora
6. **Merge Time**: Once the revisions are done and the tests are green, merge your feature branch into the main branch.

## Automatic Testing

Code testing is a critical step in software development. It's about finding and fixing problems to make sure the software works well, is of high quality and reliable.
Testing involves checking different parts of the code to make sure the software is strong and free of bugs.
The specific tests you need will depend on your project and its requirements. Here is a list of tests that are usually very useful.

### Unit Tests

These tests are for testing individual components or functions of your code to ensure they work correctly in isolation.
Find an [example for unit tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/unit_test.py){:target="_blank"} in our spack-c2sm repository.
Find an [example for unit tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/unit_test.py){:target="_blank"} in our spack-c2sm repository.

### Integrations Tests

These tests are to check how different parts of your code work together and communicate with each other.
Find an [example for integration tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/integration_test.py){:target="_blank"} in our spack-c2sm repository.

### System Tests

These tests are performed to ensure that all the components and modules of a software system work together as intended and that the system meets its specified requirements and functions correctly in its operational environment.
Find an [example for system tests :material-open-in-new:](https://github.com/C2SM/spack-c2sm/blob/main/test/system_test.py){:target="_blank"} in our spack-c2sm repository.

### Tolerance tests

These tests are used in the development of ICON, specifically when code is ported from CPU to GPU. The results when running on CPU and GPU are not bit identical, therefore a tolerance range is accepted when comparing a test case to the CPU reference. The accepted tolerance range is created by running an ensemble of the same test case with different perturbations. MeteoSwiss has developed [probtest :material-open-in-new:](https://github.com/MeteoSwiss/probtest){:target="_blank"} for handling everything related to tolerance tests with ICON. If you have a DKRZ account and are working with ICON-NWP, you can also check out the manual for [Generating tolerances for non-standard tests :material-open-in-new:](https://gitlab.dkrz.de/icon/wiki/-/wikis/GPU-development/Validating-with-probtest-without-buildbot-references-(Generating-tolerances-for-non-standard-tests){:target="_blank"}).

### Git Hooks & GitHub Actions

Git Hooks are local scripts in Git that make sure things get done right when you work on your code. GitHub Actions, on the other hand, are integrated with GitHub and allow you to automate code management workflows. They can be run automatically on GitHub whenever something is committed.

Check out our Git course for examples of [Custom Git Hooks :material-open-in-new:](https://github.com/C2SM/git-course/blob/main/advanced/Exercise_7_git-hooks.md){:target="_blank"} hooks, and read GitHub's [documentation on GitHub Actions :material-open-in-new:](https://docs.github.com/en/actions){:target="_blank"}.


## Useful tools for coding

Two popular tools for coding are [Visual Studio (VS) Code :material-open-in-new:](https://code.visualstudio.com){:target="_blank"} and [PyCharm :material-open-in-new:](https://www.jetbrains.com/pycharm/){:target="_blank"}. Here are instructions for setting up and using Visual Studio Code with SSH on a CSCS machine.

### VS Code

1. [Download :material-open-in-new:](https://code.visualstudio.com/download){:target="_blank"} and install VS Code on your computer.
2. Install extensions:
- Open VS Code.
Expand Down
49 changes: 34 additions & 15 deletions scripts/format_external_links.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,44 @@
import argparse

def modify_link(line):
replaced = False

# Define icons and attributes
icon_external_link = ':material-open-in-new:'
icon_download = ':material-download:'
open_new_tab = '{:target="_blank"}'

# Define patterns for general and download links
general_pattern = r'\[([^\]]+)\]\((http[s]?://[^\s\)]+)\)'
download_pattern = r'\[([^\]]+)\]\((https://polybox\.ethz\.ch/index\.php/s/[^\s\)]+)\)'

# Define replacements for general and download links
general_replacement = r'[\1 :material-open-in-new:](\2){:target="_blank"}'
download_replacement = r'[\1 :material-download:](\2){:target="_blank"}'
general_replacement = r'[\1 ' + icon_external_link + r'](\2)'
download_replacement = r'[\1 ' + icon_download + r'](\2)'

# Check if the line was already modified or doesn't need modification
if ':material-open-in-new:' in line and '{:target="_blank"}' in line:
return line, False
if ':material-download:' in line and '{:target="_blank"}' in line:
return line, False
# Check if the general pattern appears in the line
if re.search(general_pattern, line):
# Check for link icon
if icon_external_link not in line and icon_download not in line:
new_line = re.sub(download_pattern, download_replacement, line)
if new_line != line:
line = new_line
replaced = True
else:
new_line = re.sub(general_pattern, general_replacement, line)
if new_line != line:
line = new_line
replaced = True

# Apply the appropriate replacement based on the URL pattern
if re.search(download_pattern, line):
modified_line, num_subs = re.subn(download_pattern, download_replacement, line)
else:
modified_line, num_subs = re.subn(general_pattern, general_replacement, line)
# Check for new tab attribute
if open_new_tab not in line:
new_line = re.sub(r'(\[.*?\]\(.*?\))', r'\1' + open_new_tab, line)
if new_line != line:
line = new_line
replaced = True

return line, replaced

return modified_line, num_subs > 0

def process_markdown_file(file_path):
with open(file_path, 'r+', encoding='utf-8') as file:
Expand All @@ -34,11 +51,13 @@ def process_markdown_file(file_path):
if changed:
modified = True
lines[i] = new_line
print(f"Modifying line {i+1} in file: {file_path}")
if modified:
file.seek(0)
file.writelines(lines)
file.truncate()
print(f"Modified: {file_path}")
print(f"File modified: {file_path}")


def main(start_path):
for root, dirs, files in os.walk(start_path):
Expand All @@ -51,4 +70,4 @@ def main(start_path):
parser.add_argument('-p', '--path', default=os.getcwd(), help='Base path to search for markdown files. Defaults to current working directory.')
args = parser.parse_args()

main(args.path)
main(args.path)
Loading