Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚠️ CONFLICT! Lineage pull request for: skeleton #34

Open
wants to merge 62 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
33582a1
Add checks for semantic python versions
Feb 14, 2024
9438194
Refactor code for the semantic check
Feb 14, 2024
cea8edc
Add checks for semantic python versions
Feb 14, 2024
d5c7c4a
Refactor code for the semantic check
Feb 14, 2024
f7b9d05
Merge branch 'improvement/correct-semantic-python-version-checks' of …
Mar 18, 2024
327ab73
Remove example of correct semantic version
Mar 18, 2024
4dedf50
Refactor the error message for the user
Mar 20, 2024
e84deea
Improve the semantic error message
Mar 20, 2024
5fdc7be
Fix grammar
michaelsaki Mar 20, 2024
42ef8c2
Refactor regex, add link, and improve comments
Mar 21, 2024
a77e5e1
Update link to use semver.org over regex101.com
Mar 21, 2024
5fe14c7
Remove unnecessary period
michaelsaki Mar 21, 2024
b7896a0
Add a meta hook to the pre-commit configuration
mcdonnnj Apr 20, 2024
260566f
Remove `exclude` directive that does not apply to any files
mcdonnnj Apr 20, 2024
07e2b60
Pin ansible-core when running the ansible-lint linter
jsf9k May 4, 2024
c74e5db
Remove unnecessary line
jsf9k May 4, 2024
2e53e0d
Explain why ansible may need to be added as a dependency for ansible-…
jsf9k May 7, 2024
f51fe62
Update pre-commit hook versions
mcdonnnj May 11, 2024
8e55b8e
Manually update the prettier hook
mcdonnnj Jun 6, 2024
c617bb9
Correct commented-out ansible pin
jsf9k Jun 6, 2024
de92c3c
Merge pull request #183 from cisagov/maintenance/update_pre-commit_hooks
mcdonnnj Jun 6, 2024
7ddea47
Merge pull request #182 from cisagov/improvement/pin-packages-for-ans…
mcdonnnj Jun 6, 2024
a68994d
Add a lower-bound pin for flake8-docstrings
jsf9k Jul 1, 2024
43b91c7
Use the hashicorp/setup-packer GitHub Action
mcdonnnj Aug 12, 2024
8ada75d
Remove @jasonodoom as a codeowner
jsf9k Aug 23, 2024
2930208
Pin to a specific version
jsf9k Aug 26, 2024
46e0553
Bump actions/cache from 3 to 4
dependabot[bot] Sep 24, 2024
3167421
Bump crazy-max/ghaction-github-status from 3 to 4
dependabot[bot] Sep 24, 2024
6a58c2c
Update pre-commit hook versions
mcdonnnj Aug 12, 2024
553efcb
Manually update the prettier hook
mcdonnnj Aug 12, 2024
184e749
Merge pull request #170 from cisagov/dependabot/github_actions/crazy-…
mcdonnnj Sep 25, 2024
d99c117
Merge pull request #171 from cisagov/dependabot/github_actions/action…
mcdonnnj Sep 25, 2024
2491ca0
Merge pull request #187 from cisagov/improvement/use_setup_packer_action
mcdonnnj Sep 25, 2024
f6c9537
Merge pull request #176 from cisagov/improvement/correct-semantic-pyt…
mcdonnnj Sep 25, 2024
10e5f6f
Merge pull request #188 from cisagov/remove-odoom-as-a-codeowner
mcdonnnj Sep 25, 2024
045a998
Add a pre-commit hook to run pip-audit
mcdonnnj Jan 18, 2024
28dc4ce
Merge pull request #178 from cisagov/improvement/add_pre-commit_meta_…
mcdonnnj Sep 25, 2024
5801cec
Merge pull request #179 from cisagov/improvement/add_pip-audit_pre-co…
mcdonnnj Sep 25, 2024
e61255c
Merge pull request #184 from cisagov/improvement/pin-flake8-docstrings
mcdonnnj Sep 25, 2024
44cd1be
Merge pull request #185 from cisagov/maintenance/update_pre-commit_hooks
mcdonnnj Sep 25, 2024
c502f1a
Use the rbubley/mirrors-prettier hook for prettier
mcdonnnj Aug 12, 2024
f0e55b3
Merge pull request #186 from cisagov/improvement/switch_prettier_hook
mcdonnnj Sep 25, 2024
97f40fc
Merge remote-tracking branch 'skeleton-generic/develop' into lineage/…
jsf9k Sep 25, 2024
e452973
Upgrade an instance of actions/cache to v4
jsf9k Sep 25, 2024
a06c2b9
Upgrade bandit to 1.7.10
jsf9k Sep 25, 2024
942c0dc
Add a new trigger for the sync-labels GitHub Actions workflow
mcdonnnj Aug 13, 2024
a267662
Remove unnecessary quotes in the sync-labels workflow
mcdonnnj Aug 13, 2024
dc7f09e
Add four new hooks from pre-commit/pre-commit-hooks
mcdonnnj Sep 14, 2024
343d2cc
Add the GitHubSecurityLab/actions-permissions/monitor Action
mcdonnnj Oct 28, 2024
8a77a8b
Restrict permissions of GITHUB_TOKEN
mcdonnnj Oct 28, 2024
3b1d4ef
Update pre-commit hook versions
mcdonnnj Oct 16, 2024
1d285f2
Sort hook ids in each pre-commit hook entry
mcdonnnj Oct 29, 2024
5da1059
Merge pull request #189 from cisagov/improvement/manually_run_sync-la…
mcdonnnj Oct 30, 2024
ff221ba
Merge pull request #190 from cisagov/improvement/add_actions-permissi…
mcdonnnj Oct 30, 2024
971602a
Merge pull request #191 from cisagov/improvement/github_tokenn_polp
mcdonnnj Oct 30, 2024
bdf8a25
Merge pull request #192 from cisagov/maintenance/update_pre-commit_hooks
mcdonnnj Oct 30, 2024
6959971
Merge pull request #193 from cisagov/improvement/add_more_pre-commit_…
mcdonnnj Oct 30, 2024
f517db7
Merge pull request #194 from cisagov/improvement/ensure_pre-commit_ho…
mcdonnnj Oct 30, 2024
acd9a95
Merge https://github.com/cisagov/skeleton-generic into lineage/skeleton
Oct 30, 2024
8824475
Update the commented out dependabot ignore directives
mcdonnnj Nov 1, 2024
e6afb68
Merge pull request #195 from cisagov/bug/add_missing_dependabot_ignore
mcdonnnj Nov 1, 2024
8d15053
Merge https://github.com/cisagov/skeleton-generic into lineage/skeleton
Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,22 @@
# These owners will be the default owners for everything in the
# repo. Unless a later match takes precedence, these owners will be
# requested for review when someone opens a pull request.
* @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
* @dav3r @felddy @jsf9k @mcdonnnj

# These folks own any files in the .github directory at the root of
# the repository and any of its subdirectories.
/.github/ @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.github/ @dav3r @felddy @jsf9k @mcdonnnj

# These folks own all linting configuration files.
/.ansible-lint @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.bandit.yml @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.flake8 @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.isort.cfg @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.mdl_config.yaml @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.pre-commit-config.yaml @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.prettierignore @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.yamllint @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/requirements.txt @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/requirements-dev.txt @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/requirements-test.txt @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/setup-env @dav3r @felddy @jasonodoom @jsf9k @mcdonnnj
/.ansible-lint @dav3r @felddy @jsf9k @mcdonnnj
/.bandit.yml @dav3r @felddy @jsf9k @mcdonnnj
/.flake8 @dav3r @felddy @jsf9k @mcdonnnj
/.isort.cfg @dav3r @felddy @jsf9k @mcdonnnj
/.mdl_config.yaml @dav3r @felddy @jsf9k @mcdonnnj
/.pre-commit-config.yaml @dav3r @felddy @jsf9k @mcdonnnj
/.prettierignore @dav3r @felddy @jsf9k @mcdonnnj
/.yamllint @dav3r @felddy @jsf9k @mcdonnnj
/requirements.txt @dav3r @felddy @jsf9k @mcdonnnj
/requirements-dev.txt @dav3r @felddy @jsf9k @mcdonnnj
/requirements-test.txt @dav3r @felddy @jsf9k @mcdonnnj
/setup-env @dav3r @felddy @jsf9k @mcdonnnj
24 changes: 5 additions & 19 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ defaults:
shell: bash -Eueo pipefail -x {0}

env:
CURL_CACHE_DIR: ~/.cache/curl
DEFAULT_ARTIFACT_NAME: lambda_build.zip
PIP_CACHE_DIR: ~/.cache/pip
PRE_COMMIT_CACHE_DIR: ~/.cache/pre-commit
Expand Down Expand Up @@ -77,7 +76,7 @@ jobs:
name: Lookup Go cache directory
run: |
echo "dir=$(go env GOCACHE)" >> $GITHUB_OUTPUT
- uses: actions/cache@v3
- uses: actions/cache@v4
env:
BASE_CACHE_KEY: "${{ github.job }}-${{ runner.os }}-\
py${{ steps.setup-python.outputs.python-version }}-\
Expand All @@ -98,25 +97,12 @@ jobs:
path: |
${{ env.PIP_CACHE_DIR }}
${{ env.PRE_COMMIT_CACHE_DIR }}
${{ env.CURL_CACHE_DIR }}
${{ steps.go-cache.outputs.dir }}
restore-keys: |
${{ env.BASE_CACHE_KEY }}
- name: Setup curl cache
run: mkdir -p ${{ env.CURL_CACHE_DIR }}
- name: Install Packer
env:
PACKER_VERSION: ${{ steps.setup-env.outputs.packer-version }}
run: |
PACKER_ZIP="packer_${PACKER_VERSION}_linux_amd64.zip"
curl --output ${{ env.CURL_CACHE_DIR }}/"${PACKER_ZIP}" \
--time-cond ${{ env.CURL_CACHE_DIR }}/"${PACKER_ZIP}" \
--location \
"https://releases.hashicorp.com/packer/${PACKER_VERSION}/${PACKER_ZIP}"
sudo unzip -d /opt/packer \
${{ env.CURL_CACHE_DIR }}/"${PACKER_ZIP}"
sudo mv /usr/local/bin/packer /usr/local/bin/packer-default
sudo ln -s /opt/packer/packer /usr/local/bin/packer
- uses: hashicorp/setup-packer@v3
with:
version: ${{ steps.setup-env.outputs.packer-version }}
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ steps.setup-env.outputs.terraform-version }}
Expand Down Expand Up @@ -186,7 +172,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ steps.setup-env.outputs.python-version }}
- uses: actions/cache@v3
- uses: actions/cache@v4
env:
BASE_CACHE_KEY: "${{ github.job }}-${{ runner.os }}-\
py${{ steps.setup-python.outputs.python-version }}-"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sync-labels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
egress-policy: audit
- id: github-status
name: Check GitHub status
uses: crazy-max/ghaction-github-status@v3
uses: crazy-max/ghaction-github-status@v4
- id: dump-context
name: Dump context
uses: crazy-max/ghaction-dump-context@v2
Expand Down
81 changes: 54 additions & 27 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,13 @@ default_language_version:
python: python3

repos:
# Check the pre-commit configuration
- repo: meta
hooks:
- id: check-useless-excludes

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v4.6.0
hooks:
- id: check-case-conflict
- id: check-executables-have-shebangs
Expand All @@ -19,7 +24,6 @@ repos:
- --allow-missing-credentials
- id: detect-private-key
- id: end-of-file-fixer
exclude: files/(issue|motd)
- id: mixed-line-ending
args:
- --fix=lf
Expand All @@ -31,22 +35,15 @@ repos:

# Text file hooks
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.39.0
rev: v0.42.0
hooks:
- id: markdownlint
args:
- --config=.mdl_config.yaml
- repo: https://github.com/pre-commit/mirrors-prettier
# This is the last version of v3 available from the mirror. We should hold
# here until v4, which is currently in alpha, is more stable.
rev: v3.1.0
- repo: https://github.com/rbubley/mirrors-prettier
rev: v3.3.3
hooks:
- id: prettier
# This is the latest version of v3 available from NPM. The pre-commit
# mirror does not pull tags for old major versions once a new major
# version tag is published.
additional_dependencies:
- [email protected]
- repo: https://github.com/adrienverge/yamllint
rev: v1.35.1
hooks:
Expand All @@ -56,14 +53,14 @@ repos:

# GitHub Actions hooks
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.28.0
rev: 0.29.2
hooks:
- id: check-github-actions
- id: check-github-workflows

# pre-commit hooks
- repo: https://github.com/pre-commit/pre-commit
rev: v3.6.2
rev: v3.8.0
hooks:
- id: validate_manifest

Expand Down Expand Up @@ -98,7 +95,7 @@ repos:

# Shell script hooks
- repo: https://github.com/scop/pre-commit-shfmt
rev: v3.7.0-4
rev: v3.9.0-1
hooks:
- id: shfmt
args:
Expand All @@ -116,14 +113,14 @@ repos:
# Redirect operators are followed by a space
- --space-redirects
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.9.0.6
rev: v0.10.0.1
hooks:
- id: shellcheck

# Python hooks
# Run bandit on the tests directory with a custom configuration
- repo: https://github.com/PyCQA/bandit
rev: 1.7.7
rev: 1.7.10
hooks:
- id: bandit
name: bandit (tests directory)
Expand All @@ -132,44 +129,74 @@ repos:
- --config=.bandit.yml
# Run bandit on everything but the tests directory
- repo: https://github.com/PyCQA/bandit
rev: 1.7.4
rev: 1.7.10
hooks:
- id: bandit
name: bandit (everything but the tests directory)
exclude: tests
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.2.0
rev: 24.8.0
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0
rev: 7.1.1
hooks:
- id: flake8
additional_dependencies:
- flake8-docstrings
- flake8-docstrings==1.7.0
- repo: https://github.com/PyCQA/isort
rev: 5.13.2
hooks:
- id: isort
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
rev: v1.11.2
hooks:
- id: mypy
- repo: https://github.com/pypa/pip-audit
rev: v2.7.3
hooks:
- id: pip-audit
args:
# Add any pip requirements files to scan
- --requirement
- requirements-dev.txt
- --requirement
- requirements-test.txt
- --requirement
- requirements.txt
- repo: https://github.com/asottile/pyupgrade
rev: v3.15.1
rev: v3.17.0
hooks:
- id: pyupgrade

# Ansible hooks
- repo: https://github.com/ansible/ansible-lint
rev: v24.2.0
rev: v24.9.2
hooks:
- id: ansible-lint
# files: molecule/default/playbook.yml
additional_dependencies:
# On its own ansible-lint does not pull in ansible, only
# ansible-core. Therefore, if an Ansible module lives in
# ansible instead of ansible-core, the linter will complain
# that the module is unknown. In these cases it is
# necessary to add the ansible package itself as an
# additional dependency, with the same pinning as is done in
# requirements-test.txt of cisagov/skeleton-ansible-role.
# - ansible>=9,<10
# ansible-core 2.16.3 through 2.16.6 suffer from the bug
# discussed in ansible/ansible#82702, which breaks any
# symlinked files in vars, tasks, etc. for any Ansible role
# installed via ansible-galaxy. Hence we never want to
# install those versions.
#
# Note that any changes made to this dependency must also be
# made in requirements.txt in cisagov/skeleton-packer and
# requirements-test.txt in cisagov/skeleton-ansible-role.
- ansible-core>=2.16.7

# Terraform hooks
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.88.0
rev: v1.96.1
hooks:
- id: terraform_fmt
- id: terraform_validate
Expand All @@ -182,7 +209,7 @@ repos:

# Packer hooks
- repo: https://github.com/cisagov/pre-commit-packer
rev: v0.0.2
rev: v0.1.0
hooks:
- id: packer_validate
- id: packer_fmt
70 changes: 50 additions & 20 deletions setup-env
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,52 @@ python_versions() {
pyenv versions --bare --skip-aliases --skip-envs
}

check_python_version() {
local version=$1

# This is a valid regex for semantically correct Python version strings.
# For more information see here:
# https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
# Break down the regex into readable parts major.minor.patch
local major="0|[1-9]\d*"
local minor="0|[1-9]\d*"
local patch="0|[1-9]\d*"

# Splitting the prerelease part for readability
# Start of the prerelease
local prerelease="(?:-"
# Numeric or alphanumeric identifiers
local prerelease+="(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)"
# Additional dot-separated identifiers
local prerelease+="(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*"
# End of the prerelease, making it optional
local prerelease+=")?"
# Optional build metadata
local build="(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"

# Final regex composed of parts
local regex="^($major)\.($minor)\.($patch)$prerelease$build$"

# This checks if the Python version does not match the regex pattern specified in $regex,
# using Perl for regex matching. If the pattern is not found, then prompt the user with
# the invalid version message.
if ! echo "$version" | perl -ne "exit(!/$regex/)"; then
echo "Invalid version of Python: Python follows semantic versioning," \
"so any version string that is not a valid semantic version is an" \
"invalid version of Python."
exit 1
# Else if the Python version isn't installed then notify the user.
# grep -E is used for searching through text lines that match the specific verison.
elif ! python_versions | grep -E "^${version}$" > /dev/null; then
echo "Error: Python version $version is not installed."
echo "Installed Python versions are:"
python_versions
exit 1
else
echo "Using Python version $version"
fi
}

# Flag to force deletion and creation of virtual environment
FORCE=0

Expand Down Expand Up @@ -144,17 +190,8 @@ while true; do
-p | --python-version)
PYTHON_VERSION="$2"
shift 2
# Check the Python versions being passed in.
if [ -n "${PYTHON_VERSION+x}" ]; then
if python_versions | grep -E "^${PYTHON_VERSION}$" > /dev/null; then
echo Using Python version "$PYTHON_VERSION"
else
echo Error: Python version "$PYTHON_VERSION" is not installed.
echo Installed Python versions are:
python_versions
exit 1
fi
fi
# Check the Python version being passed in.
check_python_version "$PYTHON_VERSION"
;;
-v | --venv-name)
VENV_NAME="$2"
Expand Down Expand Up @@ -188,15 +225,8 @@ if [ $LIST_VERSIONS -ne 0 ]; then
# Read the user's desired Python version.
# -r: treat backslashes as literal, -p: display prompt before input.
read -r -p "Enter the desired Python version: " PYTHON_VERSION
# Check the Python versions being passed in.
if [ -n "${PYTHON_VERSION+x}" ]; then
if python_versions | grep -E "^${PYTHON_VERSION}$" > /dev/null; then
echo Using Python version "$PYTHON_VERSION"
else
echo Error: Python version "$PYTHON_VERSION" is not installed.
exit 1
fi
fi
# Check the Python version being passed in.
check_python_version "$PYTHON_VERSION"
fi

# Remove any lingering local configuration.
Expand Down