Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: refactor TestVectors into Positive and Negative Tests #628

Draft
wants to merge 5 commits into
base: mainline
Choose a base branch
from
Draft

test: refactor TestVectors into Positive and Negative Tests #628

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cb10b57
chore: update test vectors native so they are positive negative tests
josecorella Nov 29, 2023
22164a2
update gha
josecorella Nov 29, 2023
faf3db7
fix other gha
josecorella Nov 29, 2023
6427c39
Merge branch 'mainline' into jocorell/update-testvectors
josecorella Nov 29, 2023
743c495
Merge branch 'mainline' into jocorell/update-testvectors
josecorella Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 15 additions & 28 deletions .github/workflows/library_net_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ jobs:
run: |
PYTHON_23_VECTOR_PATH=$GITHUB_WORKSPACE/python23/vectors
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$PYTHON_23_VECTOR_PATH/manifest.json" \
dotnet test --framework net48
dotnet test --framework net48 --filter PositiveDecryptTestVector

- name: Run Decrypt Test Vectors on .NET net6.0
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors
Expand All @@ -142,10 +142,10 @@ jobs:
if [ "$RUNNER_OS" == "macOS" ]; then
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$PYTHON_23_VECTOR_PATH/manifest.json" \
DYLD_LIBRARY_PATH="/usr/local/opt/[email protected]/lib" \
dotnet test --framework net6.0
dotnet test --framework net6.0 --filter PositiveDecryptTestVector
else
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$PYTHON_23_VECTOR_PATH/manifest.json" \
dotnet test --framework net6.0
dotnet test --framework net6.0 --filter PositiveDecryptTestVector
fi

- name: Generate Test Vectors with .NET Framework net6.0
Expand Down Expand Up @@ -196,18 +196,17 @@ jobs:
DOWNLOAD_NAME=valid-Net-4.0.0.zip
unzip -o -qq $DOWNLOAD_NAME -d $NET_400_VALID_VECTORS

- name: Run ESDK-NET @ v4.0.0 Valid Vectors expect success
- name: Run ESDK-NET @ v4.0.0 Valid Vectors Positive Decrypt Test
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors
continue-on-error: true
shell: bash
run: |
NET_400_VALID_VECTORS=$GITHUB_WORKSPACE/v4Net400Valid/vectors
ESDK_NET_V400_POLICY="forbid" \
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_VALID_VECTORS/manifest.json" \
dotnet test --framework net48
dotnet test --framework net48 --filter PositiveDecryptTestVector
ESDK_NET_V400_POLICY="forbid" \
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_VALID_VECTORS/manifest.json" \
dotnet test --framework net6.0 --logger "console;verbosity=quiet"
dotnet test --framework net6.0 --logger "console;verbosity=quiet" --filter PositiveDecryptTestVector

- name: Unzip ESDK-NET @ v4.0.0 Invalid Vectors
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors/resources
Expand All @@ -218,55 +217,43 @@ jobs:
DOWNLOAD_NAME=invalid-Net-4.0.0.zip
unzip -o -qq $DOWNLOAD_NAME -d $NET_400_INVALID_VECTORS

- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET 48 expect failure
- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET 48 Negative Decrypt Test
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors
continue-on-error: true
shell: bash
run: |
NET_400_INVALID_VECTORS=$GITHUB_WORKSPACE/v4Net400Invalid/vectors
ESDK_NET_V400_POLICY="forbid" \
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
dotnet test --framework net48
# Dotnet test returns 1 for failure.
TEMP=$?; if [[ "$TEMP" -eq 1 ]]; then true; else false; fi;
# We want this to fail, so if it returned 1, step passes, else it fails
# TODO Post-#619: Refactor Test Vectors to expect failure,
# as I doubt this true false logic works
dotnet test --framework net48 --filter NegativeDecryptTestVector

- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET 6.0 expect failure
- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET 6.0 Negative Decrypt Test
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors
continue-on-error: true
shell: bash
run: |
NET_400_INVALID_VECTORS=$GITHUB_WORKSPACE/v4Net400Invalid/vectors
if [ "$RUNNER_OS" == "macOS" ]; then
ESDK_NET_V400_POLICY="forbid" \
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
DYLD_LIBRARY_PATH="/usr/local/opt/[email protected]/lib" \
dotnet test --framework net6.0
dotnet test --framework net6.0 --filter NegativeDecryptTestVector
else
ESDK_NET_V400_POLICY="forbid" \
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
dotnet test --framework net6.0
dotnet test --framework net6.0 --filter NegativeDecryptTestVector
fi
# Dotnet test returns 1 for failure.
TEMP=$?; if [[ "$TEMP" -eq 1 ]]; then true; else false; fi;
# We want this to fail, so if it returned 1, step passes, else it fails
# TODO Post-#619: Refactor Test Vectors to expect failure,
# as I doubt this true false logic works

- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET expect Success
- name: Run ESDK-NET @ v4.0.0 Invalid Vectors .NET Positive Decrypt Test
working-directory: ./AwsEncryptionSDK/runtimes/net/TestVectorsNative/TestVectors
shell: bash
run: |
NET_400_INVALID_VECTORS=$GITHUB_WORKSPACE/v4Net400Invalid/vectors
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
dotnet test --framework net48 --logger "console;verbosity=quiet"
dotnet test --framework net48 --logger "console;verbosity=quiet" --filter PositiveDecryptTestVector
if [ "$RUNNER_OS" == "macOS" ]; then
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
DYLD_LIBRARY_PATH="/usr/local/opt/[email protected]/lib" \
dotnet test --framework net6.0 --logger "console;verbosity=quiet"
dotnet test --framework net6.0 --logger "console;verbosity=quiet" --filter PositiveDecryptTestVector
else
DAFNY_AWS_ESDK_TEST_VECTOR_MANIFEST_PATH="$NET_400_INVALID_VECTORS/manifest.json" \
dotnet test --framework net6.0 --logger "console;verbosity=quiet"
dotnet test --framework net6.0 --logger "console;verbosity=quiet" --filter PositiveDecryptTestVector
fi
Loading
Loading