Return FAIL if tests are not found #1130
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stevedlawrence
approved these changes
Dec 11, 2023
| @@ -1729,7 +1732,7 @@ class Main( | |||
| ), | |||
| ) | |||
|
|
|||
| if (fail == 0) ExitCode.Success else ExitCode.TestError | |||
| if (pass > 0 && fail == 0 && notfound == 0) ExitCode.Success else ExitCode.TestError | |||
There was a problem hiding this comment.
Should we remove the pass > 0 condition? I'm thinking of an an edge case where a tdml file has no tests defined and a user runs daffodil test no_tests.tdml. If we require pass > 0 to be considered a success, then that command would exit with failure even though nothing technically failed. Maybe there's an argument that running no tests should be a failure? But using SBT as an example, running a test suite with no tests is considered a success. Maybe there are other counter examples?
There was a problem hiding this comment.
OK, agreed that pass > 0 can be removed.
Include specified test names in test list even if not found in test file in order to print names of tests not found. Return FAIL if any tests fail or are not found. DAFFODIL-2699 Main.scala: Include all test names for listing even if not found, update return code logic to check for any tests failing or not found. TestCLItdml.scala: Update expected exit codes based on new return code logic, note that test_CLI_Tdml_Listing_negativeTest01 now returns FAIL due to tests not found.
pkatlic
force-pushed
the
daffodil-2699-test-not-found
branch
from
db9257c to
35b3b4c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Include specified test names in test list even if not found in test file in order to print names of tests not found. Return FAIL if no tests pass or any tests fail or are not found.
DAFFODIL-2699
Main.scala: Include all test names for listing even if not found, update return code logic to check for no passing tests and any tests failing or not found.
TestCLItdml.scala: Update expected exit codes based on new return code logic, note that test_CLI_Tdml_Listing_negativeTest01 now returns FAIL due to tests not found and test_CLI_catch_TestNotCompatible now returns FAIL due to no passing tests.