Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Work in progress] Command line tool refactor to run tests #241

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

thegridman
Copy link
Contributor

@thegridman thegridman commented Sep 12, 2024

Introduction

This PR is work in progress.

This PR is for refactoring the command line too (Launcher) to add the xtc command so that we can run tests using xtc test and add flexibility to be able to easily add other sub-commands to xtc.

The code in the PR introduces a new "command" framework to add a hierarchical command structure, where commands can have sub-commands, which could have more sub-commands and so on. For example, xtc is a command that has a sub-command test so on the command line it is executed using xtc test <flags...>

The PR also introduces code to represent command line flags. The FlagSet represents a set of flags parsed from the command line. Each command has a set of valid flags. Each Flag in a FlagSet is types, for example a boolean flag, string flag, etc. Flags and parsing a FlagSet is posix compliant. Each flag must have a long name (which is used prefixed with -- on the command line. Optionally a Flag can have a single character shortcut (used prefixed with a - on the command line). Single characters can be concatenated on the command line (e.g. -abc is the same as -a -b -c)

Running Ecstasy Tests

The PR also add a simple test executor to make the xtc test command work. No-arg methods annotated with @Test will be executed in a module that imports xunit.

The FizzBuzz module under manualTests has had tests added to it.
If you build the local distribution, for example:

gradle xdk:installWithLauncherDist

Then export XDK_HOME to point to the distribution (e.g. on my Arm Mac)

export XDK_HOME=/Users/jonathanknight/dev/ecsatsy/xvm/xdk/build/install/xdk-macos_aarch64

Then build FizzBuzz and test it

cd manualTests
xtc build FizzBuzz
xtc test FizzBuzz

which should print something like...

Starting test execution for module TestFizzBuzz
  Executing tests in TestFizzBuzz:
    Passed: void shouldBeFizz() in TestFizzBuzz:
    Passed: void shouldBeBuzz() in TestFizzBuzz:
    Passed: void shouldBeFizzBuzz() in TestFizzBuzz:
    Passed: void shouldBeNumber() in TestFizzBuzz:
  Executed tests in TestFizzBuzz: Passed=4 Failed=0
Completed test execution for module TestFizzBuzz
Passed: 4
Failed: 0

Individual tests can be executed using the --test (or -t) flag for xtc test. The --test flag can be used multiple times.
The format of the value for the --test flag is <class-name>#<method-name>where <class-name> and <method-name> can be exact name or a sort of reg-ex pattern. Any dot '.' character in the pattern will match a dot in the test name, so not the same ad dot in a normal regex. A star * in the pattern is the same as (.*) in a normal reg-ex.

Either the class name or method name may be omitted, so <class-name> or #<method-name> are also valid.

If the value passed to the --test flag starts with a ! then matches to the filter will be excluded from the run. If a test matches both an include and and exclude filter, the include takes precedence.

For example:
To execute all tests in the class TestFizzBuzz

xtc test --test 'TestFizzBuzz'

To execute the test shouldBeBuzz in class TestFizzBuzz

xtc test --test 'TestFizzBuzz#shouldBeBuzz'

To execute all tests methods containing Fizz

xtc test --test 'TestFizzBuzz#*Fizz*'

To execute the test shouldBeBuzz in any class

xtc test --test '#shouldBeBuzz'

To NOT execute the test shouldBeBuzz in class TestFizzBuzz prefix the pattern with the "!" character

xtc test --test '!TestFizzBuzz#shouldBeBuzz'

@thegridman thegridman marked this pull request as draft September 12, 2024 17:44
@thegridman thegridman changed the title Command line tool refactor to run tests [Work in progress] Command line tool refactor to run tests Sep 13, 2024
@ggleyzer
Copy link
Collaborator

I tried a command (apparently forgot to add "run")

xtc -L build/xtc/main/lib -o build/xtc/main/lib src/main/x/FizzBuzz.x

and it blew up with an assertion

@thegridman thegridman force-pushed the jk/xtc-test-command branch 2 times, most recently from b406285 to f97f909 Compare September 25, 2024 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants