Simulator Test Cases

[cgmc]

This is intended as a master issue, allowing us to organise and track the various test cases that we may devise for the simulator. As such, each test case should be its own (sub-)issue, tracked as an item in the task list below.

[cgmc]

Editing the Task List

• Test cases should be added to the task list in the following format: - [ ] [<NAME_OF_TEST_CASE>] (https://github.com/CPSSD/robots#<ISSUE_NUMBER_OF_TEST_CASE>)
• Test cases that are deemed to have been successfully passed by the simulator should be ticked off, and vice versa.
• Test cases that are deemed to be infeasible, implausible or otherwise invalid should be struck through.
• Never delete a test case from the task list.

Since edits to an issue's description do not result in a notification, please write a comment for every edit you make to the task list, detailing the change(s) you made.

Test Case Template

A test case should be defined in the following format:

• An integer >= 3 denoting the number of walls of the room
• A numbered list of points which conforms to the following constraints:
  • Points are to be given as two integers, in the form (x, y)
  • There must be exactly as many points in the list as there are walls of the room
  • They must be in either clockwise or anti-clockwise order
  • They must be in order (since the code takes the end-point of one line to be the start-point of the next line)
• A point denoting the starting position of the simulated robot
• An integer denoting the number of objects in the room
• A numbered list of objects in the following format:
  1. (First level either blank or containing only "Object No. X")
     • (Each object must have its own second-level list)
     • An integer denoting the number of sides of the object
     • A numbered list of points, following the constraints of the equivalent list for the walls of the room
• (Can be added later, if needs be) A task list outlining the test case's acceptance criteria, i.e. the commands the simulated robot must execute successfully in order to pass this test case. Editing the task list should be done according to the rules for editing the master issue's task list.

Marking Test Cases as Passed/Failed/Invalid

I feel we should take from the code review concept for this, i.e. the same person should not be determining the acceptance criteria, testing the simulator on those criteria and then deciding if the simulator has passed or failed, or if the test case was in some way invalid in the first place.
I propose the following:

• Anyone can create a test case
• Anyone can propose acceptance criteria, but there must be some form of agreement that the proposed criteria are appropriate before accepting them
• At least two of myself, Ryan and Darragh have to agree at what point to stop adding acceptance criteria (unless the test case is re-opened later on - let's leave ourselves that option)
• "Anyone" can test the simulator against the accepted criteria (in practice, it will likely be between myself and Ryan)
• At least two of myself, Ryan and Darragh have to agree that the simulator has passed a particular criterion, otherwise it has failed
• The test case's issue can't be closed until all the acceptance criteria have been passed, but once they have, anyone can close it

I am open to suggestions on this point.