Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Action-based reporting to OPEN-CUTS #2532

Open
NeoTheThird opened this issue Mar 24, 2022 · 0 comments
Open

Action-based reporting to OPEN-CUTS #2532

NeoTheThird opened this issue Mar 24, 2022 · 0 comments
Labels
enhancement a new feature or an improvement of existing functionality

Comments

@NeoTheThird
Copy link
Member

We could create tests on OPEN-CUTS for every action to make it easier to see if the installer's features have actually been covered and where problems are frequent. The plugin index in the installer could keep track of every action that has been executed and report a run with the action's config as a comment as a run for those tests. If there is an error, only the error message could be included as a "log".

This should not replace the "main" test of installing a device, but instead augment it. The runs for every action could be reported together with the main run after the user selected a result.

Cases:

  • The installation completed and the user selected PASS: Report PASS runs for every action that has been run without error. If any errors in previous actions have been ignored, report FAIL for those actions.
  • The installation completed and the user selected WONKY or FAIL: Report WONKY runs for every action that has been run without error, as it is unclear where exactly the problem originated. If any errors in previous actions have been ignored, report FAIL for those actions.
  • The user reports an Error: Report WONKY runs for all previous actions that did not throw an error. Report a fail run for the action that did.

Note: As with all things crowdsourcing, the goal here is not to achieve the most accurate result possible, but something that allows the devs a big picture view of what's going on.

@NeoTheThird NeoTheThird added the enhancement a new feature or an improvement of existing functionality label Mar 24, 2022
@NeoTheThird NeoTheThird pinned this issue Mar 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement a new feature or an improvement of existing functionality
Projects
None yet
Development

No branches or pull requests

1 participant