Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up end-to-end testing of Send/Receive #339

Closed
rmunn opened this issue Jun 25, 2024 · 2 comments · Fixed by #342
Closed

Set up end-to-end testing of Send/Receive #339

rmunn opened this issue Jun 25, 2024 · 2 comments · Fixed by #342
Assignees

Comments

@rmunn
Copy link
Collaborator

rmunn commented Jun 25, 2024

Now that LexBox and Language Forge can be run side by side, it should be possible to set up end-to-end testing of Send/Receive scenarios with a real Language Depot deployment, rather than a simulated one. A typical test might go as follows:

  1. Create a LexBox project, and upload a .zip file containing the initial state of the project.
  2. Do an initial clone into Language Forge, verify correct number of entries and so on.
  3. Use the MkFwData tool to get a .fwdata file for that project.
  4. Use liblcm to load that .fwdata file, then add a new entry or edit an existing entry.
  5. Use the SplitFwData tool (that doesn't exist yet) to split the edited .fwdata file up into its component parts.
  6. Create a new Mercurial commit with those changes.
  7. Push that commit to the LexBox repo. (Steps 3-7 simulate making an edit in FieldWorks followed by a Send/Receive).
  8. Use Mongo to make edits in the Language Forge project (faster), or use Playwright to drive the UI to make those changes (slower).
  9. Have Language Forge trigger a Send/Receive, kicking off LfMerge.
  10. Once LfMerge has finished the Send/Receive, verify that LF's Mongo database contains the changes from steps 2-7 (or the merge result of those changes).
  11. In LexBox, verify that LfMerge created a new commit with the correct results.
  12. Download an .fwdata file from LexBox and load it into liblcm.
  13. Use liblcm to verify that the FieldWorks objects received the correct changes from LfMerge (updated field contents, comments, or whatever).

That's a lot of steps that currently have to be done by hand, but with LexBox and Language Forge running side by side on a developer machine (plus MkFwData and SplitFwData tools becoming available in the flexbridge repo so that the E2E S/R tests can use them), all of that will finally be able to be automated.

@rmunn
Copy link
Collaborator Author

rmunn commented Jul 25, 2024

More ideas: a helper class to manage various liblcm-related testing tasks:

  • Given project code, clone it, create .fwdata file, open liblcm project
  • Subset: given already-cloned Mercurial repo, create .fwdata file and open liblcm project
  • Given liblcm project, close project, hg commit, push (maybe through LfMergeBridge for simplicity)

Also do various liblcm tasks:

  • Create lexentry with given lexeme form
  • Add sense, change part of speech
  • Add example sentence to given sense
  • Etc.

Basically, anything that's a multi-step process of more than 2-3 steps in existing S/R tests should turn into a helper method.

@rmunn
Copy link
Collaborator Author

rmunn commented Aug 1, 2024

A thought about repeatability: I could use the NUnit attribute Property on tests to mark them with the project that they use. Then a test fixture could use that value (which can be accessed through the TestContext) to get the current "tip" revision of the project before starting the test, and in the teardown, reset the project to that tip revision so that we remove any commits we pushed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant