Add proof-of-concept tool to make .fwdata files

[rmunn]

WIP. Will fix #631 when complete.

Tool will be used to create .fwdata files on demand, allowing us to implement several different features we might want on the backend.

Currently this is a separate binary so as not to pull in a bunch of dependencies into the LexBoxApi project. But we could fold it into LexBoxApi if we want to.

How to test

Will require sillsdev/flexbridge#398 to be merged (and released) in order to work. To test this PR branch before that FLExBridge PR is merged, do the following:

1. Clone https://github.com/sillsdev/flexbridge/ in a sibling directory to where you have lexbox checked out.
2. Check out the feature/lfmergebridge-create-fwdata branch in the flexbridge repo.
   • You can do this in a single step by running git clone https://github.com/sillsdev/flexbridge/ -b feature/lfmergebridge-create-fwdata
3. In flexbridge, run dotnet build and make sure it's working
   • I had to add <GenerateResourceUsePreserializedResources>true</GenerateResourceUsePreserializedResources> to several .csproj files in the flexbridge repo in order to build, but I think that's Linux-only and you won't have that problem on a Windows machine. So I haven't (yet) included that change in Expose PutHumptyTogetherAgain method in LfMergeBridge flexbridge#398
4. Clone a Mercurial repo, let's say sena-3, somewhere
5. In the sena-3 repo, run hg update tip
6. cd backend/MkFwData
7. dotnet run -- /path/to/sena-3

If all goes well, you should see "Created /path/to/sena-3/sena-3.fwdata" and you should now have a sena-3.fwdata file.

Planned development

I plan to add an option to the MkFwData command to optionally check out the tip revision, or any specified revision, automatically, so that you don't have to do so as a separate step. Then after the .fwdata file is created, we would run something like hg checkout null which is Mercurial syntax for "Dispose of the working tree, please, I no longer want it". (This does not delete unversioned files, such as the .fwdata file just created by the tool). That way we won't leave lots of working trees taking up space in our data volume.

Potential uses

This could be used to easily download an .fwdata file. There's potential for using it for more advanced features, such as checking out two different revisions of a project, creating two .fwdata files, using liblcm to load them, and then comparing the two revisions using liblcm in order to present a much nicer UI to the user when they say "What changed in this project between (date) and now?" I.e. instead of Mercurial diffs, we could tell them "The definition of word X was changed from Y to Z" and so on.

This could also be used as the basis of a liblcm-based viewer. And, if the "split .fwdata file up" operation was implemented, potentially an editor as well.

[github-actions]

C# Unit Tests

30 tests  ±0   30 ✅ ±0   4s ⏱️ ±0s
 7 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 14f316a. ± Comparison against base commit 76ef56e.

♻️ This comment has been updated with latest results.

[rmunn]

As I was adding the ability to automatically check out the appropriate Mercurial commit into the proof-of-concept binary itself, I discovered that the Linux Mercurial4Chorus NuGet package is missing the zstd.cpython-310-x86_64-linux-gnu.so file. This means that repos created on our hgweb server (where the Mercurial version has a working zstd installation) can't be checkout out by the Mercurial package included in Chorus. I created sillsdev/Mercurial4Chorus#16 to fix this, but work on this PR might be stalled until I can get that fixed. (Which is complicated due to the need to compile the binary against multiple different versions of Python).

Note that this is not a problem on Windows; the Mercurial4Chorus package included the mercurial.zstd.pyd binary, so Windows users won't notice this issue. But I feel I should fix sillsdev/Mercurial4Chorus#16 before I do more work on this PR.

[rmunn]

I feel I should fix sillsdev/Mercurial4Chorus#16 before I do more work on this PR.

sillsdev/Mercurial4Chorus#18 fixes the Mercurial4Chorus package to include the missing zstd extension (along with several other extensions that were missing, but those being missing just meant slower runtime rather than errors so I hadn't originally noticed). Once that's merged, I'll be able to update the Mercurial4Chorus package version included in this tool and start making progress here again.

[rmunn]

Rebasing on top of develop now that #755 has been merged.

[rmunn]

The FlexBridge PR needed to run this has been merged into develop, but it looks like the FlexBridge build has been unable to build NuGet packages ever since it was updated to use SIL.Chorus.Mercurial version 6.5.1. In order to get this PR moving, I would need to be able to consume a NuGet package (currently it uses a ProjectReference and you have to check out the FlexBridge code into a sibling folder to the LexBox folder). Fixing the FlexBridge build does not seem like the highest priority for us right now, so I may put this PR on hold until the FlexBridge build gets fixed; then I can pick this up again and proceed with it.

[hahn-kev]

@rmunn per #631 (comment) it sounds like we can close this PR?