Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multiple UCMs/Clients to connect to the same codebase #5537

Open
ChrisPenner opened this issue Jan 14, 2025 · 1 comment
Open

Allow multiple UCMs/Clients to connect to the same codebase #5537

ChrisPenner opened this issue Jan 14, 2025 · 1 comment

Comments

@ChrisPenner
Copy link
Contributor

ChrisPenner commented Jan 14, 2025

Goal: Multiple clients can access the same codebase without errors, and without clobbering each-other's work.

The largest previous concern was that we kept the branch in-memory as part of the CLI loop-state and all operations happened there. This has changed with the new project-root structure to a model where UCM only keeps track of which project-branch it is on, then when a command is issued it will ask the codebase for the appropriate branch, perform the relevant updates and save them directly into the codebase.
The codebase contains a branch cache which allows this to still be fast, but without the same concerns about the in-memory branch going stale since it's now keyed by hash.

With that resolved, here are the remaining concerns:

  1. What happens in UCM if another UCM changes project branches
  • Currently the 'active project branch' is tracked in SQLite so that it's persistent across reloads, but that gets messy if you have multiple independent sessions. Should be easy enough to move that into the Loop State, then only check it when booting up a new instance.
  1. What happens if two sessions are in the same project-branch and one updates the root?
  • If one of the UCMs was idling, then everything is fine, it'll just pick up the new project root from SQLite as soon as it tries to do anything, it'll automatically load it into the codebase cache and continue on safely.
  • If BOTH UCMs are running mutating commands at the same time it's now possible they'd clobber each other in a strange way. AFAIK all UCM commands add new code/branches to the codebase atomically, then update the branch-head as a single atomic action so it should always appear as though one of the updates succeeded and the other disappears. Ideally we would notify the session which failed to apply, if we can detect it.
  1. What happens if one session deletes a project/branch the other session is on.
  • Presumably we can add logic to recover in the same way as though the session deleted the project itself.

So it seems number 2 causes the greatest challenge; some ideas for addressing it:

  1. Within the same transaction but before applying the final atomic update, double-check that the branch head is still on the hash it was when you applied the updates, then do a check-and-set; erroring if it has changed. Earlier transactions may have still been committed (e.g. adding definitions to the codebase) but that should be fine.
  2. Use the existing codebase-locking mechanism, but run it on a per-command basis, each command locks and unlocks the codebase preventing concurrent updates from happening, but also preventing some of the useful workflows people want this for; e.g. running a dev server. We could do a more complex reader/writer lock situation or something to avoid this for those workflows. Would be best if we only allowed one writer, but allowed any number of readers concurrently with that single writer.
  3. Double-check all our logic and ensure each command is properly atomic and does its own checks. This is probable to have the best UI/UX outcomes, but is also the most work.

One complication I saw when looking into this is that our Branch updates are typically Branch IO -> Branch IO rather than Branch Transaction -> Branch Transaction making it difficult to (safely) run updates atomically.

We should change our tooling to encourage doing the majority of the critical parts of a command in a single transaction.

@ChrisPenner
Copy link
Contributor Author

A prototype which switches branch locking to be command-level is available here: https://github.com/unisonweb/unison/tree/cp/granular-codebase-locking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant