HA conflicts with git storage

[rgarrigue]

Describe the bug

I've 2 EC2 instances running WIkiJS, installed through Ansible with the same settings, postgres database over RDS. Behind HAProxies with Consul based service discovery. Git storage is enabled for some integration. ha: true and wikijs's 2.3.77.

There are a bunch of error message when a change is made, see the screenshots below. Some of the error message appear in systemd's journal. And I can see a

May 05 13:47:39 i-0ddc6db676d296213 node[3370]: Loading configuration from /srv/wikijs/config.yml... OK
May 05 13:47:39 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:47:39.828Z [JOB] info: Rebuilding page tree...
May 05 13:47:40 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:47:40.227Z [JOB] info: Using database driver pg for postgres [ OK ]
May 05 13:47:40 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:47:40.421Z [JOB] info: Rebuilding page tree: [ COMPLETED ]
May 05 13:47:40 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:47:40.441Z [MASTER] info: (STORAGE/GIT) Committing new file [en] new-page...
May 05 13:47:40 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:47:40.458Z [MASTER] warn: fatal: cannot do a partial commit during a merge.
May 05 13:48:05 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:48:05.741Z [MASTER] info: Received event deletePageFromCache from instance t_T5hQ1m-z: [ OK ]
May 05 13:48:35 i-0ddc6db676d296213 node[3370]: Loading configuration from /srv/wikijs/config.yml... OK
May 05 13:48:35 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:48:35.343Z [JOB] info: Rendering page ID 33...
May 05 13:48:35 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:48:35.700Z [JOB] info: Using database driver pg for postgres [ OK ]
May 05 13:48:36 i-0ddc6db676d296213 node[3370]: 2020-05-05T13:48:36.502Z [JOB] info: Rendering page ID 33: [ COMPLETED ]

My guess is, one instance is committing, pushing, the other get the change through the HA mechanism, try to commit & push too, & boom. Or something alike.

To Reproduce
Steps to reproduce the behavior:

1. Set up 2 wikijs, with ha: true and a git storage
2. Create a new page, save it
3. Delete a page

Expected behavior
Users should be unaware of HA, no such message being displayed

Screenshots

Host Info (please complete the following information):

• OS: Debian 10
• Wiki.js version: 2.3.77
• Database engine: postgres

[pms1969]

I'm seeing this exact same issue. We're scaling back to be non-HA for the moment, but would love to know the current state of understanding around this.

[jhwbarlow]

We are also seeing git conflicts with HA mode enabled. We run two replicas of wikijs as Kubernetes pods, round-robin load-balanced by a regular Kubernetes service.

With multiple replicas, there are often merge conflicts between the two. My best guess is that either two people are editing the same page simultaneously (how is this usually handled in the non-HA case, I'm not sure?) using different replicas; or the same person makes a change to a page, saves it, edits the page again (but this is served by a different replica) and saves it - all within the five minute git push window. Therefore the replica which pushes first causes a merge conflict on the other replica:

/wiki/data/repo $ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 2 and 2 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

You have unmerged paths.
  (fix conflicts and run "git commit")
  (use "git merge --abort" to abort the merge)

Changes to be committed:
	modified:   security/faq.md

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both added:      scratch/bullet-bug.md

2020-05-26T11:50:39.325Z [MASTER] info: Syncing with storage target git...
2020-05-26T11:50:39.339Z [MASTER] info: (STORAGE/GIT) Performing pull rebase from origin on branch master...
error: Pulling is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
2020-05-26T11:50:39.344Z [MASTER] error: Syncing with storage target git: [ FAILED ]
2020-05-26T11:50:39.344Z [MASTER] error: error: Pulling is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.

Thanks!