Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "feat: generate addons readme in pre-commit" #220

Closed

Conversation

legalsylvain
Copy link
Contributor

This reverts commit a1551e5.

Rational: The call of oca-gen-addon-readme will generate conflicts in the readme.rst file when 2 PRs modify the same module. See #190 (comment)

@yajo
Copy link
Member

yajo commented Oct 30, 2023

As you can see in #190 (comment), that PR was solving a different issue, which is also important. We can revert it until we get to a better solution, but still we'll have the original problem.

I can imagine a possibly better approach: to add a .gitattributes file with these contents (or similar; I didn't test this):

*/README.rst merge=ours

According to the docs, that would ignore any merge conflicts in those files. Then, the OCA bot can re-generate the README after merging, which is something it would do anyway, since it would bump the manifest version and thus violate the hash written in the README.

@legalsylvain
Copy link
Contributor Author

As you can see in #190 (comment), that PR was solving a different issue, which is also important. We can revert it until we get to a better solution, but still we'll have the original problem.

Well. i'm not see the two issues at the same level :

  • generate conflicts between independant PRs : regression
  • generate automatically the last up to date README.rst file that could be read by functional people on runboat : nice to have feature

I don't have a clear opinion about alternatives, (and consequences).

@yajo
Copy link
Member

yajo commented Oct 30, 2023

However, both issues aren't as easy to fix:

  • generate conflicts between independant PRs : rebase and push force. Easy.

  • generate automatically the last up to date README.rst file that could be read by functional people on runboat : Teach functional people how to browse and interpret the file tree in a github PR, how to read Markdown and how to interpret what sections of the readme are outdated and which one are updated. Extremely hard.

Copy link
Contributor

@simahawk simahawk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well. i'm not see the two issues at the same level :

* generate conflicts between independant PRs : **regression**

* generate automatically the last up to date README.rst file that could be read by functional people on runboat : **nice to have feature**

I agree w/ @legalsylvain

I know we can disable it by repo but then... what's the purpose?

pedrobaeza

This comment was marked as duplicate.

Copy link
Member

@pedrobaeza pedrobaeza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already in the template for more than a month and I haven't still see any merge conflict, so I don't think this is a frequent problem. Wait till the first conflict arises, and then we discuss about it.

@legalsylvain
Copy link
Contributor Author

I know we can disable it by repo but then... [...]

Sorry, I missed that point. @simahawk How it is possible to disable it by repo ? I did'nt saw the option in the PR #190. could you help me ?

generate conflicts between independant PRs : rebase and push force. Easy.

I don't get it. Let's take an exemple.

  • I want to use this unmerged PR https://github.com/OCA/mis-builder/pull/575/files by @etobella on my V14 instance.
    (let's consider that this PR contains a hash change, which would have been the case if etobella had launched the pre-commit.).
  • I so add the reference to the PR in my repos.yml and all is OK.
  • Suddendly, A whatever bugfix is coming against mis_builder V14 and fast merged by the maintainer @sbidoul.
  • The PR now conflicts.

As I don't have write access on etobella repo, I can't fix the branch, and so I can't use it anymore, until etobella rebase and push.
I have to fork the branch, rebase, fix, push force on my fork and now maintain a fork. Not so "easy" IMO.

Wait till the first conflict arises, and then we discuss about it.

In general, I like this philosophy of "dealing with problems when they become real".
In our case, however, conflicts are likely to become more and more numerous over time, so I don't know if it's a good strategy.

@pedrobaeza
Copy link
Member

But as said, if this change is already some time on OCA and haven't arisen yet, it's because yours pre-thoughts are not really like that. I would like to have a real case to see the impact.

@legalsylvain
Copy link
Contributor Author

But as said, if this change is already some time on OCA and haven't arisen yet, it's because yours pre-thoughts are not really like that. I would like to have a real case to see the impact.

But I don't see this patches deployed since a month on OCA repos, but since 3 days.

This patch target V14 / V15 / V16. So it should be deployed on 200 repo x 3 versions = 600 branches.

  • ~150 repos are not patched, or have been patched 2 days ago, by manual PRs.
  • The other has been patched 3 days agos by bot.

image

Did I missed something ?

@pedrobaeza
Copy link
Member

OK, the merge was done a lot of time ago, but not the deployment. Anyway, the deployment is done, and if you revert this, another massive update should be done, so let's wait for the first conflict to arise to see the consequences.

@simahawk
Copy link
Contributor

simahawk commented Oct 30, 2023

Sorry, I missed that point. @simahawk How it is possible to disable it by repo ? I did'nt saw the option in the PR #190. could you help me ?

https://github.com/OCA/edi-framework/blob/16.0/.github/workflows/pre-commit.yml#L30

OK, the merge was done a lot of time ago, but not the deployment. Anyway, the deployment is done, and if you revert this, another massive update should be done, so let's wait for the first conflict to arise to see the consequences.

But to save what? To gain what? TBH I don't understand why this is an interesting feature.
IF you want to read the RELEVANT part of the docs, you don't read the generated HTML.
Personally I never do that because is cluttered w/ all sorts of non relevant info.
If I look at the readme folder I can precisely see what is relevant for the behavior of the module.
Plus, it adds noise to the PRs.

Getting to "experience": personally I only had issues so far in the repo where is has been enabled due to missing update of whatever library or version... can't remember now. I ended up committing w/ --no-verify.
If the majority of the repos are not using this yet and many of them simply turned it off I don't see any reason to carry on this thing if we still have to update all repos (especially due to the issue w/ py version on gh actions).

My $0.02 😁

@yajo
Copy link
Member

yajo commented Oct 31, 2023

Some points to consider.

If I look at the readme folder[...]

That's one of the unsatisfactory solutions mentioned in #190 (comment).

We are used to OCA, Odoo, Markdown and Github... we are even used to how the bot auto-generates readmes... however, amount of corner cases that we deal with in our minds when we do that is a huge entry barrier for functional contributors.

Functional contributors do this:

  1. Open runboat.
  2. Go to the module readme.
  3. See all of it (not only the things that changed in the PR, if any).
  4. Exercise the module to search for bugs.

We have to understand that an excellent QA reviewer can be a person that will never ever know how to write a line of Markdown or RST. That's the situation I was trying to fix. If our tooling scares away excellent QA contributors, the problem is on our tooling.

#190 made this scenario possible. It's just part of an effort that affected many parts of OCA, aiming to improve the contribution experience for non-techies. Some links to consider:

OTOH, I understand the problem you guys mention. It's not like I don't think it should be fixed, but certainly I wouldn't like to harm our functional team workflow yet again, after all this work is done.

Keep in mind that we already had that problem when, for example, you were merging 2 PRs that changed 2 different addons that altered external_dependencies:python. The requirements.txt file would conflict always. Of course, it's not as common as the case you say now. But I mean that you already had to have a solution for that, you just have to use it more.

There's a solution already. Long time ago, I changed my git-aggregator configs like this:

 l10n-spain:
   defaults:
     depth: $DEPTH_MERGE
   remotes:
     origin: https://github.com/moduon/l10n-spain.git
     upstream: https://github.com/OCA/l10n-spain.git
   target: origin $ODOO_VERSION
   merges:
     - origin $ODOO_VERSION
-    - upstream refs/pull/2979/head     
+  shell_command_after:
+    - curl -sSL https://github.com/OCA/l10n-spain/pull/2979.patch | git am -3 --keep-non-patch --exclude '*requirements.txt'

This resulted in less random errors and less conflicts when requirements.txt files were updated. It's easy to do the same for the README files.

Please hold on while I try the solution from #220 (comment). Maybe that works and the problem just disappears.

yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Oct 31, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Oct 31, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
@yajo
Copy link
Member

yajo commented Oct 31, 2023

Please see #222.

yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Oct 31, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Oct 31, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Nov 8, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Nov 8, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Nov 8, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
This reverts commit a1551e5.

Rational: The call of oca-gen-addon-readme will generate conflicts in the readme.rst file when 2 PRs modify the same module.
See OCA#190 (comment)
@legalsylvain legalsylvain force-pushed the master-revert-oca-gen-addon-readme branch from 397057b to 2ec4122 Compare November 8, 2023 16:16
@legalsylvain
Copy link
Contributor Author

Definitively, call generate-addons-readme during pre-commit generate a lot of problem for developers.
new one discovered today :

Scenario :

  1. make changes "A".

  2. commit "A". Fail, because pre-commit changed files. adding + !! source digest: sha256:171e8fe23de2635a7720e851e222f97cd021e97b5ba772b630484c22d42d031c

  3. commit "A'", with change in readme.rst

  4. make changes "B"

  5. commit "B". Fail, because pre-commit changed files. adding + !! source digest: sha256:b309e66df73fcf5de5629e2cc3645e23340d22335fcd405f771e5cce15341741

  6. commit "B'", with change in readme.rst

Then, if you to reorder your commit with a rebase to put B before A, here is the final content of README.rst file :

   !! source digest: sha256:080d86ef900f4f27baf1f53f92e407247e033c7b14a46667d53597711605af32
   !! source digest: sha256:b309e66df73fcf5de5629e2cc3645e23340d22335fcd405f771e5cce15341741
   !! source digest: sha256:171e8fe23de2635a7720e851e222f97cd021e97b5ba772b630484c22d42d031c

Problem 1 : all commits are poluted by systematic two changes in README.rst and index.html file now.
Problem 2 : when we commit "valid" changes, it forces developper to commit two times, because of the recompute of the hash.
Problem 3 : reordering commits generate weird things.

Please, revert accept that revert, and find another solution. The current design is not good.

CC : @sbidoul

yajo added a commit to moduon/oca-addons-repo-template that referenced this pull request Nov 9, 2023
As seen in OCA#190 (comment), when somebody tried to merge 2 PRs for the same module, chances are that both imply different readme digests and the merge fails.

As seen in OCA#220 (comment), this was already happening under some circumstances with `requirements.txt` files.

Here I added a `.gitattributes` file which specifies a `merge=union` driver for those files. [Git docs][1] explain that this driver will just keep the changes from all files, marking the merge as succeeded and not including merge markers. This is not ideal, but is OK for our current situation because:

1. Duplicate/conflicting lines in those files should not really harm a lot.
2. The bot will re-render the README upon merge anyways, fixing that conflict automatically.
3. We could define a custom merge driver, but `union` is built into Git, meaning less configurations required downstream.

Still, it doesn't seem to work in octopus merges. At least, if you do consecutive merges instead, all will work as expected.

At the same time, I moved the hook to be the last. This is because previous hooks can reformat the files used to obtain the digest.

Closes OCA#220.

[1]: https://git-scm.com/docs/gitattributes.html#Documentation/gitattributes.txt-union
@sbidoul sbidoul closed this in #229 Nov 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants