Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Over-triggering with rules_go and Gazelle under certain circumstances #83

Open
mikberg opened this issue Jul 30, 2021 · 9 comments
Open

Comments

@mikberg
Copy link

mikberg commented Jul 30, 2021

This issue is basically a better documented version of #70

Example repository: https://github.com/mikberg/bazel-diff-problem

This repository sets up rules_go with Gazelle, a go binary with an external dependency and some rule which depends on the WORKSPACE file. I think all three are important.

The second commit adds a comment to the WORKSPACE file, demonstrating that any change to WORKSPACE will now result in bazel-diff regarding all external Go dependencies as impacted. This presumably also is the reason why the go_binary is also regarded as impacted.

./bazel-diff-example.sh <path-to>/bazel-diff-test /usr/local/bin/bazelisk d075588217ec39bac4064dc7feb7026cd899a1b0 d6e1e901dcd5dd9e9b32c2ce575938946839055f
Generating Hashes for Revision 'd075588217ec39bac4064dc7feb7026cd899a1b0'
Generating Hashes for Revision 'd6e1e901dcd5dd9e9b32c2ce575938946839055f'
Determining Impacted Targets
Impacted Targets between d075588217ec39bac4064dc7feb7026cd899a1b0 and d6e1e901dcd5dd9e9b32c2ce575938946839055f:
//:project_lib //external:com_github_kr_pretty //external:com_github_bazelbuild_buildtools //external:in_gopkg_check_v1 //external:com_github_pelletier_go_toml //external:org_golang_x_mod //external:bazel_gazelle_go_repository_config //external:com_github_fsnotify_fsnotify //external:com_github_rs_zerolog //:some-script //external:in_gopkg_yaml_v2 //external:com_github_bazelbuild_rules_go //external:com_github_kr_text //:project //external:com_github_davecgh_go_spew //external:com_github_bmatcuk_doublestar //external:org_golang_x_net //external:org_golang_x_text //:WORKSPACE //external:com_github_kr_pty //external:com_github_pmezard_go_difflib //external:org_golang_x_sync //external:com_github_burntsushi_toml //external:com_github_google_go_cmp //external:org_golang_x_crypto

I haven't been able to nail this down further.

As far as I can tell, this problem was introduced between 2.3.0 and 2.4.0.

Thank you for a fantastic tool!

@tinder-maxwellelliott
Copy link
Collaborator

Thank you for this great example repo. I will put together an integration test now in a draft PR

tinder-maxwellelliott added a commit that referenced this issue Aug 2, 2021
Adds a new github action test
to validate the issue reported in
#83
@tinder-maxwellelliott
Copy link
Collaborator

Alright my PR above captures the issue

tinder-maxwellelliott added a commit that referenced this issue Aug 23, 2021
Adds a new github action test
to validate the issue reported in
@tinder-maxwellelliott
Copy link
Collaborator

Okay so here is the result of doing the diff manually, somehow all the external targets have changed

diff /tmp/starting.json /tmp/final.json              
2,3c2,3
<   "//external:com_github_kr_pretty": "c4e4c3bc011fa6d15dfd99fa463bd11765e8c654518313e428aa08bd64d82b0a",
<   "//external:com_github_bazelbuild_buildtools": "b8b5d58a377e5e4318f83a0faf8681a9d3620c676008a661e908613fe24a6ecc",
---
>   "//external:com_github_kr_pretty": "f3d19f7fee981d26d6859ff30e10c715d37f78dc823d6d2783df63461347b181",
>   "//external:com_github_bazelbuild_buildtools": "e3f000b9c665df29a67ab9f47e8b8fa7e644b14fd244f9f3a596e6a32d1c8856",
6c6
<   "//external:com_github_pelletier_go_toml": "1d9442e54b744300238e0590d7216c2ce34ec1c51516e50f422946a60810f71f",
---
>   "//external:com_github_pelletier_go_toml": "2131bbfc71e19722cbcfcd6262f349de139bd198dc1478e1078f77468aef5ac8",
11,12c11,12
<   "//external:com_github_fsnotify_fsnotify": "705f7f1800547b80625ed9550543f276b3cc1124f58a97a86e5d2c4fe0e10be3",
<   "//external:com_github_rs_zerolog": "1ae495c13ad68f81f8a12d15e4ca23b9e359e7e52dd51bf29b8e6abd3e45cb9e",
---
>   "//external:com_github_fsnotify_fsnotify": "78fadeb3f4f535b0d74b674b93769e6819fa89b68ab2f528a34148e95806ad5d",
>   "//external:com_github_rs_zerolog": "1028e56c28478575f8d3a64ffec25dc85d2469ef89b75ec425e1525823a99488",
15c15
<   "//external:in_gopkg_yaml_v2": "2b4d8cfde049e875c93e38528bdb7a249bff551b7ff4caf0547f07953b4409bb",
---
>   "//external:in_gopkg_yaml_v2": "152fc1a853ba006b778634f18bfd3c086e519a259bce6941ec731deb9ab835ec",
18c18
<   "//external:com_github_bazelbuild_rules_go": "3a0865bfddf75458c31485e4c305904c04cdb2df52c93986053ad4b24c30b2aa",
---
>   "//external:com_github_bazelbuild_rules_go": "edf5a21b54d40aeda6585964afb7a7fbcbe2c860ed67c726038d3bd6e23046f4",
22c22
<   "//external:com_github_kr_text": "e8d75b53525ddb2ec3ba29b3a4aaa827c204dac696a6762f0a23659939ec9220",
---
>   "//external:com_github_kr_text": "998574c1d7bdcdecd2c3dd8cb3afef39ab5a63427b5aff9fd24ee0970b3cdbe4",
27,28c27,28
<   "//external:com_github_davecgh_go_spew": "e3212ef7ab80aab70c7250597d5fee5ebe8e34e69369a804f55f4dbc07a3bea6",
<   "//external:org_golang_x_net": "31c6917695194619df8f9a9fdec4c17fa1f3c5a691e6ede28e97640dd708eed5",
---
>   "//external:com_github_davecgh_go_spew": "d86252a55c6032a749ecacc7c900719a3d14ab718e4325516b62b7529fdbc920",
>   "//external:org_golang_x_net": "764f0843cdc8367c7e789f724817fe441692a5fd53cc2a257dd109affa6bd09e",
32c32
<   "//external:org_golang_x_text": "40fa854ffc099e0c8d89b3ae50aea9c9bf40fc588b081dfa0923243e6867d010",
---
>   "//external:org_golang_x_text": "d5ce531a28c54e3682e2fc64d685cba107d7891dd15a6e7b65a6fcd416643870",
38,40c38,40
<   "//:WORKSPACE": "946f540400d0819f2a9560a5e1415b1a09b95a046106b8f23588b7a457d7de5b",
<   "//external:com_github_kr_pty": "09277840b3f11e949947b07314020a7579e7ead4fc0abd87729d778d7b273b71",
<   "//external:com_github_pmezard_go_difflib": "a334ff2164b1018f256343f6729efd976d1659777fea888dcf4a8820b0d8fd56",
---
>   "//:WORKSPACE": "ec1cc9b48b848154725a374f8fddb8948ed740618d8fd02c9cf3bf8b02faf718",
>   "//external:com_github_kr_pty": "3485edec5acd4d701fde18c7f897daeb1e8a86be2a384eb9990d3298f04f3da9",
>   "//external:com_github_pmezard_go_difflib": "b1300b1184c695f203e22e51ce6f9b1d5994e7a59c68ca93695c585f700381b8",
42c42
<   "//external:org_golang_x_sync": "e39921babe450e121cfad126dc06988762a65c0b4825168212099aa92435e269",
---
>   "//external:org_golang_x_sync": "ae56798306b072b1c3bde8df25915800b4c5964546bd0919624d59221b7ef2ae",
47,48c47,48
<   "//external:org_golang_x_crypto": "c4e38b3ab791ab1643a17495962211eb5c2278ef0241c44e6104d4844b4ad4c2",
<   "//:project_lib": "b10df648bbb657679c6ab4344992e50c56ccf1d1d45d75364a67fad0c737a742",
---
>   "//external:org_golang_x_crypto": "90c56911e12a985da508b84ebfb6e887bea61e12da102a0ebce44221509f5676",
>   "//:project_lib": "1fefe55c301fe583b399dd6b1b930ef8d0a4f3962ba4f1b1435e2770232cecd8",
50c50
<   "//external:in_gopkg_check_v1": "b8e2106e05b8f536647c5d7e827f12a7b711e1a0b53e4010821cdebbe3846d07",
---
>   "//external:in_gopkg_check_v1": "fbce66b424002962111fbd882f6d09ca8b1039f3fb9dffdc2d715a010e31e29e",
54c54
<   "//external:org_golang_x_mod": "25083e56a5d2fd373223f9752e32114c999959046f0db85b0a5841ae77d16532",
---
>   "//external:org_golang_x_mod": "8716352ae28861ccdc1f4bb1ae9a0839d7aaf224d76bd0d2782a37d84a19b0f7",
56c56
<   "//external:bazel_gazelle_go_repository_config": "4dbf6519f3eecdc7aafad3f4ccf4a8db35c90d5a5686b9625e943250ca993c42",
---
>   "//external:bazel_gazelle_go_repository_config": "81bb955050b1682e632e51f1c12a741e853df7d361a1c489f5fd581e3dad1bd4",
65c65
<   "//:some-script": "60564e9161cd8c233f139c1c337eabe54a6ec9f01348576e12a805abda6d1993",
---
>   "//:some-script": "d63f3f179cf609cc35ac5fb514c69ff2498a2aa1d55f56d9f5bb86abe76f0523",
72c72
<   "//:project": "bff5ea58ba16b1a8046fc7fadc4fa5aa4938f469650b2783c8901b9cd12704ae",
---
>   "//:project": "90dddabbd0ddc6609defa9161b796a401c3596fc5ec23c558a8e7489b15206c2",
81c81
<   "//external:com_github_bmatcuk_doublestar": "a7d16a635ff53fafc705aae4532399724acd8f385236b6e4e3b5e2ce1743a2a9",
---
>   "//external:com_github_bmatcuk_doublestar": "bf52aea7749804d168dfdc9a7259eb5c0b16a34f5835bf7065027c4e7ca4b504",
93,94c93,94
<   "//external:com_github_burntsushi_toml": "c04a2cccf68f0a6bffbe27264d859db5ff17e53db8ed9da68ae8c87b967f0426",
<   "//external:com_github_google_go_cmp": "0b67469f337ad2e7bf5e488aa2a1fda52faf50616d4d6b29c4daeaf2df4803ca",
---
>   "//external:com_github_burntsushi_toml": "5334099e2e590d474aae6b19d9c9e4fc370ea8b70dc686db9a8d587c2a43a13a",
>   "//external:com_github_google_go_cmp": "2f11107a60838e66150a634f619061902870db30d7d89cdae32c2b2e5592f77e",

@tinder-maxwellelliott
Copy link
Collaborator

@mikberg is there any way to debug the BUILD files that are automatically created for go deps in a Gazelle project? Trying to understand how we are getting differences

@mikberg
Copy link
Author

mikberg commented Aug 23, 2021

@mikberg is there any way to debug the BUILD files that are automatically created for go deps in a Gazelle project? Trying to understand how we are getting differences

Hmm, you can at least bazel query @com_github_rs_zerolog//... --output=location to get the filesystem locations of the BUILD files.

@com6056
Copy link

com6056 commented Apr 20, 2022

This seems to also be impacting us, any debug info I can provide to help figure this out?

@cristifalcas
Copy link

I believe we are impacted as well because of this.
Between practically no changes between main and a test branch, we get a diff with hundred of targets

@com6056
Copy link

com6056 commented Apr 15, 2023

Curious if #174 will solve this, might give it a try and see 👀

@aw185176
Copy link

We are also running into this while trying to adopt bazel-diff. I looked into #174 and I don't think its quite right, since rules_go creates individual external repositories for each third party dependency. IIUC, #174 helps with situations where all external packages get put under one external repo namespace (eg @npm//..., etc).

I can confirm that a simple diff only touching WORKSPACE file produces a list of impacted targets that includes all external dependencies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants