Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: update rebased gossipsub from libp2p #1185

Closed
wants to merge 6 commits into from
Closed

chore: update rebased gossipsub from libp2p #1185

wants to merge 6 commits into from

Conversation

chaitanyaprem
Copy link
Collaborator

@chaitanyaprem chaitanyaprem commented Aug 7, 2024 •
edited
Loading

@chaitanyaprem
Copy link
Collaborator Author

@richard-ramos any idea how to fix the nix build error?

[2024-08-09T02:43:46.698Z] error: builder for '/nix/store/c525b19gb2ch5q1yqxnmr7l422vwgr18-go-waku.drv' failed with exit code 1;
[2024-08-09T02:43:46.698Z]        last 10 log lines:
[2024-08-09T02:43:46.698Z]        > Building subPackage ./cmd/waku
[2024-08-09T02:43:46.698Z]        > go: inconsistent vendoring in /build/source:
[2024-08-09T02:43:46.698Z]        >  github.com/libp2p/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
[2024-08-09T02:43:46.698Z]        >         github.com/libp2p/go-libp2p-pubsub: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
[2024-08-09T02:43:46.698Z]        >    github.com/libp2p/[email protected]: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
[2024-08-09T02:43:46.698Z]        >         github.com/libp2p/[email protected]: is marked as replaced by github.com/waku-org/[email protected] in vendor/modules.txt, but replaced by github.com/waku-org/[email protected] in go.mod

@richard-ramos
Copy link
Member

Let's see now!

@chaitanyaprem
Copy link
Collaborator Author

Let's see now!

Tried that locally already and didn't work. This seems to be something else.
Can't we configure nix to not use vendors model and just build with go.mod?

@richard-ramos
Copy link
Member

cc: @jakubgs

@richard-ramos
Copy link
Member

I'll try creating a tag in the meantime. Because nix used to work with replaces (it does with go-ethereum) so maybe there's something weird going on when using a commit.

@richard-ramos
Copy link
Member

Meh... it did not work. I think however this has to be fixed because status-go and mobile do use nix in their build process!
I'm sure @jakubgs knows some magic incantation that will get nix to work!

@jakubgs
Copy link
Contributor

jakubgs commented Aug 19, 2024

Can't we configure nix to not use vendors model and just build with go.mod?

@chaitanyaprem @richard-ramos but it does use go.mod, you just need to update the SHA:

go-waku/default.nix

Lines 28 to 29 in bc16c74

# FIXME: This needs to be manually changed when updating modules.
vendorHash = "sha256-cOh9LNmcaBnBeMFM1HS2pdH5TTraHfo8PXL37t/A3gQ=";

Just like the comment says.

@chaitanyaprem
Copy link
Collaborator Author

Can't we configure nix to not use vendors model and just build with go.mod?

@chaitanyaprem @richard-ramos but it does use go.mod, you just need to update the SHA:

go-waku/default.nix

Lines 28 to 29 in bc16c74

# FIXME: This needs to be manually changed when updating modules.
vendorHash = "sha256-cOh9LNmcaBnBeMFM1HS2pdH5TTraHfo8PXL37t/A3gQ=";

Just like the comment says.

we did try updating the SHA, but still it was giving an error #1185 (comment)

@jakubgs
Copy link
Contributor

jakubgs commented Aug 20, 2024

Okay, there is a bug here, because there's two buildPackage functions:

go-waku/flake.nix

Lines 16 to 35 in d3b5113

buildPackage = system: subPackages:
let
pkgs = pkgsFor.${system};
commit = builtins.substring 0 7 (self.rev or "dirty");
version = builtins.readFile ./VERSION;
in pkgs.buildGo121Module {
name = "go-waku";
src = self;
inherit subPackages;
tags = [ ];
ldflags = [
"-X github.com/waku-org/go-waku/waku/v2/node.GitCommit=${commit}"
"-X github.com/waku-org/go-waku/waku/v2/node.Version=${version}"
];
doCheck = false;
# FIXME: This needs to be manually changed when updating modules.
vendorHash = "sha256-cOh9LNmcaBnBeMFM1HS2pdH5TTraHfo8PXL37t/A3gQ=";
# Fix for 'nix run' trying to execute 'go-waku'.
meta = { mainProgram = "waku"; };
};

You changed the one in flake.nix but the on in default.nix is in effect. Here's the fix:

Rebase and fix the hash.

@chaitanyaprem
Copy link
Collaborator Author

Okay, there is a bug here, because there's two buildPackage functions:

go-waku/flake.nix

Lines 16 to 35 in d3b5113

buildPackage = system: subPackages:
let
pkgs = pkgsFor.${system};
commit = builtins.substring 0 7 (self.rev or "dirty");
version = builtins.readFile ./VERSION;
in pkgs.buildGo121Module {
name = "go-waku";
src = self;
inherit subPackages;
tags = [ ];
ldflags = [
"-X github.com/waku-org/go-waku/waku/v2/node.GitCommit=${commit}"
"-X github.com/waku-org/go-waku/waku/v2/node.Version=${version}"
];
doCheck = false;
# FIXME: This needs to be manually changed when updating modules.
vendorHash = "sha256-cOh9LNmcaBnBeMFM1HS2pdH5TTraHfo8PXL37t/A3gQ=";
# Fix for 'nix run' trying to execute 'go-waku'.
meta = { mainProgram = "waku"; };
};

You changed the one in flake.nix but the on in default.nix is in effect. Here's the fix:

* [`go-waku#c324e3df`](https://github.com/status-im/go-waku/commit/c324e3df) - fix: remove duplicate buildPackage function from flake.nix

Rebase and fix the hash.

ok, Thanks!

i used to get the updated hash by running nix build locally using below command. But running that command throws error. is there any other way to get the hash to update it in default.nix file?

prem@Prems-MacBook-Pro go-waku % nix build --extra-experimental-features nix-command --extra-experimental-features flakes

warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled
error: builder for '/nix/store/p4gpdqcc2q785l1f2s5rs1415mrwk7hm-go-waku.drv' failed with exit code 1;
       last 10 log lines:
       > Building subPackage ./cmd/waku
       > go: inconsistent vendoring in /private/tmp/nix-build-go-waku.drv-0/source:
       > 	github.com/libp2p/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
       > 	github.com/libp2p/[email protected]: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
       > 	github.com/libp2p/[email protected]: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
       > 	github.com/libp2p/[email protected]: is marked as replaced in vendor/modules.txt, but not replaced in go.mod
       >
       > 	To ignore the vendor directory, use -mod=readonly or -mod=mod.
       > 	To sync the vendor directory, run:
       > 		go mod vendor
       For full logs, run 'nix-store -l /nix/store/p4gpdqcc2q785l1f2s5rs1415mrwk7hm-go-waku.drv'.

@jakubgs
Copy link
Contributor

jakubgs commented Aug 21, 2024 •
edited
Loading

You can try modifying the hash to be clearly incorrect, that will force Nix to re-evaluate that part.

@chaitanyaprem
Copy link
Collaborator Author

Closing this as #1208 already has these changes included and some more.

@jakubgs jakubgs deleted the chore/update-gossipsub branch September 3, 2024 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richard-ramos richard-ramos richard-ramos approved these changes

Assignees

@chaitanyaprem chaitanyaprem

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@chaitanyaprem @richard-ramos @jakubgs