GitHub Actions to replace grahamofborg-eval

[domenkozar]

For all the eval jobs running on ofborg we can easily port to GitHub actions and maintain them here.

After that we just build if there are less packages than let's say 500.

I do not have time to do this, but maybe someone is interested to help out.

[06kellyjac]

I'm interested but I'm not familiar with the inner workings of ofborg. The readme does look very useful though and the rust is nice and readable.
https://github.com/NixOS/ofborg#how-does-ofborg-call-nix-instantiate

What's the vision here?
Can the code in ofborg to pull out package names and safely evaluate (etc) be used as is or can it be pulled out to be used in an action?
https://github.com/NixOS/ofborg/blob/released/ofborg/src/nix.rs#L170-L187
Or would a bunch of this need re-implementing?

Also what's the plan for the chat commands e.g. @ofborg eval?

[domenkozar]

Can the code in ofborg to pull out package names and safely evaluate (etc) be used as is or can it be pulled out to be used in an action?

I'm not familiar with the code, so it can be factored out or rewritten, whatever is easiest.

Also what's the plan for the chat commands e.g. @ofborg eval?

I don't see a need for those (yet?), we can just do these things given it's a trusted environment.

[Mic92]

chat commands are easy to implement with github actions as well. I think there was a PR to add /build somewhere by @kloenk I don't think we can port eval jobs to github without having our own runners as it takes a lot of memory for evaluation.

[domenkozar]

How much is a lot? :)

[Mic92]

How much is a lot? :)

Our spot evaluator are equipped with 120GB RAM: https://monitoring.nix.ci/d/000000002/ofborg?orgId=1&refresh=10s&fullscreen&panelId=2

[domenkozar]

Looking at those graphs evaluations can take 15GB, while github actions runners have 7GB of memory.

I see a couple of ways out though:

• we could use swap since it's SSD
• it's on github roadmap to be able to pick machine size
• we could evaluate parts of nixpkgs/NixOS in different jobs

[FRidh]

I've described more actions that we could use/need in #104594.

[Mic92]

Looking at those graphs evaluations can take 15GB, while github actions runners have 7GB of memory.

I see a couple of ways out though:

* we could use swap since it's SSD

* it's on github roadmap to be able to pick machine size

* we could evaluate parts of nixpkgs/NixOS in different jobs

It might be possible to evaluate different prefix ranges on different nodes. I evaluated this option with nixpkgs-review. You need evaluate significantly less than half of all packages so to reduce the memory usage half.

[FRidh]

It might be possible to evaluate different prefix ranges on different nodes. I evaluated this option with nixpkgs-review. You need evaluate significantly less than half of all packages so to reduce the memory usage half.

Evaluating NixOS and the various Nixpkgs package sets sequentially could be an option?

[Mic92]

It might be possible to evaluate different prefix ranges on different nodes. I evaluated this option with nixpkgs-review. You need evaluate significantly less than half of all packages so to reduce the memory usage half.

Evaluating NixOS and the various Nixpkgs package sets sequentially could be an option?

Evaluating all packages sequentially would be quite slow. Partitioning is faster.

[stale]

I marked this as stale due to inactivity. → More info

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/infrastructure-announcement-the-future-of-ofborg-your-help-needed/56025/9

[wolfgangwalther]

This was done in #356023 and follow ups. If anything remains to be done, it would be tracked in #355847.