Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Babeltrace updates and Python modules #285681

Closed
wants to merge 7 commits into from
Closed

Babeltrace updates and Python modules #285681

wants to merge 7 commits into from

Conversation

hacker1024
Copy link
Member

@hacker1024 hacker1024 commented Feb 2, 2024 •
edited
Loading

Description of changes

This PR adds the Babeltrace 2 package along with its Python bindings. It is "completely independent from Babeltrace 1".

It also updates Babeltrace 1 to 1.5.11 (which mainly improves compatibility with newer Python and dependency versions) and adds a derivation for its Python bindings as well.

Cross-compilation (targeting aarch64-linux on x86_64-linux) has been tested.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@bjornfor

@hacker1024
Copy link
Member Author

Hmm, I'm not sure how this would fit into the by-name structure as it shares code with the 1.x derivation...

@hacker1024 hacker1024 force-pushed the package/python-babeltrace branch 2 times, most recently from 7243eda to 60ddae2 Compare February 2, 2024 01:55
@hacker1024 hacker1024 changed the title {,python3Packages}.babeltrace2: init at 2.0.5 Bbabeltrace updates and Python modules Feb 2, 2024
@hacker1024 hacker1024 changed the title Bbabeltrace updates and Python modules Babeltrace updates and Python modules Feb 2, 2024
@ofborg ofborg bot requested a review from bjornfor February 2, 2024 02:21
@infinisil
Copy link
Member

infinisil commented Feb 2, 2024 •
edited
Loading

There's some options:

  • Don't use pkgs/by-name and create something like pkgs.babeltraceVersions instead, alias the versions into all-packages.nix
  • Create a babeltrace builder function like pkgs.babeltraceFor, then use pkgs/by-name for the individual versions, calling that function within
  • Use .override on one version to define the other, then both can be in pkgs/by-name. Though this will give a messy .override forone of them, I don't recommend this!
  • Use something like inherit (callPackages ...) babeltrace babeltrace 2

What all of these have in common is that the different versions are somehow linked together. Thete might be other ways of doing it though :)

@hacker1024
Copy link
Member Author

Don't use pkgs/by-name and create something like pkgs.babeltraceVersions instead, alias the versions into all-packages.nix

I have attempted to do this, but it doesn't seem to be passing?

@infinisil
Copy link
Member

Ahh yeah sorry, I forgot to mention that the CI check has a false positive! It will be fixed once #285089 is used in CI, but for now you can just ignore the failing CI. It looks fine as is!

@infinisil infinisil mentioned this pull request Feb 9, 2024
Merged
@infinisil
Copy link
Member

The PR to fix the pkgs/by-name check is now merged and used in CI, so it won't fail anymore the next time you push :)

@infinisil infinisil mentioned this pull request Mar 18, 2024
Open
@hacker1024
babeltrace2: init at 2.0.5
7e1dccc 
Separate from babeltrace 1.x as "the Babeltrace 2 project is completely
independent from Babeltrace 1".

https://github.com/efficios/babeltrace/releases/tag/v2.0.0
@hacker1024
python3Packages.babeltrace2: init at 2.0.5
0d06439 
A new attribute overriding babeltrace2 is created, rather than enabling
Python support in babeltrace2 itself, in order to reduce the closure
size and leave babeltrace2 Python-version-agnostic.
@hacker1024
babeltrace: 1.5.8 -> 1.5.11
9db7679
@hacker1024
python3Packages.babeltrace: init at 1.5.11
51d77e8
@hacker1024
python3Packages.babeltrace, python3Packages.babeltrace2: group under …
c90fddf 
…babeltracePackages
@hacker1024
python3Packages.babeltrace, python3Packages.babeltrace2: Explain use …
8831cd4 
…of PYTHON environment variable
@hacker1024
Copy link
Member Author

@bjornfor I have added the comment from #285390 and wrapped commit messages to 72 lines once more.

@bjornfor
Copy link
Contributor

Result of nixpkgs-review pr 285681 run on x86_64-linux 1

6 packages marked as broken and skipped:
  • linuxKernel.packages.linux_4_19.perf
  • linuxKernel.packages.linux_4_19.perf.debug
  • linuxKernel.packages.linux_4_19_hardened.perf
  • linuxKernel.packages.linux_4_19_hardened.perf.debug
  • linuxPackages_4_19_hardened.perf
  • linuxPackages_4_19_hardened.perf.debug
18 packages failed to build:
  • ceph (ceph-dev)
  • ceph-client
  • ceph-csi
  • ceph.dev (ceph-dev.dev)
  • ceph.doc (ceph-dev.doc)
  • libceph (ceph.lib ,libceph.dev ,libceph.doc ,libceph.lib ,libceph.man)
  • ceph.man (ceph-dev.man)
  • python312Packages.babeltrace
  • python312Packages.babeltrace2
  • qemu_full
  • qemu_full.debug
  • qemu_full.ga
  • rocmPackages.mivisionx (rocmPackages.mivisionx-hip ,rocmPackages_5.mivisionx ,rocmPackages_5.mivisionx-hip)
  • rocmPackages.mivisionx-cpu (rocmPackages_5.mivisionx-cpu)
  • rocmPackages.mivisionx-opencl (rocmPackages_5.mivisionx-opencl)
  • sambaFull (samba4Full)
  • sambaFull.dev (samba4Full.dev)
  • sambaFull.man (samba4Full.man)
41 packages built:
  • babeltrace
  • babeltrace2
  • cargo-flamegraph
  • linuxPackages_5_10_hardened.perf (linuxKernel.packages.linux_5_10.perf)
  • linuxPackages_5_10_hardened.perf.debug (linuxKernel.packages.linux_5_10.perf.debug)
  • linuxPackages_5_15_hardened.perf (linuxKernel.packages.linux_5_15.perf)
  • linuxPackages_5_15_hardened.perf.debug (linuxKernel.packages.linux_5_15.perf.debug)
  • linuxKernel.packages.linux_5_4.perf
  • linuxKernel.packages.linux_5_4.perf.debug
  • linuxPackages_5_4_hardened.perf (linuxKernel.packages.linux_5_4_hardened.perf)
  • linuxPackages_5_4_hardened.perf.debug (linuxKernel.packages.linux_5_4_hardened.perf.debug)
  • linuxPackages.perf (linuxKernel.packages.linux_6_1.perf)
  • linuxPackages.perf.debug (linuxKernel.packages.linux_6_1.perf.debug)
  • linuxPackages_hardened.perf (linuxPackages_6_1_hardened.perf)
  • linuxPackages_hardened.perf.debug (linuxPackages_6_1_hardened.perf.debug)
  • linuxPackages_6_5_hardened.perf (linuxKernel.packages.linux_6_5.perf)
  • linuxPackages_6_5_hardened.perf.debug (linuxKernel.packages.linux_6_5.perf.debug)
  • linuxKernel.packages.linux_6_6.perf
  • linuxKernel.packages.linux_6_6.perf.debug
  • linuxPackages_6_6_hardened.perf (linuxKernel.packages.linux_6_6_hardened.perf)
  • linuxPackages_6_6_hardened.perf.debug (linuxKernel.packages.linux_6_6_hardened.perf.debug)
  • linuxPackages_latest.perf (linuxKernel.packages.linux_6_7.perf)
  • linuxPackages_latest.perf.debug (linuxKernel.packages.linux_6_7.perf.debug)
  • linuxPackages_6_7_hardened.perf (linuxKernel.packages.linux_6_7_hardened.perf)
  • linuxPackages_6_7_hardened.perf.debug (linuxKernel.packages.linux_6_7_hardened.perf.debug)
  • linuxPackages_latest-libre.perf (linuxKernel.packages.linux_latest_libre.perf)
  • linuxPackages_latest-libre.perf.debug (linuxKernel.packages.linux_latest_libre.perf.debug)
  • linuxPackages-libre.perf (linuxKernel.packages.linux_libre.perf)
  • linuxPackages-libre.perf.debug (linuxKernel.packages.linux_libre.perf.debug)
  • linuxPackages_lqx.perf (linuxKernel.packages.linux_lqx.perf)
  • linuxPackages_lqx.perf.debug (linuxKernel.packages.linux_lqx.perf.debug)
  • linuxPackages_xanmod.perf (linuxKernel.packages.linux_xanmod.perf)
  • linuxPackages_xanmod.perf.debug (linuxKernel.packages.linux_xanmod.perf.debug)
  • linuxPackages_xanmod_latest.perf (linuxKernel.packages.linux_xanmod_latest.perf ,linuxPackages_xanmod_stable.perf)
  • linuxPackages_xanmod_latest.perf.debug (linuxKernel.packages.linux_xanmod_latest.perf.debug ,linuxPackages_xanmod_stable.perf.debug)
  • linuxPackages_zen.perf (linuxKernel.packages.linux_zen.perf)
  • linuxPackages_zen.perf.debug (linuxKernel.packages.linux_zen.perf.debug)
  • lttv
  • python311Packages.babeltrace
  • python311Packages.babeltrace2
  • qtcreator

@infinisil infinisil mentioned this pull request Feb 29, 2024
Merged
@wentasah
Copy link
Contributor

wentasah commented May 9, 2024

I fixed build failures with Python 3.12. The needed changes are in my branch, which is rebased to nixos-unstable. @hacker1024 feel free to cherry pick the last three patches and put them here (or let me know and I'll submit a new PR from my branch).

It seems that the failures of ceph and qemu are not caused by this PR. I may have time to investigate the other failures reported by nixpkgs-review later.

@wentasah wentasah mentioned this pull request May 10, 2024
Merged
13 tasks
@wentasah
Copy link
Contributor

So I created a draft PR from my branch and run nixpkgs-review on it. Results are here. Only 3 packages fail, all due to a failure in python3Packages.torchWithRocm. However, this seems to be caused by non-reproducibility of the package. nix build --rebuild .#python3Packages.torchWithRocm complains about non-reproducibility and I can build the package on another system even with this PR.

@hacker1024
Copy link
Member Author

Thanks @wentasah, I'm happy to close this PR in favour of yours if that works for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bjornfor bjornfor Awaiting requested review from bjornfor

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@hacker1024 @infinisil @bjornfor @wentasah