[Git] Upgrade to v2.36.1 and add hack for macOS #4987

giordano · 2022-06-05T18:36:54Z

staticfloat · 2023-02-01T19:46:35Z

Does this not work properly?

giordano · 2023-02-01T20:00:01Z

The build per se works fine, but I also wanted to address JuliaVersionControl/Git.jl#40. However last time I tried it, it didn't seem to work around the SIP limitation, there's a build at https://github.com/giordano/Git_jll.jl if you want to play with it.

G/Git/build_tarballs.jl

Co-authored-by: Elliot Saba <[email protected]>

giordano · 2023-02-16T23:02:42Z

Uhm, this isn't working though:

julia> run(`$(git()) credential-osxkeychain`)
dyld[2558]: Library not loaded: '@rpath/libpcre2-8.0.dylib'
  Referenced from: '/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/bin/_git'
  Reason: tried: '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/bin/../lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/bin/../lib/libpcre2-8.0.dylib' (no such file), '/libpcre2-8.0.dylib' (no such file)
ERROR: failed process: Process(setenv(`/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/bin/git credential-osxkeychain`,["XPC_FLAGS=0x0", "PATH=/Users/mose/.julia/artifacts/41d2d4f312d93b56961767987cf48ea1f6d6fadc/bin:/Users/mose/.julia/artifacts/b1a055a9c307b137556560efcc2cee3a6405dfd7/bin:/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/bin:/Users/mose/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/mose/.cargo/bin", "COMMAND_MODE=unix2003", "PWD=/Users/mose", "XPC_SERVICE_NAME=0", "TERM_PROGRAM=iTerm.app", "HISTSIZE=1000000", "GIT_EXEC_PATH=/Users/mose/.julia/artifacts/1e557aed8ecb7576d4215c8ef8670079e81c11ef/libexec/git-core", "SHELL=/bin/zsh", "DYLD_FALLBACK_LIBRARY_PATH=/Users/mose/repo/julia/usr/lib:/Users/mose/.julia/artifacts/41d2d4f312d93b56961767987cf48ea1f6d6fadc/lib:/Users/mose/.julia/artifacts/b1a055a9c307b137556560efcc2cee3a6405dfd7/lib:/Users/mose/.julia/artifacts/b376941014acf8e5501996cdf9932036cfb3bb71/lib:/Users/mose/repo/julia/usr/bin/../lib/julia:/Users/mose/repo/julia/usr/bin/../lib:/Users/mose/lib:/usr/local/lib:/lib:/usr/lib"  …  "USER=mose", "JULIA_PKG_USE_CLI_GIT=true", "ITERM_SESSION_ID=w0t0p0:C27C9DE9-878A-4627-8A7B-9C9C7CE07D2F", "GIT_SSL_CAINFO=/Users/mose/repo/julia/usr/share/julia/cert.pem", "TERM=xterm-256color", "HOME=/Users/mose", "TERM_PROGRAM_VERSION=3.4.19", "JULIA_PKG_SERVER_REGISTRY_PREFERENCE=eager", "COLORTERM=truecolor", "OPENBLAS_MAIN_FREE=1"]), ProcessSignaled(6)) [0]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run(::Cmd)
   @ Base ./process.jl:477
 [4] top-level scope
   @ REPL[5]:1

giordano · 2023-02-16T23:11:06Z

Ah, nevermind, I was using vanilla Git.jl, which of course at the moment doesn't set JLL_DYLD_FALLBACK_LIBRARY_PATH. If I manually do that, then it works:

julia> run(ignorestatus(`$(git()) credential-osxkeychain`));
usage: git credential-osxkeychain <get|store|erase>

This will be fixed in JuliaVersionControl/Git.jl#45.

staticfloat · 2023-02-16T23:12:29Z

Ah, I was juuuuust about to post that! :)

giordano · 2023-02-16T23:14:59Z

Good to go now? 🙂

staticfloat

SGTM, I'll let you merge when you think enough other people have tried it out.

giordano · 2023-02-17T00:46:06Z

I'm going to merge to let people test it more easily in JuliaVersionControl/Git.jl#45, and this PR isn't any worse than the current situation anyway.

maleadt · 2023-02-20T07:45:07Z

I think this broke Git_jll for me on macOS: https://buildkite.com/julialang/metal-dot-jl/builds/251#01866c4e-d540-461b-af94-dca033cdc492

dyld[81854]: Library not loaded: @rpath/libpcre2-8.0.dylib
  Referenced from: /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/_git
  Reason: tried: '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/../lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/workspace/destdir/lib/libpcre2-8.0.dylib' (no such file), '/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/../lib/libpcre2-8.0.dylib' (no such file), '/libpcre2-8.0.dylib' (no such file)
ERROR: LoadError: failed process: Process(setenv(`/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/git -C /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/build/default-macmini-aarch64-5-0/julialang/metal-dot-jl/deps log -1 --format=%ct cmt`,["PATH=/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/41d2d4f312d93b56961767987cf48ea1f6d6fadc/bin:/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/b1a055a9c307b137556560efcc2cee3a6405dfd7/bin:/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin:/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/julia_installs/bin/mac/aarch64/1.8/julia-1.8-latest-macaarch64/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/Users/julia/.julia/artifacts/a81c923ea521c5513024eec57507055e4daa978a", "BUILDKITE_BUILD_CREATOR_TEAMS=", "BUILDKITE_LAST_HOOK_EXIT_STATUS=0", "BUILDKITE_PROJECT_PROVIDER=github", "GKS_ENCODING=utf8", "SECRET_CODECOV_TOKEN=[REDACTED]", "BASH_FUNC_nproc%%=() {  sysctl -n \"hw.perflevel0.logicalcpu\"\n}", "BUILDKITE_BUILD_CREATOR=github-actions[bot]", "BUILDKITE_REPO_MIRROR=/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/repos/https---github-com-JuliaGPU-Metal-jl-git", "BUILDKITE_AGENT_META_DATA_NUM_CPUS=4"  …  "BUILDKITE_AGENT_NAME=default-macmini-aarch64-5.0", "BUILDKITE_STEP_ID=01866c4e-d50b-40d3-afee-b485664c81a3", "CI=true", "BUILDKITE_STEP_KEY=", "BUILDKITE_TAG=", "BUILDKITE_LABEL=Julia 1.8", "BUILDKITE_COMMIT=bf5c3ca10d0fb172f4e586ff0b1b212c09920191", "BUILDKITE_PULL_REQUEST=99", "BUILDKITE_AGENT_PID=62594", "GIT_SSL_CAINFO=/Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-5.0/julia_installs/bin/mac/aarch64/1.8/julia-1.8-latest-macaarch64/share/julia/cert.pem"]), ProcessSignaled(6)) [0]

This is on the juliaecosystem CI machines, doing just run('$(git()) ...').

giordano · 2023-02-20T08:56:18Z

You should really use Git.jl, this jll is quite useless anyway without that package or without setting all the environment variables that package sets

maleadt · 2023-02-20T09:09:42Z

I am using Git.jl, which depends on Git_jll.

DilumAluthge · 2023-02-20T09:10:44Z

Presumably we need to cut a new release of Git.jl that includes JuliaVersionControl/Git.jl#45

DilumAluthge · 2023-02-20T09:11:40Z

JuliaRegistries/General#78066

maleadt · 2023-02-22T16:47:21Z

With Git.jl 1.3.0 I'm getting a different failure, again on macOS from juliaecosystem:

readlink: illegal option -- f
usage: readlink [-n] [file ...]
usage: dirname string [...]
/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/git: line 5: /Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/build/default-macmini-aarch64-1-0/julialang/metal-dot-jl/_git: No such file or directory
/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/git: line 5: exec: /Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/build/default-macmini-aarch64-1-0/julialang/metal-dot-jl/_git: cannot execute: No such file or directory
ERROR: LoadError: failed process: Process(setenv(`/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin/git -C /Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/build/default-macmini-aarch64-1-0/julialang/metal-dot-jl/deps log -1 --format=%ct cmt`,["PATH=/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/41d2d4f312d93b56961767987cf48ea1f6d6fadc/bin:/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/b1a055a9c307b137556560efcc2cee3a6405dfd7/bin:/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/depots/5cd495a2-4a16-4674-ae02-c839447744bb/artifacts/6a8cdbf9e758f66cee5f4f87fa20eb9be7521865/bin:/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/julia_installs/bin/macos/aarch64/1.9/julia-latest-macos-aarch64/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/Users/administrator/.julia/artifacts/a81c923ea521c5513024eec57507055e4daa978a", "BUILDKITE_BUILD_CREATOR_TEAMS=committers-at-julialang-slash-committers", "BUILDKITE_LAST_HOOK_EXIT_STATUS=0", "BUILDKITE_PROJECT_PROVIDER=github", "GKS_ENCODING=utf8", "SECRET_CODECOV_TOKEN=[REDACTED]", "BASH_FUNC_nproc%%=() {  sysctl -n \"hw.perflevel0.logicalcpu\"\n}", "BUILDKITE_BUILD_CREATOR=Tim Besard", "BUILDKITE_REPO_MIRROR=/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/repos/https---github-com-JuliaGPU-Metal-jl-git", "BUILDKITE_AGENT_META_DATA_NUM_CPUS=4"  …  "BUILDKITE_STEP_ID=01867a01-95ea-4f86-b65c-2e224efc44dc", "CI=true", "BUILDKITE_STEP_KEY=", "BUILDKITE_TAG=", "BUILDKITE_LABEL=Julia 1.9", "BUILDKITE_COMMIT=dd2be50b81e793c47b19592d4baee5c84ecaaf53", "BUILDKITE_PULL_REQUEST=85", "BUILDKITE_AGENT_PID=37308", "GIT_SSL_CAINFO=/Users/administrator/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-aarch64-1.0/julia_installs/bin/macos/aarch64/1.9/julia-latest-macos-aarch64/share/julia/cert.pem", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(126)) [126]

JuliaVersionControl/Git.jl#48

giordano · 2023-02-22T16:49:56Z

Right, readlink -f is probably GNU-only. Do you know what's a good replacement for the BSD tools?

giordano · 2023-02-22T17:27:49Z

Actually, I'm not sure -f is really needed, it was more an extra goody.

maleadt · 2023-02-22T18:15:46Z

Right, readlink -f is probably GNU-only. Do you know what's a good replacement for the BSD tools?

Sorry, no. Stack Overflow has some suggestions, https://stackoverflow.com/questions/1055671/how-can-i-get-the-behavior-of-gnus-readlink-f-on-a-mac, but yeah it'd be better if we can just drop it :-)

giordano force-pushed the mg/git branch from f0f2338 to 3d055e9 Compare June 5, 2022 18:52

giordano mentioned this pull request Feb 1, 2023

stop using libgit2 JuliaLang/Pkg.jl#2679

Open

giordano added 2 commits February 16, 2023 22:08

[Git] Upgrade to v2.36.1 and add hack for macOS

1069a88

[Git] Remove uneeded hack for FreeBSD

2bc06db

giordano force-pushed the mg/git branch from 3d055e9 to 60f9065 Compare February 16, 2023 22:08

[Git] Add some compat bounds

33f854f

giordano force-pushed the mg/git branch 2 times, most recently from 15d4219 to aa64caa Compare February 16, 2023 22:38

staticfloat reviewed Feb 16, 2023

View reviewed changes

G/Git/build_tarballs.jl Outdated Show resolved Hide resolved

giordano mentioned this pull request Feb 16, 2023

Trouble with auth on macOS due to SIP JuliaVersionControl/Git.jl#40

Closed

giordano and others added 2 commits February 16, 2023 22:50

[Git] Create git shell driver for macOS

b7e470a

Co-authored-by: Elliot Saba <[email protected]>

[Git] Try removing the hack inside the shell and perl scripts

9dfcde2

giordano force-pushed the mg/git branch from aa64caa to 9dfcde2 Compare February 16, 2023 22:51

giordano marked this pull request as ready for review February 16, 2023 22:57

giordano requested a review from staticfloat February 16, 2023 22:58

giordano marked this pull request as draft February 16, 2023 23:01

giordano marked this pull request as ready for review February 16, 2023 23:11

giordano mentioned this pull request Feb 16, 2023

Add work around for SIP restrictions on macOS JuliaVersionControl/Git.jl#45

Merged

staticfloat approved these changes Feb 17, 2023

View reviewed changes

giordano merged commit 9736441 into JuliaPackaging:master Feb 17, 2023

giordano deleted the mg/git branch February 17, 2023 00:46

maleadt mentioned this pull request Feb 20, 2023

Update manifest JuliaGPU/GPUCompiler.jl#393

Merged

maleadt mentioned this pull request Feb 20, 2023

Update manifest JuliaGPU/Metal.jl#99

Merged

giordano mentioned this pull request Feb 20, 2023

New version: Git v1.3.0 JuliaRegistries/General#78066

Merged

giordano mentioned this pull request Feb 23, 2023

Issues with wrapper scripts on the juliaecosysten macOS CI machines JuliaVersionControl/Git.jl#48

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Git] Upgrade to v2.36.1 and add hack for macOS #4987

[Git] Upgrade to v2.36.1 and add hack for macOS #4987

giordano commented Jun 5, 2022 •

edited

Loading

staticfloat commented Feb 1, 2023

giordano commented Feb 1, 2023

giordano commented Feb 16, 2023

giordano commented Feb 16, 2023 •

edited

Loading

staticfloat commented Feb 16, 2023

giordano commented Feb 16, 2023

staticfloat left a comment

giordano commented Feb 17, 2023

maleadt commented Feb 20, 2023

giordano commented Feb 20, 2023 •

edited

Loading

maleadt commented Feb 20, 2023

DilumAluthge commented Feb 20, 2023

DilumAluthge commented Feb 20, 2023

maleadt commented Feb 22, 2023 •

edited

Loading

giordano commented Feb 22, 2023

giordano commented Feb 22, 2023

maleadt commented Feb 22, 2023

[Git] Upgrade to v2.36.1 and add hack for macOS #4987

[Git] Upgrade to v2.36.1 and add hack for macOS #4987

Conversation

giordano commented Jun 5, 2022 • edited Loading

staticfloat commented Feb 1, 2023

giordano commented Feb 1, 2023

giordano commented Feb 16, 2023

giordano commented Feb 16, 2023 • edited Loading

staticfloat commented Feb 16, 2023

giordano commented Feb 16, 2023

staticfloat left a comment

Choose a reason for hiding this comment

giordano commented Feb 17, 2023

maleadt commented Feb 20, 2023

giordano commented Feb 20, 2023 • edited Loading

maleadt commented Feb 20, 2023

DilumAluthge commented Feb 20, 2023

DilumAluthge commented Feb 20, 2023

maleadt commented Feb 22, 2023 • edited Loading

giordano commented Feb 22, 2023

giordano commented Feb 22, 2023

maleadt commented Feb 22, 2023

giordano commented Jun 5, 2022 •

edited

Loading

giordano commented Feb 16, 2023 •

edited

Loading

giordano commented Feb 20, 2023 •

edited

Loading

maleadt commented Feb 22, 2023 •

edited

Loading