Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline setup for aarch64-linux #9326

Draft
wants to merge 43 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
9600c36
build hab on aarch64-linux
sajjaphani Jul 23, 2024
e350fd7
try with arm64-linux
sajjaphani Jul 23, 2024
59f4bfb
try with arm-linux
sajjaphani Jul 23, 2024
3e96c31
with docker-linux-arm64 agent
sajjaphani Jul 23, 2024
bb7bba4
another try
sajjaphani Jul 23, 2024
e54284a
fix target
sajjaphani Jul 23, 2024
ba752f1
temporary channel for arm64
sajjaphani Sep 6, 2024
c84339e
adjust rustc_target from pkg_target
sajjaphani Sep 6, 2024
fcaa748
build pkgs on aarch64 in ci
sajjaphani Sep 9, 2024
f20b171
try with default-privileged-aarch64
sajjaphani Sep 9, 2024
73602a2
target default-privileged-aarch64 queue
sajjaphani Sep 9, 2024
ba7af1f
try varied config
sajjaphani Sep 9, 2024
fdfcace
default-arm64
sajjaphani Sep 9, 2024
386e737
try without privileged
sajjaphani Sep 9, 2024
90d6edf
revert privileged
sajjaphani Sep 9, 2024
6cb1171
fix shellcheck
sajjaphani Sep 9, 2024
5df4362
trigger pipeline
sajjaphani Sep 11, 2024
911d845
try without quotes
sajjaphani Sep 11, 2024
106a9c4
add missing space in cmd aqrgs
sajjaphani Sep 11, 2024
9c59e35
try with docker-linux-amazon-2023-arm64
sajjaphani Sep 19, 2024
e135a22
try sudo
sajjaphani Sep 19, 2024
2ef816a
target docker-linux-amazon-2023-arm64 with sudo
sajjaphani Sep 19, 2024
52f7b10
fix component name
sajjaphani Sep 23, 2024
bd0f999
disable shellcheck SC2086
sajjaphani Sep 25, 2024
15391a0
first cargo test
sajjaphani Oct 4, 2024
ed5925b
fix command
sajjaphani Oct 4, 2024
1257f99
add deps
sajjaphani Oct 4, 2024
11ae276
add all unit tests
sajjaphani Oct 4, 2024
d69bf24
fix shellcheck
sajjaphani Oct 4, 2024
0830856
adjust script, fix test
sajjaphani Oct 7, 2024
176c16f
install hab
sajjaphani Oct 7, 2024
93bce85
install hab with curl
sajjaphani Oct 7, 2024
2cca391
install hab
sajjaphani Oct 7, 2024
e3725c5
fix cfg
sajjaphani Oct 7, 2024
3657ac5
fix fmt
sajjaphani Oct 7, 2024
8e10602
updage agent
sajjaphani Oct 8, 2024
3abacab
adds space
sajjaphani Oct 8, 2024
ed14c12
should be an error
sajjaphani Oct 17, 2024
cc10dc0
aarch64-linux has standard pkgs
sajjaphani Oct 17, 2024
21b2dd0
fix unused import
sajjaphani Oct 17, 2024
ff04592
remove cfize, mesomise and fix container plan
sajjaphani Oct 17, 2024
781b585
set interpreter for aarch64 tests
sajjaphani Oct 18, 2024
779441c
fix rustfmt
sajjaphani Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .expeditor/scripts/shared.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,18 @@ curlbash_hab() {
sudo rm -rf /hab/pkgs/core/hab/0.82.0 || \
rm -rf /hab/pkgs/core/hab/0.82.0
fi
curl https://raw.githubusercontent.com/habitat-sh/habitat/main/components/hab/install.sh | sudo bash -s -- -t "$pkg_target"

# Use the temporarily uploaded Habitat package from the 'aarch64-linux' channel for aarch64.
# Once hab is released in the LTS channel, we can use either HAB_BLDR_CHANNEL or HAB_REFRESH_CHANNEL to install hab.
install_args="-t $pkg_target"
if [ "$pkg_target" = "aarch64-linux" ]; then
install_args+=" -c aarch64-linux -v 1.6.706"
fi
echo "install_args: $install_args"
# shellcheck disable=SC2086
curl https://raw.githubusercontent.com/habitat-sh/habitat/main/components/hab/install.sh | sudo bash -s -- ${install_args}
case "${pkg_target}" in
x86_64-linux | x86_64-linux-kernel2)
x86_64-linux | aarch64-linux | x86_64-linux-kernel2)
hab_binary="/bin/hab"
;;
x86_64-darwin)
Expand Down
5 changes: 5 additions & 0 deletions .expeditor/scripts/verify/build_package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,10 @@ HAB_CACHE_KEY_PATH="$JOB_TEMP_ROOT/keys"

echo "--- :key: Generating fake origin key"
${hab_binary} origin key generate
# Install the hab-studio from the 'aarch64-linux' channel.
# Once hab is released in the LTS channel, we can use either HAB_BLDR_CHANNEL or HAB_REFRESH_CHANNEL to install the studio.
if [ "$BUILD_PKG_TARGET" = "aarch64-linux" ]; then
${hab_binary} pkg install core/hab-studio/1.6.706 --channel aarch64-linux
fi
echo "--- :hab: Running hab pkg build for $package_path"
${hab_binary} pkg build "$package_path"
78 changes: 78 additions & 0 deletions .expeditor/scripts/verify/run_cargo_test-aarch64.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/bin/bash

set -eou pipefail

# Install hab
curl https://raw.githubusercontent.com/habitat-sh/habitat/main/components/hab/install.sh | sudo bash -s -- -t aarch64-linux -c aarch64-linux -v 1.6.706

# shellcheck source=.expeditor/scripts/shared.sh
source .expeditor/scripts/verify/shared.sh

if [[ ${1:-"--"} = "--" ]]; then
scope="habitat workspace"
else
component="$1"
shift
scope="$component"
fi

toolchain=$(get_toolchain)

sudo -E hab pkg install core/zeromq
sudo -E hab pkg install core/protobuf
sudo -E hab pkg install core/rust/"$toolchain"
sudo -E hab pkg install core/xz
sudo -E hab pkg install core/coreutils
sudo -E hab pkg install core/openssl
sudo -E hab pkg install core/perl
sudo -E hab pkg install core/make

export OPENSSL_DIR
OPENSSL_DIR="$(hab pkg path core/openssl)/bin"
export OPENSSL_LIB_DIR
OPENSSL_LIB_DIR="$(hab pkg path core/openssl)/lib"

export LIBZMQ_PREFIX
LIBZMQ_PREFIX=$(hab pkg path core/zeromq)
# now include zeromq and gcc so they exist in the runtime library path when cargo test is run
export LD_LIBRARY_PATH
LD_LIBRARY_PATH="$(hab pkg path core/gcc-base)/lib:$(hab pkg path core/zeromq)/lib:$(hab pkg path core/xz)/lib:$(hab pkg path core/openssl)/lib"

export LIBRARY_PATH
LIBRARY_PATH="$(hab pkg path core/xz)/lib"

export PROTOC_NO_VENDOR=1
export PROTOC
PROTOC=$(hab pkg path core/protobuf)/bin/protoc

_oldPth=$PATH
_pth="$(hab pkg path core/coreutils)/bin:$(hab pkg path core/openssl)/bin:$(hab pkg path core/perl)/bin:$(hab pkg path core/make)/bin"
eval "$(hab pkg env core/rust/"$toolchain"):$PATH"
export PATH="$PATH:$_pth:$_oldPth"

# Set testing filesystem root
export FS_ROOT
FS_ROOT=$(mktemp -d /tmp/testing-fs-root-XXXXXX)

export RUST_BACKTRACE=1

# Build the all the hab binaries so that we can run integration tests
if [[ "$scope" == "sup" ]]; then
cargo build
fi

echo "--- Running cargo test with scope '$scope' and args '$*'"

if [[ -n ${component:-} ]]; then
cd "components/$component"
fi

# We do not have any packages in the stable channel for aarch64 and probably never will.
# Set the HAB_INTERPRETER_IDENT to point to LTS-2024 to proceed with the tests.
export HAB_INTERPRETER_IDENT="core/busybox-static/1.36.1/20240805133911"

# Always add `--quiet` to avoid the noise of compilation in test output.
# The invocation to this script can add `--format pretty` to the test runner
# args (that is, after --, like --nocapture and --test-threads) if the names
# of the tests being run is desired in the output.
cargo test --quiet "$@"
Loading
Loading