Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump mysql integration CI to 8.3 #1508

Merged
merged 2 commits into from
Apr 15, 2024
Merged

Bump mysql integration CI to 8.3 #1508

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bc4f05e
bump to mysql 8.3
samuel40791765 Mar 14, 2024
3ee9a8c
add ldd checks to mysql integration
samuel40791765 Apr 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 26 additions & 20 deletions tests/ci/integration/run_mysql_integration.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

source tests/ci/common_posix_setup.sh

MYSQL_VERSION_TAG="mysql-8.1.0"
MYSQL_VERSION_TAG="mysql-8.3.0"
# This directory is specific to the docker image used. Use -DDOWNLOAD_BOOST=1 -DWITH_BOOST=<directory>
# with mySQL to download a compatible boost version locally.
BOOST_INSTALL_FOLDER=/home/dependencies/boost
Expand Down Expand Up @@ -36,7 +36,8 @@ rm -rf "${SCRATCH_FOLDER:?}"/*
cd ${SCRATCH_FOLDER}

function mysql_patch_reminder() {
LATEST_MYSQL_VERSION_TAG=mysql-`curl https://api.github.com/repos/mysql/mysql-server/tags | jq '.[].name' |grep '\-8.0' |sed -e 's/"mysql-cluster-\(.*\)"/\1/' |sort | tail -n 1`
# Check latest MySQL version. MySQL often updates with large changes depending on OpenSSL all at once, so we pin to a specific version.
LATEST_MYSQL_VERSION_TAG=`git describe --tags --abbrev=0`
if [[ "${LATEST_MYSQL_VERSION_TAG}" != "${MYSQL_VERSION_TAG}" ]]; then
aws cloudwatch put-metric-data --namespace AWS-LC --metric-name MySQLVersionMismatch --value 1
else
Expand All @@ -45,7 +46,7 @@ function mysql_patch_reminder() {
}

function mysql_build() {
cmake ${MYSQL_SRC_FOLDER} -GNinja -DWITH_BOOST=${BOOST_INSTALL_FOLDER} -DWITH_SSL=${AWS_LC_INSTALL_FOLDER} "-B${MYSQL_BUILD_FOLDER}" -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake ${MYSQL_SRC_FOLDER} -GNinja -DWITH_SSL=system -DCMAKE_PREFIX_PATH=${AWS_LC_INSTALL_FOLDER} "-B${MYSQL_BUILD_FOLDER}" -DCMAKE_BUILD_TYPE=RelWithDebInfo
time ninja -C ${MYSQL_BUILD_FOLDER}
ls -R ${MYSQL_BUILD_FOLDER}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a ldd ... | grep $AWS_LC_INSTALL_FOLDER/lib/libssl.so check to assert that mysql is properly built against AWS_LC?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, just added

Expand All @@ -56,9 +57,8 @@ function mysql_run_tests() {
#
# Tests marked with Bug#0000 are tests that have are known to fail in containerized environments. These tests aren't exactly relevant
# to testing AWS-LC functionality.
# Tests marked with Bug#0001 use DHE cipher suites for the connection. AWS-LC has no intention of supporting DHE cipher suites.
# Tests marked with Bug#0002 use stateful session resumption, otherwise known as session caching. It is known that AWS-LC does not
# currently support this.
# Tests marked with Bug#0001 use stateful session resumption, otherwise known as session caching. It is known that AWS-LC does not
# currently support this with TLS 1.3.
echo "main.mysqlpump_bugs : Bug#0000 Can't create/open a file ~/dump.sql'
main.restart_server : Bug#0000 mysqld is not managed by supervisor process
main.udf_bug35242734 : Bug#0000 mysqld is not managed by supervisor process
Expand All @@ -77,16 +77,9 @@ main.persisted_variables_bugs_fast : Bug#0000 Unsure
main.mysqldump : Bug#0000 contains nonaggregated column
main.func_math : Bug#0000 should have failed with errno 1690
main.derived_condition_pushdown : Bug#0000 Fails with OpenSSL as well. Not relevant to AWS-LC.
main.grant_alter_user_qa : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.grant_user_lock_qa : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.openssl_1 : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.ssl : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.ssl_cipher : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.ssl_dynamic : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.ssl-sha512 : Bug#0001 Uses DHE cipher suites in test, which AWS-LC does not support.
main.client_ssl_data_print : Bug#0002 AWS-LC does not support Stateful session resumption (Session Caching).
main.ssl_cache : Bug#0002 AWS-LC does not support Stateful session resumption (Session Caching).
main.ssl_cache_tls13 : Bug#0002 AWS-LC does not support Stateful session resumption (Session Caching).
main.client_ssl_data_print : Bug#0001 AWS-LC does not support Stateful session resumption (Session Caching).
main.ssl_cache : Bug#0001 AWS-LC does not support Stateful session resumption (Session Caching).
main.ssl_cache_tls13 : Bug#0001 AWS-LC does not support Stateful session resumption (Session Caching).
"> skiplist
./mtr --suite=main --force --parallel=auto --skip-test-list=${MYSQL_BUILD_FOLDER}/mysql-test/skiplist --retry-failure=3 --retry=3 --report-unstable-tests
popd
Expand Down Expand Up @@ -118,16 +111,29 @@ function mysql_patch_tests() {
done
}

# Get latest MySQL version. MySQL often updates with large changes depending on OpenSSL all at once, so we pin to a specific version.
mysql_patch_reminder
git clone https://github.com/mysql/mysql-server.git ${MYSQL_SRC_FOLDER} -b ${MYSQL_VERSION_TAG} --depth 1
git clone https://github.com/mysql/mysql-server.git ${MYSQL_SRC_FOLDER} --depth 1 --no-single-branch
mkdir -p ${AWS_LC_BUILD_FOLDER} ${AWS_LC_INSTALL_FOLDER} ${MYSQL_BUILD_FOLDER}
ls

aws_lc_build "$SRC_ROOT" "$AWS_LC_BUILD_FOLDER" "$AWS_LC_INSTALL_FOLDER" -DBUILD_TESTING=OFF -DBUILD_TOOL=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=0
aws_lc_build "$SRC_ROOT" "$AWS_LC_BUILD_FOLDER" "$AWS_LC_INSTALL_FOLDER" -DBUILD_TESTING=OFF -DBUILD_TOOL=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=1

pushd ${MYSQL_SRC_FOLDER}

mysql_patch_reminder
git checkout ${MYSQL_VERSION_TAG}

mysql_patch_tests
mysql_patch_error_strings

mysql_build
mysql_run_tests
popd

ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlclient.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libcrypto.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlclient.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libssl.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlharness_tls.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libcrypto.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlharness_tls.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libssl.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlrouter_routing.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libcrypto.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlrouter_routing.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libssl.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlrouter_http.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libcrypto.so" || exit 1
ldd "${MYSQL_BUILD_FOLDER}/lib/libmysqlrouter_http.so" | grep "${AWS_LC_INSTALL_FOLDER}/lib/libssl.so" || exit 1
Loading