Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: add workflow with sanitizers check #356

Merged
merged 122 commits into from
Jul 22, 2024

Conversation

alexander-e1off
Copy link
Collaborator

When PR is approved, run sanitizers check.

alexander-e1off and others added 30 commits March 28, 2024 18:28
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
Signed-off-by: Aleksandr Ivanov <[email protected]>
@alexander-e1off alexander-e1off marked this pull request as ready for review July 17, 2024 11:49
@678098 678098 self-requested a review July 17, 2024 12:04
@678098 678098 self-assigned this Jul 17, 2024
Copy link
Collaborator

@678098 678098 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments

key: deps-${{ steps.get-hash.outputs.deps_hash }}

run_asan:
if: github.event.review.state == 'APPROVED'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if: github.event.review.state == 'APPROVED'

These checks are not necessary for run_*san, since they already depend on build_dependencies which checks for this condition

set_build_type(DEBUG)

# Disable GNU c++ extensions.
set(CMAKE_CXX_EXTENSIONS OFF)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 4 *.cmake files which are almost identical.

diff asan.txt msan.txt

1c1
< # Compiler-less toolchain for building with {Clang, libc++} + AddressSanitizer.
---
> # Compiler-less toolchain for building with {Clang, libc++} + MemorySanitizer.
27a28,29
> set(MSAN_SUPPRESSION_LIST_PATH "$ENV{DIR_SRC_BMQ}/etc/msansup.txt")
> 
35d36
<        "-fno-optimize-sibling-calls "
36a38,39
>        "-fsanitize=memory "
>        "-fsanitize-blacklist=${MSAN_SUPPRESSION_LIST_PATH} "
97,99c100,107
< set( TOOLCHAIN_DEBUG_FLAGS
<      "-fsanitize=address "
<      )
---
> # Conditionally add flags helpful for debugging MemorySanitizer issues.
> if (DEBUG_MEMORY_SANITIZER)
>     string(CONCAT TOOLCHAIN_DEBUG_FLAGS
>            "${TOOLCHAIN_SHARED_FLAGS} "
>            "-fsanitize-memory-track-origins=2 "
>            "-fno-optimize-sibling-calls "
>           )
> endif()

diff asan.txt tsan.txt

1c1
< # Compiler-less toolchain for building with {Clang, libc++} + AddressSanitizer.
---
> # Compiler-less toolchain for building with {Clang, libc++} + ThreadSanitizer.
96a97,99
> set(TSAN_SUPPRESSION_LIST_PATH "$ENV{DIR_SRC_BMQ}/etc/tsansup.txt")
> set(THREAD_SANITIZER TRUE)
> 
98c101
<      "-fsanitize=address "
---
>      "-fsanitize=thread "
105a109
>

diff asan.txt ubsan.txt

96a97
> # Add MemorySanitizer-related flags.
98,99c99,100
<      "-fsanitize=address "
<      )
---
>   "-fsanitize=undefined "
>   )

Does it make sense to combine sanitizers configuration in one cmake file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial thought was to keep separate files for better readability, but since the differences are small it's worth to put sanitizers specific config in one file.

@@ -0,0 +1,106 @@
# Compiler-less toolchain for building with {Clang, libc++} + AddressSanitizer.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Compiler-less toolchain for building with {Clang, libc++} + AddressSanitizer.
# Compiler-less toolchain for building with {Clang, libc++} + UndefinedBehavoiurSanitizer.

Copy link
Collaborator

@678098 678098 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a few more questions, if some of these are too complex or obscure we might skip them

set(LIBCXX_BUILD_PATH "$ENV{LIBCXX_BUILD_PATH}")

# Force disabling the use of Readline. This is a holdover until readline builds with -fPIC.
set(BMQ_DISABLE_READLINE TRUE)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This setting doesn't have any effect now and should be removed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

string(CONCAT TOOLCHAIN_DEBUG_FLAGS
"${TOOLCHAIN_DEBUG_FLAGS} "
"-fsanitize-memory-track-origins=2 "
"-fno-optimize-sibling-calls "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why we don't enable -fno-optimize-sibling-calls for a general case launch of memory sanitizer here? Is the check too slow without it? This option is included for other sanitizers unconditionally.

At least docs advice to enable it for better understanding results:

https://releases.llvm.org/8.0.0/tools/clang/docs/MemorySanitizer.html#usage

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MemorySanitizer is the slowest one, but I checked that this option does not affect at the speed. So, made this option shared for all sanitizers.

"environment": {
"UBSAN_OPTIONS": {
"external_symbolizer_path": "/usr/bin/llvm-symbolizer",
"suppressions": "%%SRC%%/etc/ubsansup.txt",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We set these suppressions in two different places, here and in the .cmake file, e.g.:

set(TSAN_SUPPRESSION_LIST_PATH "$ENV{DIR_SRC_BMQ}/etc/tsansup.txt")

Is it possible to move all these suppressions to this common json config?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed it from cmake file. Now all suppressions that are passed at runtime are located in json config.

# Set the compiler flags. Use the CACHE instead of the *_INIT variants
# which are modified in the in the compiler initialization modules.
set(CMAKE_C_FLAGS "-march=westmere -m64 -fno-strict-aliasing" CACHE STRING "ABI C flags.")
set(CMAKE_C_FLAGS_RELWITHDEBINFO "-g -O2 -fno-omit-frame-pointer" CACHE STRING "ABI C flags.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we build everything in Debug, why do we specify this setting for "release with debug info"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

set(CMAKE_CXX_STANDARD 20 CACHE STRING "C++ standard.")
set(CMAKE_CXX_STANDARD_REQUIRED 1 CACHE BOOL "Standard version is required.")
set(CMAKE_CXX_FLAGS "-D_GLIBCXX_USE_CXX11_ABI=0 -march=westmere -m64 -fno-strict-aliasing" CACHE STRING "ABI C++ flags.")
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-g -O2 -fno-omit-frame-pointer" CACHE STRING "ABI C++ flags.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question

Copy link
Collaborator

@678098 678098 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexander-e1off the PR looks good, let's test it!

if(SANITIZER_NAME STREQUAL "asan")
set(TOOLCHAIN_DEBUG_FLAGS "-fsanitize=address ")
elseif(SANITIZER_NAME STREQUAL "msan")
set(MSAN_SUPPRESSION_LIST_PATH "$ENV{DIR_SRC_BMQ}/etc/msansup.txt")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this setting is still in the .cmake file

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, it is needed, because this file is passed at compile time.

build_dependencies:
name: Build deps [ubuntu]
runs-on: ubuntu-latest
if: github.event.review.state == 'APPROVED'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the state case sensitive?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In github actions doc it is in uppercase , but I saw also in lowercase, e.g. here

@alexander-e1off alexander-e1off requested a review from 678098 July 22, 2024 11:11
@alexander-e1off alexander-e1off merged commit b72e820 into bloomberg:main Jul 22, 2024
54 checks passed
@alexander-e1off alexander-e1off deleted the add_sanitizers_to_ci branch August 29, 2024 07:30
alexander-e1off added a commit to alexander-e1off/blazingmq that referenced this pull request Oct 24, 2024
alexander-e1off added a commit to alexander-e1off/blazingmq that referenced this pull request Oct 24, 2024
alexander-e1off added a commit to alexander-e1off/blazingmq that referenced this pull request Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants