Skip to content

Commit

Permalink
ggml : prevent builds with -ffinite-math-only (ggerganov#7726)
Browse files Browse the repository at this point in the history
This enforces a check that -fno-finite-math-only was set and that the operating
compiling mode is not in finite maths mode. This is because during rewriting of
silu and softmax for cpu ggerganov#7154 there emerged an issue where the result that was
observed when >1 slot was nondeterministic as found by @JohannesGaessler.

@LostRuins narrowed the problem down to -ffinite-math-only which was theorised
to be due to SiLU, instead of flushing small values to 0, returns NaN or some 
other garbage. @jart proposed a fix that @ggerganov then implemented in this fix

ref ggerganov#7154 (comment)
  • Loading branch information
Georgi Gerganov authored Jun 4, 2024
1 parent bde7cd3 commit 6d16169
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 1 deletion.
2 changes: 1 addition & 1 deletion cmake/arm64-windows-llvm.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ set( CMAKE_CXX_COMPILER clang++ )
set( CMAKE_C_COMPILER_TARGET ${target} )
set( CMAKE_CXX_COMPILER_TARGET ${target} )

set( arch_c_flags "-march=armv8.7-a -fvectorize -ffp-model=fast" )
set( arch_c_flags "-march=armv8.7-a -fvectorize -ffp-model=fast -fno-finite-math-only" )
set( warn_c_flags "-Wno-format -Wno-unused-variable -Wno-unused-function -Wno-gnu-zero-variadic-macro-arguments" )

set( CMAKE_C_FLAGS_INIT "${arch_c_flags} ${warn_c_flags}" )
Expand Down
5 changes: 5 additions & 0 deletions ggml.c
Original file line number Diff line number Diff line change
Expand Up @@ -2272,6 +2272,11 @@ inline static float ggml_silu_f32(float x) {
return x/(1.0f + expf(-x));
}

#if __FINITE_MATH_ONLY__
#error "some routines in ggml.c require non-finite math arithmetics -- pass -fno-finite-math-only to the compiler to fix"
#error "ref: https://github.com/ggerganov/llama.cpp/pull/7154#issuecomment-2143844461"
#endif

#if defined(__ARM_NEON) && defined(__aarch64__)

// adapted from arm limited optimized routine
Expand Down

0 comments on commit 6d16169

Please sign in to comment.