Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AVX-512 support for RSA Signing #1273

Merged
merged 36 commits into from
Sep 17, 2024
Merged

AVX-512 support for RSA Signing #1273

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
b9088fc
Use IFMA_AVX512 when possible for modular exponentiation.
pittma Aug 7, 2023
e6269ff
Add test coverage for consttime_x2 mod exp function
pittma Oct 23, 2023
6d2ece9
Add fuzzer coverage for BN_mod_exp_mont_consttime_x2
pittma Oct 23, 2023
e0ad9da
prevent empty translation units for compilers that don't like them
pittma Oct 30, 2023
024a9ec
properly handle AVX-512 build conditions
pittma Oct 31, 2023
cd2a3d1
fips builds require subsections
pittma Oct 31, 2023
d4d89fc
fix disallowed interaction with `OPENSSL_ia32_cap_P` in fips mode
pittma Nov 2, 2023
a0f3737
reset sections when they change for variable declaration
pittma Nov 2, 2023
8e55af5
include avx512ifma flag
pittma Nov 3, 2023
7d1ea20
handle AVX-512 mask register usage in fips delocation process
pittma Nov 15, 2023
407df8d
address review comments
pittma Jan 30, 2024
e67bbda
regen generated source
pittma Feb 1, 2024
b33709e
regenerate delocate parser
pittma Feb 1, 2024
0e7c607
AVX-512 RSA Signing: address first PR review
pittma Apr 10, 2024
b2d1327
Merge remote-tracking branch 'origin/main'
pittma Apr 10, 2024
14fefe0
Still export the parallel mod_exp implementation
pittma Apr 12, 2024
5e1c7ee
second set of review comments and documentation
pittma Apr 24, 2024
73d389d
fix generated source conflict
pittma Apr 24, 2024
087bf5c
Merge branch 'main' of github.com:aws/aws-lc into pmain
pittma Jul 25, 2024
c439bf0
address review 3 comments
pittma Jul 25, 2024
abe1124
Merge branch 'main' of github.com:aws/aws-lc
pittma Aug 7, 2024
37b4a4a
Merge branch 'main' of github.com:aws/aws-lc into pmain
pittma Sep 5, 2024
e06d8d0
further review comments
pittma Sep 4, 2024
bf9fc29
add ABI tests for new RSA AVX-512 assmebly routines
pittma Sep 5, 2024
e626c2c
add dispatch tests for AVX-512 enabled RSA signing
pittma Sep 5, 2024
92b9e3f
fix dispatch test
pittma Sep 6, 2024
1055b42
Merge remote-tracking branch 'origin/main'
pittma Sep 6, 2024
58af762
Merge branch 'main' of github.com:aws/aws-lc
pittma Sep 9, 2024
56d8fd6
fix conditional build logic in dispatch test
pittma Sep 9, 2024
f925e7c
generated asm should properly exclude when using old assembler
pittma Sep 9, 2024
2473469
Merge branch 'main' of github.com:aws/aws-lc
pittma Sep 10, 2024
ef26ced
in ninja-based build, old assembler logic is already handled
pittma Sep 10, 2024
73b7b8f
Merge branch 'main' of github.com:aws/aws-lc
pittma Sep 10, 2024
506dced
Increasing the capacity of ubuntu2004_android_fips_static_release.
nebeid Sep 11, 2024
0dd53a1
Merge branch 'main' into main
nebeid Sep 11, 2024
f3715bb
Merge branch 'main' of github.com:aws/aws-lc
pittma Sep 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions crypto/fipsmodule/bn/rsaz_exp_x2.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@ int RSAZ_mod_exp_avx512_x2(uint64_t *res1,
uint64_t k0_2,
int modlen)
{
dkostic marked this conversation as resolved.
Show resolved Hide resolved
#ifdef BORINGSSL_DISPATCH_TEST
BORINGSSL_function_hit[8] = 1;
#endif
typedef void (*AMM)(uint64_t *res, const uint64_t *a,
const uint64_t *b, const uint64_t *m, uint64_t k0);
int ret = 0;
Expand Down Expand Up @@ -226,6 +229,7 @@ int rsaz_mod_exp_x2_ifma256(uint64_t *out,
const uint64_t k0[2],
int modlen)
{

typedef void (*DAMM)(uint64_t *res, const uint64_t *a,
const uint64_t *b, const uint64_t *m,
const uint64_t k0[2]);
Expand Down
37 changes: 37 additions & 0 deletions crypto/impl_dispatch_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
#include "internal.h"
#include "fipsmodule/cpucap/internal.h"
#include "fipsmodule/modes/internal.h"
#include "fipsmodule/bn/rsaz_exp.h"


class ImplDispatchTest : public ::testing::Test {
Expand All @@ -39,6 +40,7 @@ class ImplDispatchTest : public ::testing::Test {
aes_hw_ = CRYPTO_is_AESNI_capable();
avx_movbe_ = CRYPTO_is_AVX_capable() && CRYPTO_is_MOVBE_capable();
aes_vpaes_ = CRYPTO_is_SSSE3_capable();
ifma_avx512 = CRYPTO_is_AVX512IFMA_capable();
sha_ext_ =
// TODO(CryptoAlg-2137): sha_ext_ isn't enabled on Windows Debug Builds with newer
// 32-bit Intel processors.
Expand Down Expand Up @@ -119,6 +121,7 @@ class ImplDispatchTest : public ::testing::Test {
bool is_x86_64_ = false;
bool is_assembler_too_old = false;
bool is_assembler_too_old_avx512 = false;
bool ifma_avx512 = false;
#else // AARCH64
bool aes_gcm_pmull_ = false;
bool aes_gcm_8x_ = false;
Expand All @@ -139,6 +142,7 @@ constexpr size_t kFlag_sha256_hw = 6;
#if defined(OPENSSL_X86) || defined(OPENSSL_X86_64)
constexpr size_t kFlag_aesni_gcm_encrypt = 2;
constexpr size_t kFlag_aes_gcm_encrypt_avx512 = 7;
constexpr size_t kFlag_RSAZ_mod_exp_avx512_x2 = 8;
#else // AARCH64
constexpr size_t kFlag_aes_gcm_enc_kernel = 2;
constexpr size_t kFlag_aesv8_gcm_8x_enc_128 = 7;
Expand Down Expand Up @@ -243,6 +247,39 @@ TEST_F(ImplDispatchTest, SHA512) {
}
#endif // OPENSSL_AARCH64


#if defined(OPENSSL_X86_64) && !defined(MY_ASSEMBLER_IS_TOO_OLD_512AVX) && \
defined(RSAZ_512_ENABLED)
TEST_F(ImplDispatchTest, BN_mod_exp_mont_consttime_x2) {
AssertFunctionsHit(
{
{kFlag_RSAZ_mod_exp_avx512_x2,
is_x86_64_ &&
!is_assembler_too_old_avx512 &&
ifma_avx512},
},
[] {
uint64_t res1 = 0;
uint64_t base1 = 0;
uint64_t exp1 = 0;
uint64_t m1 = 0;
uint64_t rr1 = 0;
uint64_t k0_1 = 0;
uint64_t res2 = 0;
uint64_t base2 = 0;
uint64_t exp2 = 0;
uint64_t m2 = 0;
uint64_t rr2 = 0;
uint64_t k0_2 = 0;
int modlen = 0;

RSAZ_mod_exp_avx512_x2(&res1, &base1, &exp1, &m1, &rr1, k0_1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be calling BN_mod_exp_mont_consttime_x2 in exponentiation.c to make sure this function gets called. It may be why the test is failing as follows: the function is called where the conditions in flag.second are false.

[ RUN      ] ImplDispatchTest.BN_mod_exp_mont_consttime_x2
../crypto/impl_dispatch_test.cc:105: Failure
Expected equality of these values:
  flag.second
    Which is: false
  BORINGSSL_function_hit[flag.first] == 1
    Which is: true
Google Test trace:
../crypto/impl_dispatch_test.cc:103: 8

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course you're right. I'm not sure what I was thinking here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I even named the test the right thing! But got mixed up while filling it in. Anyways, 92b9e3f fixes this, and with that I think we're caught up.

&res2, &base2, &exp2, &m2, &rr2, k0_2,
modlen);
});
}
#endif // OPENSSL_X86_64 && !MY_ASSEMBLER_IS_TOO_OLD_512AVX && RSAZ_512_ENABLED

#endif // !OPENSSL_NO_ASM && (OPENSSL_X86 || OPENSSL_X86_64 || OPENSSL_AARCH64)

#endif // DISPATCH_TEST && !SHARED_LIBRARY
1 change: 1 addition & 0 deletions crypto/internal.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1343,6 +1343,7 @@ OPENSSL_INLINE int boringssl_fips_break_test(const char *test) {
// 5: vpaes_set_encrypt_key
// 6: sha256_block_data_order_shaext
// 7: aes_gcm_encrypt_avx512
// 8: RSAZ_mod_exp_avx512_x2
// On AARCH64:
// 0: aes_hw_ctr32_encrypt_blocks
// 1: aes_hw_encrypt
Expand Down