Hashing and signing in two distinct operations yield different signature

[hermanclau]

I need to incrementally hash a ~10MB data file before generating a digital signature. To confirm the equivalency in signing with a specified algorithm, and hashing and signing separately, I ran the following test with dummy data payload. The resulting signatures are different, can you point out any mistakes in the usage?

Note: On a separate system with a 3rd party secure signing library, the generated signature matches that of the single operation below.

print(data)
b'Hello, world!\n'
sig2 = private2.sign(data, mechanism=pkcs11.mechanisms.Mechanism.SHA256_RSA_PKCS)
sig2.hex()
'7a3e06a472a4ddcc16d1264ce1678097a566f2ae07deef45a5ce1fb203b16c805b778a8b3f252bd3f4395194f7b7ba4e5aba2a09d07d475f445476930c804487deed101ed28866a6a59997ea1e941e34a6188baeb5a49d3b758fba9245c1b4266007c04035fff05e0d8533e868722f70dec7490a8ac83306f36aa32922375231ba375d0fb289e85b719f8aad8c716b1c3289557d3f5152e348b9ae5ce9626a199c87daf1edc03361b729e07509dc88a22e2651026785fc0102aa01d90e36a246e22eb5a0bf7bc484c266ac4c34ff8af66d88ff4c11d552508d67e355adcf3b0cd951762599881b26d7b65b2c240da339c341b009f19d3faf56cab2e6f7c635de'
data_digest = session.digest(data, mechanism=pkcs11.mechanisms.Mechanism.SHA256)
sig2 = private2.sign(data_digest, mechanism=pkcs11.mechanisms.Mechanism.RSA_PKCS)
sig2.hex()
'8040ffd4be2008b7124524c6d17d6a20936b2e6bc39870e6d212e442e130ce1290540d7e808b82fb5f06d3545996f1aec38287df744b75a758b705c3e99e0b5bba79f79bca3ed4aabaab8d0d5582a8bec2bb818bb6c21dd5e7e1bc78b61e454fbb6095b7b2ac64a6aa1c2c3d9b23913f6bcffabb9a2704bc98367a264746949b4ad9b313a1bc070084f0ec9931c309de6db0cdda5fb8d41b26b632c2a7e8fefb7d0bc53a7a7c06bc9109e0767510250d045506166e5b8a4add241848740427ab8516f583be8ad1b567201ba9756e8fc6ca4e75754d8449b67c360ec0bb0550f3c8ced9226cec700f1f9fbc9b1c4aceca45a1958814e01207343a6341086a7290'

[m32]

Mechanims CKM_SHA256_RSA_PKCS does following things:

1. Computes SHA256 hash of the data just like CKM_SHA256 does
2. Constructs DER encoded DigestInfo structure defined in RFC 8017
3. Signs DigestInfo structure with private key just like CKM_RSA_PKCS does

if 1:
from asn1crypto import algos
data = algos.DigestInfo({"digest_algorithm": algos.DigestAlgorithm({"algorithm": "sha256","parameters": algos.Null()}),"digest": data_digest,}).dump()
else:
DigestInfo = {
"MD5": (0x30,0x20,0x30,0x0C,0x06,0x08,0x2A,0x86,0x48,0x86,0xF7,0x0D,0x02,0x05,0x05,0x00,0x04,0x10,),
"SHA1": (0x30,0x21,0x30,0x09,0x06,0x05,0x2B,0x0E,0x03,0x02,0x1A,0x05,0x00,0x04,0x14,),
"SHA256":(0x30,0x31,0x30,0x0D,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x01,0x05,0x00,0x04,0x20,),
"SHA384":(0x30,0x41,0x30,0x0D,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x02,0x05,0x00,0x04,0x30,),
"SHA512":(0x30,0x51,0x30,0x0D,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x03,0x05,0x00,0x04,0x40,),
}
data = bytes(DigestInfo["SHA256"]) + data_digest

sig2 = private2.sign(data, mechanism=pkcs11.mechanisms.Mechanism.RSA_PKCS)
sig2.hex()