Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect checksum calc for odd-sized buffers on big-endian #1044

Open
ccpearson opened this issue Oct 28, 2022 · 0 comments
Open

Incorrect checksum calc for odd-sized buffers on big-endian #1044

ccpearson opened this issue Oct 28, 2022 · 0 comments

Comments

@ccpearson
Copy link

ccpearson commented Oct 28, 2022

On big-endian processors, the CalculateSum() func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:

  // Add remaining 8-bit to the one's complement sum
  if (odd) {
    sum64 += *reinterpret_cast<const uint8_t *>(buf16);
  }

This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)

For example, consider a five-byte buffer containing:

aabb ccdd ee

On BE, the expected adds should be:

     aabb
   + ccdd
   + ee00
    ~~~~~
    26598 = 659a (folded to 16 bits)

but will actually be:

     aabb
   + ccdd
   + 00ee
    ~~~~~
    17886 = 7887 (folded)

On LE, the adds should be (and will be):

     bbaa
   + ddcc
   + 00ee
    ~~~~~
    19a64 = 9a65 (folded)

Note that the LE result when byte-swapped equals the correct result from the BE calc, as expected.

This should give the correct result on both BE and LE:

   sum64 += be16_t(uint16_t(*reinterpret_cast<const uint8_t *>(buf16)) << 8).raw_value();

I can provide a test program to simulate this fix on an LE processor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant