[Bug] Binary Data is not being correctly written to DynamoDb Outbox

[iancooper]

Describe the bug

If the payload for a message is binary, then it is being mangled by the conversion between Message and MessageItem, which is using the Message Value property UTF-8 encoding on the bytes. We should be using Convert.ToBase64String when we have binary data.

To Reproduce

This has popped up where we are writing a Kafka payload with a payload header of the first 5 bytes of the message as:

magic byte (0) + 4byte schema-id

This means that although the payload after might well be JSON (although it could be Avro) we need to treat the content as it if were binary.

Suggested Fix

Where the body type is "application/octet-stream" we should treat the data as binary and use Convert.ToBase64String and not Encoding.UTF8.GetString. The simplest approach may be to fix this in the Message property which means we get this for 'free' wherever anyone marks up their content as "application/octet-stream"