fix(transforms|batch): Allows batch operator to emit partial batches

[mdboon]

As the chief of the wind tribe, I don't mind if you have some other way you want to do this. I avoided adding another option because there's already a bunch of stuff going on in batch.

Basically I'm trying to solve the situation where:

1. Two consumers (for arguments sake) are listening for SQS events for email tracking,
2. One email (for arguments sake) is sent,
3. Each consumer receives the message 5 times, but neither reaches the options.n size of 10 and, thus, don't release the batch that tells AWS to delete the message,
4. The SQS queue, having sent the message 10 times, considers it dead and puts it on the DLQ.

By allowing partial batches to be emitted on a timeout, those messages can be properly cleaned up. To give you an idea of how often this happens, there's 1300 messages in the DLQ right now and I emptied it at the start of the week.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2dcb77f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[andogq]

Just to confirm, does the timeout option not work here?

[mdboon]

Just to confirm, does the timeout option not work here?

It depends on what you mean by work. If you have n and timeout in options then a timeout will only send a batch if there's n items in it, which leads to the scenario above (where a partial batch can be held for one or more timeouts).

[andogq]

Addressed in #41