refactor: rework how http source uses the transformer

[DavieReid]

The original http source implementation needed to dynamically import the transformer function because source options must be serialisable as they are eventually passed to a source worker thread.

When the createHttpSource function was recently introduced we sort of made the transformer optional and that responses would be transformed later. This was probably the wrong approach for the following reasons:

1. Sources should always return pages. By allowing the createHttpSource function to return raw responses that are transformed into pages later we break this contract.
2. A failed http request returns an empty array which then is flattened. The result is that come transformation time the index passed to the transform function can be out of sync with the response collection. Not good for transforms.

This change makes the transformer required again but you can pass it directly to the createHttpSource function.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4d3d967

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
mosaic	✅ Ready (Inspect)	Visit Preview	Dec 5, 2023 1:25pm