Break TaskRunEngine into Sync and Async classes

[bunchesofdonald]

This PR is meant to address the issue where the TaskRunEngine is blocking async tasks. The PR fixes this issue by doing a major refactor of TaskRunEngine into three parts:

• BaseTaskRunEngine - The core parts that are all sync and non-blocking.
• SyncTaskRunEngine - The bits that are sync and blocking, mostly dealing with the client.
• AsyncTaskRunEngine - The bits that are async, again mostly dealing with the client.

A lot of the refactor was also to break apart longer methods, like setup_run_context or begin_run into component parts that can be used by both the sync and async implementations.

Closes #14113

[codspeed-hq]

CodSpeed Performance Report

Merging #14643 will not alter performance

_{Comparing issue-14113 (10621f8) with main (21dd874)}

Summary

✅ 5 untouched benchmarks

[chrisguidry]

I was expecting to see more new tests, how well are we covering the async engine right now? I'm assuming we have a ton of tests for async tasks already and we're leaning on those? If so, that's super cool

[bunchesofdonald]

I was expecting to see more new tests, how well are we covering the async engine right now? I'm assuming we have a ton of tests for async tasks already and we're leaning on those? If so, that's super cool

Yeah, we cover them both pretty equally because we have two different entrypoints depending on sync/async. I considered trying to have literal test parity, but it didn't seem to make much sense given the current layout of the suite.