Remove all uses of asyncio.shield()
in our codebase
#359
Labels
comp:agent
Related to Agent component
comp:common
Related to Common component
comp:manager
Related to Manager component
type:refactor
Refactor codes or add tests.
Milestone
It may make dangling but running tasks when shutting down, because it only cancels the outer
await
but keeps the inner task intact.asyncio.shield()
is useful to protect tasks that are "not well written to react on cancellation" but requires a strong assumption that all shielded tasks should terminate within a finite, short period of time.Historically aiopg did not handle cancellation of transactions during entering of them (aio-libs/aiopg#332), and we have experienced exhaustion of database connections due to accumulation of unreleased transactions and connections when there are cloud users from unstable WiFi (aiohttp cancels request handler tasks when the connection is interrupted). This is where we started to use
asyncio.shield()
for some database-critical operations, such asAgentRegistry.enqueue_session()
called from API handlers. Now we don't use aiopg with SQLAlchemy but asyncpg, which handles this better.Let's remove uses of
asyncio.shield()
and replace it withaiotools.PersistentTaskGroup
if necessary.Let's make separate PRs for each module code update.
I expect that this would reduce the possibility of getting stuck while shutting down services.
The text was updated successfully, but these errors were encountered: