Implement generational test isolation in bun:test

[Jarred-Sumner]

What does this PR do?

WIP. Towards #9243

Adds --isolate=lite to bun test

After each test() ends:

• kill() all Subprocess and Bun.$ created during the test
• Close all Bun.serve(), Bun.connect(), Bun.listen() created during the test

After each .test.* file finishes running:

• kill() all Subprocess and Bun.$ created since the module started
• Close all Bun.serve(), Bun.connect(), Bun.listen() created since the module started

Before bun test's process exits:

• kill() all Subprocess and Bun.$
• Close all Bun.serve(), Bun.connect(), Bun.listen()

How it works

Each test scope gets an (incremented) generation ID. I/O resources add themselves to the list of active resources and add the current generation ID. Each time we finish a test, a module, or a test scope we run a cleanup function for each I/O resource that has a generation ID >= the current generation ID.

TODO:

• Write tests for each type
• Store a pointer to ActiveSpySet in DescribeScope and TestScope, associate all mocks with these and restore all these mocks created per scope

How did you verify your code works?

[github-actions]

❌ @Jarred-Sumner 3 files with test failures on linux-x64-baseline:

• test/integration/next-pages/test/dev-server.test.ts 1 failing
• test/integration/next-pages/test/next-build.test.ts 1 failing
• test/js/node/child_process/child_process-node.test.js 1 failing

View test output

^{#15b808db07ded015d99c956a89d65fe28a0be4c7}

[github-actions]

❌ @Jarred-Sumner 2 files with test failures on linux-x64:

• test/integration/next-pages/test/dev-server.test.ts 1 failing
• test/integration/next-pages/test/next-build.test.ts 1 failing

View test output

^{#15b808db07ded015d99c956a89d65fe28a0be4c7}

[github-actions]

❌ @Jarred-Sumner 1 files with test failures on bun-darwin-aarch64:

• test/js/bun/http/serve.test.ts 2 failing

View test output

^{#15b808db07ded015d99c956a89d65fe28a0be4c7}

[github-actions]

❌ @Jarred-Sumner 9 files with test failures on bun-windows-x86_64-haswell

• test\cli\install\bunx.test.ts panic "integer overflow"
• test\cli\install\registry\bun-install-registry.test.ts 20 failing
• test\integration\next-pages\test\dev-server-ssr-100.test.ts 1 failing
• test\integration\next-pages\test\dev-server.test.ts 1 failing
• test\integration\next-pages\test\next-build.test.ts 1 failing
• test\js\bun\shell\bunshell.test.ts 2 failing
• test\js\node\child_process\child-process-exec.test.ts 2 failing
• test\js\node\dns\node-dns.test.js 2 failing
• test\js\web\timers\setTimeout.test.js SIGTERM

View test output

^{#f95bceba25884384b4a718c75fe52ab060cbebc0}

[github-actions]

❌ @Jarred-Sumner 3 files with test failures on bun-darwin-x64:

• test/js/bun/spawn/spawn.test.ts 2 failing
• test/js/bun/test/test-test.test.ts 1 failing
• test/js/web/workers/worker.test.ts 1 failing

View test output

^{#15b808db07ded015d99c956a89d65fe28a0be4c7}

[MonsterDeveloper]

This would be awesome. Currently mock restoration doesn't work with mock.module (see #7823 and #7376), and this is a big blocker for using Bun as a test runner.

Hopefully test isolation fixes this issue.

[Jarred-Sumner]

Closing due to bitrot. But would like to revisit in the future.