Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up the test harness a bit #55

Merged
merged 2 commits into from
Jan 12, 2024
Merged

Clean up the test harness a bit #55

merged 2 commits into from
Jan 12, 2024

Conversation

bengl
Copy link
Member

@bengl bengl commented Jan 12, 2024

Primarily, this gets rid of the test/runtest executable, preventing the need for spawning an additional subprocess for each test. The current approach handles all the things that test/runtest would in a --require and in a loader that switches whether or not to use the IITM loader via the same filename check that was originally done in test/runtest.

One interesting thing that happened while making this change is that we need some way of passing the entrypoint filename to the loader. Since newer versions of Node.js run the loader in a separate thread, process.argv[1] no longer works. The approved alternative is to use inter-thread messages via the register() method, but that's not available on some versions of Node.js that also use loader threads. To get around this, an environment variable is set in the --require that's then propagated to the loader thread. This is a bit hacky, but probably the only thing that will work across the supported version range.

An attempt was made to use the more popular tap test package, rather than imhotap, but there's no version of tap that supports both the entire Node.js version range that IITM supports and also supports ESM. For now, this means sticking with imhotap, but were it not for the incompatibility, tap would be just as suitable here, and is a much more popular test package.

Copy link
Contributor

@jsumners-nr jsumners-nr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am concerned this will make debugging more difficult since this introduces a loader on top of the loader being tested.

Comment on lines 7 to 8
// Do not want this running on loader thread
if (wt.isMainThread) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style nit: I would keep the success case from being nested:

Suggested change
// Do not want this running on loader thread
if (wt.isMainThread) {
// Do not want this running on loader thread
if (wt.isMainThread === false) return

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I have muscle memory around linters that don't understand that CJS files are actually function blocks, so generally avoid top-level returns, even if they're warranted. Since the linter here allows it, I'll use it. Update incoming.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I've been ignoring such lint errors or just haven't seen them 😅

Primarily, this gets rid of the `test/runtest` executable, preventing
the need for spawning an additional subprocess for each test. The
current approach handles all the things that `test/runtest` would in a
`--require` and in a loader that switches whether or not to use the IITM
loader via the same filename check that was originally done in
`test/runtest`.

One interesting thing that happened while making this change is that we
need some way of passing the entrypoint filename to the loader. Since
newer versions of Node.js run the loader in a separate thread,
`process.argv[1]` no longer works. The approved alternative is to use
inter-thread messages via the `register()` method, but that's not
available on some versions of Node.js that also use loader threads. To
get around this, an environment variable is set in the `--require`
that's then propagated to the loader thread. This is a bit hacky, but
probably the only thing that will work across the supported version
range.

An attempt was made to use the more popular `tap` test package, rather
than `imhotap`, but there's no version of `tap` that supports both the
entire Node.js version range that IITM supports _and_ also supports ESM.
For now, this means sticking with `imhotap`, but were it not for the
incompatibility, `tap` would be just as suitable here, and is a much
more popular test package.
@bengl bengl force-pushed the bengl/test-harness-cleanup branch from bf2a083 to 68abc1d Compare January 12, 2024 14:08
@bengl
Copy link
Member Author

bengl commented Jan 12, 2024

@jsumners-nr

I am concerned this will make debugging more difficult since this introduces a loader on top of the loader being tested.

The wrapper loader used for testing is just a pass-through. It's not chaining loaders or anything like that. It should be completely transparent. What's the concern?

@jsumners-nr
Copy link
Contributor

@jsumners-nr

I am concerned this will make debugging more difficult since this introduces a loader on top of the loader being tested.

The wrapper loader used for testing is just a pass-through. It's not chaining loaders or anything like that. It should be completely transparent. What's the concern?

It's not a blocker. I'm just unclear what a debugging session will be like.

Qard
Qard previously approved these changes Jan 12, 2024
@bengl
Copy link
Member Author

bengl commented Jan 12, 2024

@jsumners-nr You make a good point here that the test README should be updated. I'll do that now.

@bengl bengl merged commit e29c02d into main Jan 12, 2024
89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants