Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support default test timeout in bunfig.toml #13988

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

treyturner
Copy link

@treyturner treyturner commented Sep 16, 2024

Hello there! I'm new to the project and Zig and thought I would try to help implement a small feature for which I voted.

What does this PR do?

This PR addresses #7789 by allowing default test timeout to be configured in bunfig.toml.

Thanks to @Jarred-Sumner for guidance 🙌

  • Documentation or TypeScript types (it's okay to leave the rest blank in this case)
  • Code changes

How did you verify your code works?

I wrote automated tests here.

A Zig file changed.

  • I checked the lifetime of memory allocated to verify it's (1) freed and (2) only freed when it should be
  • I included a test for the new code, or an existing test covers it
  • JSValue used outside outside of the stack is either wrapped in a JSC.Strong or is JSValueProtect'ed
  • I wrote TypeScript/JavaScript tests and they pass locally (bun-debug test test-file-name.test)

@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch 4 times, most recently from d5a2fa9 to 984ea2a Compare September 17, 2024 06:01
@treyturner treyturner marked this pull request as ready for review September 17, 2024 06:18
Copy link
Collaborator

@Jarred-Sumner Jarred-Sumner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this. Would you be able to add a test? I suggest putting it in test/cli/test and use tempDirWithFiles from "harness" to create a temp folder programmatically with the bunfig.toml. Probably set the default test timeout to something like 1 or 5 and then verify the process prints a timed out message

@Jarred-Sumner
Copy link
Collaborator

The caveat this creates is that specifying the default timeout via CLI will cause a non-standard value from bunfig.toml to be used instead. I'm not sure how to address this without more extensive changes that seem like overkill for such a corner case, but this is almost surely due to my inexperience with the project and Zig. Any assistance addressing this concern or improving this aspect would be greatly appreciated.

I think this block of code could be moved to run after the bunfig.toml is read:

bun/src/cli.zig

Lines 456 to 528 in ab07cf4

if (cmd == .TestCommand) {
if (args.option("--timeout")) |timeout_ms| {
if (timeout_ms.len > 0) {
ctx.test_options.default_timeout_ms = std.fmt.parseInt(u32, timeout_ms, 10) catch {
Output.prettyErrorln("<r><red>error<r>: Invalid timeout: \"{s}\"", .{timeout_ms});
Global.exit(1);
};
}
}
if (!ctx.test_options.coverage.enabled) {
ctx.test_options.coverage.enabled = args.flag("--coverage");
}
if (args.options("--coverage-reporter").len > 0) {
ctx.test_options.coverage.reporters = .{ .text = false, .lcov = false };
for (args.options("--coverage-reporter")) |reporter| {
if (bun.strings.eqlComptime(reporter, "text")) {
ctx.test_options.coverage.reporters.text = true;
} else if (bun.strings.eqlComptime(reporter, "lcov")) {
ctx.test_options.coverage.reporters.lcov = true;
} else {
Output.prettyErrorln("<r><red>error<r>: --coverage-reporter received invalid reporter: \"{s}\"", .{reporter});
Global.exit(1);
}
}
}
if (args.option("--coverage-dir")) |dir| {
ctx.test_options.coverage.reports_directory = dir;
}
if (args.option("--bail")) |bail| {
if (bail.len > 0) {
ctx.test_options.bail = std.fmt.parseInt(u32, bail, 10) catch |e| {
Output.prettyErrorln("<r><red>error<r>: --bail expects a number: {s}", .{@errorName(e)});
Global.exit(1);
};
if (ctx.test_options.bail == 0) {
Output.prettyErrorln("<r><red>error<r>: --bail expects a number greater than 0", .{});
Global.exit(1);
}
} else {
ctx.test_options.bail = 1;
}
}
if (args.option("--rerun-each")) |repeat_count| {
if (repeat_count.len > 0) {
ctx.test_options.repeat_count = std.fmt.parseInt(u32, repeat_count, 10) catch |e| {
Output.prettyErrorln("<r><red>error<r>: --rerun-each expects a number: {s}", .{@errorName(e)});
Global.exit(1);
};
}
}
if (args.option("--test-name-pattern")) |namePattern| {
const regex = RegularExpression.init(bun.String.fromBytes(namePattern), RegularExpression.Flags.none) catch {
Output.prettyErrorln(
"<r><red>error<r>: --test-name-pattern expects a valid regular expression but received {}",
.{
bun.fmt.QuotedFormatter{
.text = namePattern,
},
},
);
Global.exit(1);
};
ctx.test_options.test_filter_regex = regex;
}
ctx.test_options.update_snapshots = args.flag("--update-snapshots");
ctx.test_options.run_todo = args.flag("--todo");
ctx.test_options.only = args.flag("--only");
}

@treyturner
Copy link
Author

treyturner commented Sep 22, 2024

I think this block of code could be moved to run after the bunfig.toml is read:

10-4, thanks! Will likely be a few days before I can look at this again in detail, but it's on my TODO list.

@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch from 984ea2a to a1bbae5 Compare September 23, 2024 11:27
@treyturner
Copy link
Author

treyturner commented Sep 23, 2024

Ok, I've got the tests for this pushed. Will be looking at the implementation next chance I get.

@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch 4 times, most recently from c4148ff to acb7a62 Compare September 23, 2024 15:07
@treyturner
Copy link
Author

Ok @Jarred-Sumner, ready for another look when you get time!

@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch 12 times, most recently from 00128bf to 2b4c342 Compare September 27, 2024 14:43
@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch from 2b4c342 to 78bfbec Compare October 1, 2024 16:42
@treyturner
Copy link
Author

treyturner commented Oct 1, 2024

This is good to go from my perspective, but I'll pop in every now and again to rebase from main, and would be happy to address any further concerns or change requests that may arise.

@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch 2 times, most recently from 465c025 to c8eec81 Compare October 7, 2024 18:38
@treyturner treyturner force-pushed the feat/set_test_timeout_in_bunfig branch from c8eec81 to b48b712 Compare October 15, 2024 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants