Wrong call to phpunit when asking for test groups

[tenzap]

Apache NetBeans version

Apache NetBeans 24

What happened

The call to phpunit when asking for test groups isn't correct so that it reports that there is no group.

Updating the call would fix the issue.

Language / Project Type / NetBeans Component

PHP, testing with phpunit

How to reproduce

I setup a PHP project to run tests with PHPUnit
In the phpunit config (project properties / Testing / PHPUnit) I check "Ask for test groups before running tests".
I also have "test project using phpunit command only" set. As well as the XML file, and path to phpunit.

netbeans will invoke phpunit with

"/usr/bin/php" ".../vendor/bin/phpunit" "--colors" "--configuration" ".../application/tests/phpunit.xml" "--list-groups" "."

Output it

PHPUnit 10.5.40 by Sebastian Bergmann and contributors.

Available test group(s):
Done.

The part "--list-groups" "." should actually be "--list-groups" (ie. without the "." part).

When I run the command myself without the "." part), phpunit returns the list of groups, otherwise it doesn't.

"/usr/bin/php" ".../vendor/bin/phpunit" "--colors" "--configuration" ".../application/tests/phpunit.xml" "--list-groups"

PHPUnit 10.5.40 by Sebastian Bergmann and contributors.

Available test group(s):
 - default
 - html5

Did this work correctly in an earlier version?

No / Don't know

Operating System

Debian 12

JDK

Java: 17.0.13; OpenJDK 64-Bit Server VM 17.0.13+11-Debian-2deb12u1

Apache NetBeans packaging

Apache NetBeans provided installer

Anything else

No response

Are you willing to submit a pull request?

No

[junichi11]

@tenzap Can you provide an example project as a zip file to reproduce it?

[tenzap]

See: https://github.com/tenzap/phpunit-simple-example/tree/nb24-issue8082
(branch nb24-issue8082)

Launching composer install should be sufficient to install phpunit 10.5

For the configuration:

It looks like if I tick "test project using just phpunit", it shouldn't add the . to the phpunit command.

Actually the . refers to the directory in which phpunit searches test files. For phpunit, if --test-suffix is not set, it defaults searching only for files having suffix Test.php,.phpt. However, in this project the suffix is _test.php, so the . wouldn't find the test files, and that would explain why no groups are found.

BTW, if I tick "use XML configuration", maybe netbeans shouldn't even append the test dir to any phpunit call? I don't know if this makes sense.

[junichi11]

@tenzap Thanks for the example.

[junichi11]

@tmysik Can we remove .? (Is there a problem?)

netbeans/php/php.phpunit/src/org/netbeans/modules/php/phpunit/commands/PhpUnit.java

Lines 348 to 354 in be051eb

	List<String> params = createParams(true);
	addBootstrap(phpModule, params);
	addConfiguration(phpModule, params);
	params.add(LIST_GROUPS_PARAM);
	// list test groups from the current workdir
	params.add("."); // NOI18N
	phpUnit.additionalParameters(params);

[tmysik]

@tmysik Can we remove .? (Is there a problem?)

netbeans/php/php.phpunit/src/org/netbeans/modules/php/phpunit/commands/PhpUnit.java

Lines 348 to 354 in be051eb

	List<String> params = createParams(true);
	addBootstrap(phpModule, params);
	addConfiguration(phpModule, params);
	params.add(LIST_GROUPS_PARAM);
	// list test groups from the current workdir
	params.add("."); // NOI18N
	phpUnit.additionalParameters(params);

@junichi11 I would guess so, if it works correctly without it. You know, this was implemented a very long time ago, so PHPUnit CLI could have changed several times already 🙂

[junichi11]

@tmysik OK, I thought it may have a specific reason :)
Thank you, Tomas!

[tmysik]

It was not working without it, definitely 😁 In ~2010 or so 😁

[tenzap]

I think that if a XML config file is set or if we tick to use just phpunit, the dir should not be set,
Otherwise it should be set to the same dir as the one used when not ticking to use just phpunit.
The '.' just reflects the dir in which to search for tests (see phpunit command line options). If not set it would work according to XML config (either the default one, or the one set by '-c' argument) I suppose.

[junichi11]

It doesn't work without . if a config file is not set. So, I will fix it to not add . if a config file is set. Thanks!

[tenzap]

Why do you use '.' instead of the test dirs? Wouldn't it be more correct to run the command with the test dir as set in the 'properties/testing' window of the project (which is the value used in the phounit command when not ticking 'use just phpunit' option)?

[tmysik]

Maybe, because you can run it on a directory underneath the test one? In such a case, you need the actual directory. Just guessing, I can be wrong, of course.

[tenzap]

For example, with phpunit 10.5

$ vendor/bin/phpunit --help
PHPUnit 10.5.40 by Sebastian Bergmann and contributors.

Usage:
  phpunit [options] <directory|file> ...

From what I understand in source code of phpunit 10.5

• [options] from cli will override the one set in xml config file if it exists (as per the default search location of that file) or if its location is set by the corresponding command line argument.

  See: Registry::init & (new Merger)->merge($cliConfiguration, $xmlConfiguration); which are initially called from Application::run()
  https://github.com/sebastianbergmann/phpunit/blob/10.5.40/src/TextUI/Application.php#L100
  https://github.com/sebastianbergmann/phpunit/blob/main/src/TextUI/Configuration/Registry.php#L109

• <directory|file> (ie. like the . presently used by netbeans when querying groups, or like the test dir that is set when phpunit is invoked without ticking Test project using just phpunit command), if set, will also superseed the one in the xml config file. If not set, it will search for files this the --test-suffix <suffixes> (ie. Only search for test in files with specified suffix(es). Default: Test.php,.phpt) in the current working dir.

  See TestSuiteBuilder::build()
  https://github.com/sebastianbergmann/phpunit/blob/10.5.40/src/TextUI/Application.php#L397
  https://github.com/sebastianbergmann/phpunit/blob/main/src/Framework/TestBuilder.php#L40
  https://github.com/sebastianbergmann/phpunit/blob/main/src/TextUI/Configuration/TestSuiteBuilder.php#L42

So netbeans could work like this:

• if user checks/ticks Test project using just phpunit command, don't append the <directory|file>, the inclusions of the dirs would be determined as per the XML config file if set, or as per the phpunit default. Otherwise, <directory|file> should be the director(y/ies) set in the "project properties / testing / test dirs" window of netbeans.

• if user checks/ticks Ask for test groups before running tests, it should append the <directory|file> depending on whether Test project using just phpunit command is ticked. That's because one would want the list of groups for the tests that phpunit would run based on the way it is configured in netbeans.

[junichi11]

• if user checks/ticks Test project using just phpunit command, don't append the <directory|file>, the inclusions of the dirs would be determined as per the XML config file if set, or as per the phpunit default. Otherwise, <directory|file> should be the director(y/ies) set in the "project properties / testing / test dirs" window of netbeans.
• if user checks/ticks Ask for test groups before running tests, it should append the <directory|file> depending on whether Test project using just phpunit command is ticked. That's because one would want the list of groups for the tests that phpunit would run based on the way it is configured in netbeans.

Can you provide examples(work/don't work) with exact steps to reproduce them?

[tenzap]

isn't the sample I provided sufficient?
It's just about checking/unchecking the boxes in netbeans.
BTW I'm new to phpunit, so that's just supposition and would probably be above my skills to do this in a time-efficient manner.

[junichi11]

Well, Misunderstandings often occur because I'm not you...(We can't often reproduce issues.)
So, please write exact steps.
e.g.

Works fine:

1. check ...
2. ...
3. Run the test file
4. ...

Don't work:

1. check ...
2. ...
3. ...