CTest Test Suite Delimiter prevents running all tests

[Jesseeee]

Brief Issue Summary

When using the test explorer and enabling the test suite delimiter, I see it correctly get split up and I can run the test suites separately, however when I click on the run of the application itself in the explorer, execute "CMake: run tests" or "Test: run all tests" it only runs a single test suite (presumably the first one it finds)

One peculiarity is that when I click on the run button to run them all, it first shows "0/49" which is the correct amount of tests and then only ends up with "15/15" from that first test suite

CMake Tools Diagnostics

No response

Debug Log

No response

Additional Information

No response

Example project & gif:

ctest-project.zip

[Yingzi1234]

@Jesseeee We can't reproduce the problem based on the information you provided, can you provide us with a project that can reproduce the problem? This will go a long way in helping us to investigate this problem, thank you

[Jesseeee]

Hello @Yingzi1234 , hopefully this gif makes the issue more clear, so when just adding the delimiter nothing is seen as run, I click start and would expect all tests to run, and only one test suite is run.

[Yingzi1234]

Hi @Jesseeee , we can't reproduce your issue on latest CMake tools version 1.20.10(pre-release), you can get the details repro steps below, please let us know if it is wrong. Thank you!

ENV:
C/C++: V1.21.6
CMake tools: v1.20.10(pre-release)
VSCode: 1.93.1

Repro steps:

1. Open this folder with VSCode
2. Click "Run tests" button
3. Open settings page and set the "CTest: Test Suite Delimiter" to "."
4. Re-click "Run tests" button
5. Check if all the test suites can be run successfully

Actual result:
All test suites can be run successfully

[Jesseeee]

Hello @Yingzi1234 that's not really what I'm running into, I've made a small example project because I can't share my actual project, but it demonstrates the issue well, it's really when you have what is seen as 2 separate test suites and you try to run them all, be that through clicking that button or executing a command

ctest-project.zip

[Yingzi1234]

@Jesseeee Thanks for the update on this issue, we can reproduce the issue based on the items you provided, you can get the specific information below.
@gcampbell-msft We were able to reproduce this issue on the latest version, and we have added the bug label to this issue, you can get the exact reproduction steps and reproduction information from below, thanks!

ENV:
C/C++: V1.21.6
CMake tools: v1.20.10(pre-release)
VSCode: 1.93.1

Repro steps:

1. Open this folder ctest-project.zip with VSCode
2. Click "Run tests" button
3. Open settings page and set the "CTest: Test Suite Delimiter" to "."
4. Re-click "Run tests" button
5. Check if all the test suites can be run successfully

Actual result:
Only one suite can be run successfully after setting the value of “CTest: Test Suite Delimiter”. When this setting is not set, only all suites can be run successfully.