[BUG] Test Explorer Does Not Refresh Tests

[acgritt]

Describe the Issue

This seems to be related to using a multi-root workspace as when commenting out the folders it works fine. When adding new Unit Tests and building the Test Explorer is not updating to reflect the new tests. I tried clicking the refresh button in Test Explorer and when I choose Run All Tests the tests do not appear to be run. If I reload the window in VS Code then when it loads the Test Explorer all the new tests show up.

Steps To Reproduce

1. Workspace with Multi-Root Folder
2. Unit Tests in Solution
3. View Test Explorer
4. Add Unit Tests

Expected Behavior

Adding new unit tests to the loaded solution should cause the Test Explorer to refresh and show the new tests.

Environment Information

No response

[f3]

Git branch switching appears to suffer from the same issue (e.g., the "switched-to" branch includes new/removed unit tests).

[timheuer]

@acgritt do you perform a build after adding the new tests?

[SelfRef]

Can confirm on my system. Multi-root (with .NET project in a second folder), I'm adding a new xUnit test file, then build/rebuild/cleanbuild/build+run/restart VSC and no matter what I do it doesn't show in Test Explorer. So I went to Visual Studio an it's shown right away (no build or anything). I run it. After that it's also shown in VS Code.

Is it like about some kind of tests cache that's only "big" VS can create?

VSCode 1.80.1, C# 2.0.249 (pre-rel), C# Dev Kit 0.3.2 (pre-rel)

[acgritt]

@timheuer Yes, I do a build after adding the new tests. Nothing shows up in the explorer. Same happens if I remove or rename a test. The explorer doesn't change. When I "Reload Window" in VS Code or close and relaunch then the updated list of tests shows after I run the build again, but it is not refreshing the list without a heavier hand on the refresh process.

[ProTip]

I started running into this after re-structuring my solution a bit. It was showing duplicate tests, tests I removed, and was using the wrong path to look for the .dll.

A workaround was to remove the test store. If you check the solution explorer extension output it will tell you the location on startup:

~/.config/Code/User/workspaceStorage/1b7d19f80d33b5a0409cb3dab5cd9aa2/ms-dotnettools.csdevkit

In there I deleted the testStore folder and after re-starting VSCode everything came right.

[jpringle1]

A workaround was to remove the test store.

I am also having this problem, and can confirm that solution does work to a degree.
Unfortunately, I have to do this every time I create new test, which is not ideal.

Update:

The issue has now resolved itself, and now all i have to do to get the tests to show up is build the project. No idea what fixed this or how to reproduce the problem.

[ivanjx]

i always need to use dotnet watch build on my test projects to make the tests appear correctly on the tests pane.

[joshuabrink]

I managed to resolve it by removing the bin path from my files.watcherExclude settings and reloading my vscode.

[LynxWall]

I'm experiencing similar issues when changing the build target. Duplicate tests, errors trying to load invalid assembly, etc. The only way to resolve these issues is to delete the testStore under workspaceStorage.

@ProTip Thank you for posting that workaround, was getting very frustrated with bad tests until I found this.

[peterwald]

@ivanjx @LynxWall @acgritt @jpringle1 @ProTip

Can you give me an idea of what the structure of your multi-root workspace looks like? Do you have all of your C# code & tests in one root, and other languages in the other roots? (i.e. javascript, web assets, etc...)

Or are you creating a solution that somehow spans C# projects across multiple workspace roots?

[acgritt]

@peterwald
Generally I have the following:

• Root Folder
  • Workspace File
  • .gitignore
  • etc
  • src
    • Solution File
    • Code
  • docs
    • documentation files

The workspace is set to display the src folder as Source and docs as Documentation at the root. Sometimes, when working with a dotnet angular project I also have Angular as a folder that points to the client folder inside the API project. When I click the refresh button to refresh the tests the status line that appears just blips and the logs show nothing as having been done. It isn't until I reload the VS Code window that the new tests show up.

[peterwald]

A fix has been merged and will be available in the next pre-release build.

[tatejones]

I am assuming this fix has been released in a pre-release build. I am running C# Dev Kit v0.5.150 and I am still experiencing the following issues.

1. New tests not detected when clicking the refresh
2. Changes to test code are not reflected in the test outcome. The old test logic continues to be executed.

Executing
dotnet watch test --project /csproject
works fine.
Even after that the test explorer still continues to execute the old code. How is that possible? Only after performing a dotnet clean will the test explorer will start executing the correct code.

The layout is:
Root:

• Workspace file
• Solution file (references projects)
• Project A
  • cs project file
  • source files
• Project B
  • cs project file
  • source files

[tatejones]

It looks like the test explorer is creating binaries in bin/Debug/net6.0//...
But the VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22) is looking for binaries in the bin/x64/Debug/net6.0/

bin/Debug vs bin/x64/Debug

[peterwald]

@tatejones C# Dev Kit v0.5.150 should include the fix I referenced above.

The test explorer invokes the build command provided by the solution/project, it doesn't do anything special on its own. This should be the same as right clicking the Project or Solution node in the Solution Explorer section of the file pane and selecting build from there.

When you say...

VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22) is looking for binaries in the bin/x64/Debug/net6.0/

What does that mean? Which VSTest Adapter? Are you looking at what dotnet test does on the command line?

[tatejones]

When I perform a build at the solution or project (right click in solution explorer) level I see binaries been created with the platform /bin/x64/Debug/net6.0/... When I execute a test using the test explorer the build doesn’t have the platform. /bin/Debug/net6.0/...

What does that mean? Which VSTest Adapter? Are you looking at what dotnet test does on the command line?

From the Test Explorer, not via the command line. The output - Executing task: dotnet build /home/tate/code/<project>.csproj /property:GenerateFullPaths=true /consoleloggerparameters:NoSummary MSBuild version 17.4.8+6918b863a for .NET MSBuild version 17.4.8+6918b863a for .NET Determining projects to restore... All projects are up-to-date for restore. Utilities -> /home/tate/code/<Solution>/<Project>.Utilities/bin/Debug/net6.0/Utilities.dll .... Any changes made are not detected for the tests to pick up. Strange

…

On 7 Oct 2023, at 1:19 am, Peter Waldschmidt ***@***.***> wrote: @tatejones <https://github.com/tatejones> C# Dev Kit v0.5.150 should include the fix I referenced above. The test explorer invokes the build command provided by the solution/project, it doesn't do anything special on its own. This should be the same as right clicking the Project or Solution node in the Solution Explorer section of the file pane and selecting build from there. When you say... VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22) is looking for binaries in the bin/x64/Debug/net6.0/ What does that mean? Which VSTest Adapter? Are you looking at what dotnet test does on the command line? — Reply to this email directly, view it on GitHub <#252 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADONC2I5RSQ4NMZ4BXZCW3X6AORZAVCNFSM6AAAAAA2I5I3LKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQHA4DGMBYGI>. You are receiving this because you were mentioned.

[tatejones]

When it is discovering tests the log indicates it is searching through the binaries that have a platform in the path.

In Summary:

1. The build triggered via the Test Explorer doesn't have the platform in the binary path
2. The build triggered via the Solution Explorer does have the platform in the path
3. The VSTest Adapter is using the builds generated by the Solution Explorer build.

Hence the VSTest Adapter never see a new build unless a build is manually triggered by the Solution Explorer or via the command line that includes the platform. The trigger that builds the binaries from the Test Explorer needs to use the same build instructions that of the Solution Explorer.

[thegoatherder]

FWIW I am seeing the same issue on a node.js project. New tests incoming from a git pull are not detected in Test Explorer, nor do they have the test controls (run, debug) in the gutter area.

[peterwald]

FWIW I am seeing the same issue on a node.js project. New tests incoming from a git pull are not detected in Test Explorer, nor do they have the test controls (run, debug) in the gutter area.

C# Dev Kit only supports C# projects. A node.js extension for VS Code should be able to help with node.js tests.

[tatejones]

The solution for me was to introduce another platform. Originally I had only x64 for Release and Debug. The test adapter was always specifying a platform (project level?), hence was missing the build that didn't have a platform. Once I added another platform like "AnyCPU" the build finally added the platform and then the test adapter picked it up.

According to the docs.
"If a project configuration doesn't specify a platform or specifies just one platform, then a solution configuration whose name matches that of the new project configuration is either found or added. The default name of this solution configuration does not include a platform name; it takes the form < project configuration name >."

"If a project supports multiple platforms, a solution configuration is either found or added for each supported platform. The name of each solution configuration includes both the project configuration name and the platform name, and has the form < project configuration name > < platform name >."

Understand build configurations

[QuintinWillison]

Refresh button does nothing for me. I have to restart VS Code to get new tests, every time I add a new one. 😞
We have a single root solution, with multiple projects, pretty standard.
Visual Studio handles it fine.
C# .NET 7.

[tengl]

I have the same issue. I need to do CTRL+SHIFT+P and run Developer: Reload Window in order to get new tests in the Test Explorer.

I've configured the outputs from build to end up in the artifacts directory using the Directory.Build.props file in the root directory.

<Project>
  <PropertyGroup>
    <BaseOutputPath>$(MSBuildThisFileDirectory)artifacts\$(MSBuildProjectName)\bin</BaseOutputPath>
    <BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)artifacts\$(MSBuildProjectName)\obj</BaseIntermediateOutputPath>
  </PropertyGroup>
</Project>

I have two solution (.sln) files in the root directory, one of them is default in .vscode/settings.json.

{
  "dotnet.defaultSolution": "Main.sln"
}

All code is in the src directory and tests in the tests directory. All projects are .Net 8. OS is Pop_OS!

About VS Code

Version: 1.85.1
Commit: 0ee08df0cf4527e40edc9aa28f4b5bd38bbff2b2
Date: 2023-12-13T09:47:11.635Z
Electron: 25.9.7
ElectronBuildId: 25551756
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Linux x64 6.6.6-76060606-generic

[tengl]

The solution for me was to introduce another platform. Originally I had only x64 for Release and Debug. The test adapter was always specifying a platform (project level?), hence was missing the build that didn't have a platform. Once I added another platform like "AnyCPU" the build finally added the platform and then the test adapter picked it up.

According to the docs. "If a project configuration doesn't specify a platform or specifies just one platform, then a solution configuration whose name matches that of the new project configuration is either found or added. The default name of this solution configuration does not include a platform name; it takes the form < project configuration name >."

"If a project supports multiple platforms, a solution configuration is either found or added for each supported platform. The name of each solution configuration includes both the project configuration name and the platform name, and has the form < project configuration name > < platform name >."

Understand build configurations

I can also confirm that changing Directory.Build.props file to include AnyCPU in the path fixes the issue, the refresh button works now.

<Project>
  <PropertyGroup>
    <BaseOutputPath>$(MSBuildThisFileDirectory)artifacts\$(MSBuildProjectName)\bin\AnyCPU</BaseOutputPath>
    <BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)artifacts\$(MSBuildProjectName)\obj\AnyCPU</BaseIntermediateOutputPath>
  </PropertyGroup>
</Project>

None of my projects targets any specific platform. Here is an example from a .csproj file.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <InternalsVisibleTo Include="$(AssemblyName).Tests" />
  </ItemGroup>

</Project>

[hutch120]

Took a while to figure this out, but one of my devs picked a specific csproj file when the solution first loaded in vscode. From then on it would only load tests for that project. We were able to fix this by deleting the cache output in the C# Dev Kit log. Although thinking about it, we might have been able to fix this by adding the additional projects to the solution explorer.

So, the dirty fix is to find the path to the C# Dev Kit cache in the in VSCode Output container/tab panel and delete the cache.

Looks something like this:
c:\Users\[username]\AppData\Roaming\Code\User\workspaceStorage\[GUID]\ms-dotnettools.csdevkit

Then, reload VSCode, and when the popup to select the project appears, select the solution instead of a specific project.

The Test explorer should then try to find tests in all the projects.

References:

• https://code.visualstudio.com/docs/csharp/project-management

[chrisdel101]

I've been using tests all day and they just stopped working. This has happened before many times. Restarting, deleting bin, rebuilding, cleaning...nothing works, Strangely enough I tried Window Reload (read about it above) and this fixed it, at least it fixed it right now. This issue is very derailing.

[AbhitejJohn]

Thanks for sharing your feedback folks. I'm re-opening this issue so we can take another look at why this is as flaky. Based on the comments here, it might be tied to this issue: #996 , we've been tracking with the filewatcher API in VSCode. The workaround there was to delete the bin+obj folders on all the projects, reopen vscode and trigger a build. That's a little on the heavy-handed side compared to the window reload workaround posted earlier. However tagging @ocallesp on this so we can investigate what is happening here and verify if it is the same problem as the one linked.

If anyone has a sharable repro that could help us track this down, that would be very much appreciated as well.