Regression: `dotnet tool restore` with a `dotnet-tools.json` is broken in .NET 8 RC 2 #35989

alexrp · 2023-10-10T22:41:27Z

$ git clone https://github.com/vezel-dev/celerity.git
$ cd celerity
$ sed -i 's/8.0.100-rc.1.23455.8/8.0.100-rc.2.23502.2/' global.json
$ sed -i 's/7.0.442301/3.0.0-preview9.19454.1/' dotnet-tools.json
$ dotnet tool restore
Skipping NuGet package signature verification.
Unhandled exception: System.AggregateException: One or more errors occurred. (The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.) (The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.) (The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.) (The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.)
 ---> System.IO.IOException: The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at NuGet.ProjectModel.LockFileFormat.Write(String filePath, LockFile lockFile)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.CreateAssetFile(PackageId packageId, NuGetVersion version, DirectoryPath packagesRootPath, DirectoryPath assetFileDirectory, String runtimeJsonGraph, String targetFramework)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.<>c__DisplayClass8_0.<InstallPackage>b__0()
   at Microsoft.DotNet.Cli.TransactionalAction.Run[T](Func`1 action, Action commit, Action rollback)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.InstallPackages(ToolManifestPackage package, Nullable`1 configFile)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.<>c__DisplayClass10_0.<Execute>b__0(ToolManifestPackage package)
   at System.Linq.Parallel.SelectQueryOperator`2.SelectQueryOperatorResults.GetElement(Int32 index)
   at System.Linq.Parallel.QueryResults`1.get_Item(Int32 index)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.StopAndGoSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
   at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
   at System.Linq.Parallel.SpoolingTask.SpoolStopAndGo[TInputOutput,TIgnoreKey](QueryTaskGroupState groupState, PartitionedStream`2 partitions, SynchronousChannel`1[] channels, TaskScheduler taskScheduler)
   at System.Linq.Parallel.DefaultMergeHelper`2.System.Linq.Parallel.IMergeHelper<TInputOutput>.Execute()
   at System.Linq.Parallel.MergeExecutor`1.Execute()
   at System.Linq.Parallel.MergeExecutor`1.Execute[TKey](PartitionedStream`2 partitions, Boolean ignoreOutput, ParallelMergeOptions options, TaskScheduler taskScheduler, Boolean isOrdered, CancellationState cancellationState, Int32 queryId)
   at System.Linq.Parallel.PartitionedStreamMerger`1.Receive[TKey](PartitionedStream`2 partitionedStream)
   at System.Linq.Parallel.UnaryQueryOperator`2.UnaryQueryOperatorResults.GivePartitionedStream(IPartitionedStreamRecipient`1 recipient)
   at System.Linq.Parallel.QueryOperator`1.ExecuteAndGetResultsAsArray()
   at System.Linq.ParallelEnumerable.ToArray[TSource](ParallelQuery`1 source)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.Execute()
   at System.CommandLine.Invocation.InvocationPipeline.Invoke(ParseResult parseResult)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, TimeSpan startupTime, ITelemetry telemetryClient)
 ---> (Inner Exception #1) System.IO.IOException: The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at NuGet.ProjectModel.LockFileFormat.Write(String filePath, LockFile lockFile)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.CreateAssetFile(PackageId packageId, NuGetVersion version, DirectoryPath packagesRootPath, DirectoryPath assetFileDirectory, String runtimeJsonGraph, String targetFramework)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.<>c__DisplayClass8_0.<InstallPackage>b__0()
   at Microsoft.DotNet.Cli.TransactionalAction.Run[T](Func`1 action, Action commit, Action rollback)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.InstallPackages(ToolManifestPackage package, Nullable`1 configFile)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.<>c__DisplayClass10_0.<Execute>b__0(ToolManifestPackage package)
   at System.Linq.Parallel.SelectQueryOperator`2.SelectQueryOperatorResults.GetElement(Int32 index)
   at System.Linq.Parallel.QueryResults`1.get_Item(Int32 index)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.StopAndGoSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)<---

 ---> (Inner Exception #2) System.IO.IOException: The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at NuGet.ProjectModel.LockFileFormat.Write(String filePath, LockFile lockFile)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.CreateAssetFile(PackageId packageId, NuGetVersion version, DirectoryPath packagesRootPath, DirectoryPath assetFileDirectory, String runtimeJsonGraph, String targetFramework)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.<>c__DisplayClass8_0.<InstallPackage>b__0()
   at Microsoft.DotNet.Cli.TransactionalAction.Run[T](Func`1 action, Action commit, Action rollback)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.InstallPackages(ToolManifestPackage package, Nullable`1 configFile)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.<>c__DisplayClass10_0.<Execute>b__0(ToolManifestPackage package)
   at System.Linq.Parallel.SelectQueryOperator`2.SelectQueryOperatorResults.GetElement(Int32 index)
   at System.Linq.Parallel.QueryResults`1.get_Item(Int32 index)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.StopAndGoSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)<---

 ---> (Inner Exception #3) System.IO.IOException: The process cannot access the file 'C:\Users\Alex Rønne Petersen\AppData\Local\Temp\cbdfc8b6-d7cf-4926-8cac-e05a4d1adbe4\project.assets.json' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at NuGet.ProjectModel.LockFileFormat.Write(String filePath, LockFile lockFile)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.CreateAssetFile(PackageId packageId, NuGetVersion version, DirectoryPath packagesRootPath, DirectoryPath assetFileDirectory, String runtimeJsonGraph, String targetFramework)
   at Microsoft.DotNet.Cli.ToolPackage.ToolPackageDownloader.<>c__DisplayClass8_0.<InstallPackage>b__0()
   at Microsoft.DotNet.Cli.TransactionalAction.Run[T](Func`1 action, Action commit, Action rollback)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.InstallPackages(ToolManifestPackage package, Nullable`1 configFile)
   at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.<>c__DisplayClass10_0.<Execute>b__0(ToolManifestPackage package)
   at System.Linq.Parallel.SelectQueryOperator`2.SelectQueryOperatorResults.GetElement(Int32 index)
   at System.Linq.Parallel.QueryResults`1.get_Item(Int32 index)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.StopAndGoSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)<---

Notes:

3.0.0-preview9.19454.1 is arbitrary; pick any version that isn't already installed on the machine.
--disable-parallel and --no-cache make no difference.
I have no idea what is up with the signature verification. I haven't disabled it myself.
This regression is a hard blocker for upgrading from .NET 8 RC 1 in most of my projects, as I rely on local tools during the build. Seems to me that this should be a .NET 8 release blocker.

The text was updated successfully, but these errors were encountered:

KalleOlaviNiemitalo · 2023-10-11T04:07:45Z

It has been fixed for .NET SDK 8.0.100 in #35884.

The RC 2 known issue doc PR dotnet/core#8799 hasn't been merged yet.

alexrp · 2023-10-11T04:27:58Z

Huh, ok. I guess I'm going to skip RC 2. Those suggested workarounds really don't scale to non-trivial projects, and judging by the linked issues, it seems even the different .NET teams gave up trying to work around the bug.

Honestly a bit baffling that RC 2 shipped with such a major known regression. 😕

KalleOlaviNiemitalo · 2023-10-11T05:47:19Z

The ToolRestoreCommandTests.ItRestoresMultipleTools test is not on the release/8.0.2xx and main branches yet. I guess the fix isn't on those branches either. Will those now be tracked in this issue, or is there a different issue for that?

molinch · 2023-10-11T12:38:47Z

I used Docker image 8.0.100-rc.2 but still got issues with dotnet tool restore. Is it really fixed already?
If yes, which image contains the fix?

dsplaisted · 2023-10-11T16:25:08Z

The ToolRestoreCommandTests.ItRestoresMultipleTools test is not on the release/8.0.2xx and main branches yet. I guess the fix isn't on those branches either. Will those now be tracked in this issue, or is there a different issue for that?

Here's the PR merging these changes to 8.0.2xx (and it will flow to main at some point after that PR is merged): #35914

I used Docker image 8.0.100-rc.2 but still got issues with dotnet tool restore. Is it really fixed already? If yes, which image contains the fix?

The issue is fixed in the code but there isn't a release or docker image with the fix. That will come with the 8.0.100 GA release.

KalleOlaviNiemitalo · 2023-10-11T21:27:23Z

From the workaround in the known issue PR dotnet/core#8811:

If you are using .NET tools in CI where you can't easily modify the tools config file between multiple tool restores, you can create a separate dotnet-tools.json config file for each .NET tool in a separate subfolder. Then run dotnet tool restore in each subfolder.

Do these per-subfolder dotnet tool restore commands have to be run serially, or is it safe to run them in parallel?

JL03-Yue · 2023-10-11T22:52:41Z

Do these per-subfolder dotnet tool restore commands have to be run serially, or is it safe to run them in parallel?

I believe it's safe to run in parallel.

mattleibow · 2023-10-12T17:23:35Z

I added this to our startup script:

$tools = Get-Content ".config/dotnet-tools.json" | ConvertFrom-Json
foreach ($tool in $tools.tools.PsObject.Properties) {
    & dotnet tool install $tool.Name --version $tool.Value.version
}

JL03-Yue · 2023-10-20T19:37:56Z

This is targeted to be fixed in .NET GA.

dotnet-issue-labeler bot added Area-Tools untriaged Request triage from a team member labels Oct 10, 2023

TheAngryByrd mentioned this issue Oct 12, 2023

Dotnet 8 builds ionide/FsAutoComplete#1175

Merged

mattleibow mentioned this issue Oct 12, 2023

Revert "Work around the dotnet tool restore issues" dotnet/maui#17983

Merged

lauxjpn mentioned this issue Oct 14, 2023

Upgrade to EF Core 8.0.0-rc.2 PomeloFoundation/Pomelo.EntityFrameworkCore.MySql#1800

Merged

manfred-brands mentioned this issue Oct 19, 2023

.NET 8 support nunit/nunit#4511

Merged

JL03-Yue closed this as completed Oct 20, 2023

belav mentioned this issue Nov 9, 2023

dotnet tool restore often failing on actions belav/csharpier#996

Closed

This was referenced Nov 11, 2023

Reformatting a list expression with if/else changes the list expression fsprojects/fantomas#2972

Closed

Dev container fsprojects/fantomas#2980

Merged

shocklateboy92 mentioned this issue Nov 12, 2023

Sort global System usings before other global usings belav/csharpier#1004

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regression: `dotnet tool restore` with a `dotnet-tools.json` is broken in .NET 8 RC 2 #35989

Regression: `dotnet tool restore` with a `dotnet-tools.json` is broken in .NET 8 RC 2 #35989

alexrp commented Oct 10, 2023 •

edited

Loading

KalleOlaviNiemitalo commented Oct 11, 2023

alexrp commented Oct 11, 2023

KalleOlaviNiemitalo commented Oct 11, 2023

molinch commented Oct 11, 2023 •

edited

Loading

dsplaisted commented Oct 11, 2023

KalleOlaviNiemitalo commented Oct 11, 2023 •

edited

Loading

JL03-Yue commented Oct 11, 2023 •

edited

Loading

mattleibow commented Oct 12, 2023

JL03-Yue commented Oct 20, 2023

Regression: dotnet tool restore with a dotnet-tools.json is broken in .NET 8 RC 2 #35989

Regression: dotnet tool restore with a dotnet-tools.json is broken in .NET 8 RC 2 #35989

Comments

alexrp commented Oct 10, 2023 • edited Loading

KalleOlaviNiemitalo commented Oct 11, 2023

alexrp commented Oct 11, 2023

KalleOlaviNiemitalo commented Oct 11, 2023

molinch commented Oct 11, 2023 • edited Loading

dsplaisted commented Oct 11, 2023

KalleOlaviNiemitalo commented Oct 11, 2023 • edited Loading

JL03-Yue commented Oct 11, 2023 • edited Loading

mattleibow commented Oct 12, 2023

JL03-Yue commented Oct 20, 2023

Regression: `dotnet tool restore` with a `dotnet-tools.json` is broken in .NET 8 RC 2 #35989

Regression: `dotnet tool restore` with a `dotnet-tools.json` is broken in .NET 8 RC 2 #35989

alexrp commented Oct 10, 2023 •

edited

Loading

molinch commented Oct 11, 2023 •

edited

Loading

KalleOlaviNiemitalo commented Oct 11, 2023 •

edited

Loading

JL03-Yue commented Oct 11, 2023 •

edited

Loading