-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG]: DownloadPipelineArtifact@2 not working ("cannot access ... because it is being used by another process") #20567
Comments
I started running into this today as well but it's on a self hosted agent. Last successful run was yesterday, few runs today mostly on the same pipelines, others are working |
We are also facing the same issue any update on this or any workaround |
Hello, started running into the same issue since yesterday as well |
we are also facing same issue , what is quick fix? |
Also in my company we are facing this issue now more and more. What helps to a certain degree is to use the RetryOnErrorCount feature on those steps. |
Hello guys,
As you can't set a target path with the download step, I added an extra step after to copy (CopyFiles@2) my downloaded artifact to the target path I was using with DownloadPipelineArtifact@2. |
This issue has been affecting some our production builds for over a week now, is there no system in place to revert the active version on these tasks from your end, no A/B testing? Please look into this guys... |
One more thing that I have realized is that we got the error often from files within the workspace folder. And in the pipeline that produced the errors we were not using the workspace - clean: all feature defined in the pipeline yml file. Once we added the workspace clean action there the errors did no more happen. But of course I do not know if cleaning the workspace is an option for you. For us it was. |
@MTomBosch are you on self hosted agents? According to the docs, the workspace on MS hosted ones is always clean https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#workspace |
Yes, we are on self-hosted agents. |
We've taken the trade-off of adding ~10 mins to our build for now by building the same artifact in each of the steps instead of building once and downloading the artifact in the two next steps. Would be great if there was any progress on this! |
We are hitting this same issue with on-premises self-hosted Mac agents. |
I am sorry for the inconvenience. Can we get agent version and also if possible logs for failed pipeline as well? |
@viralpandya there are agent versions (ubuntu 20.04) and both simple and verbose logs in the original post. Is there anything else you require that I can provide? |
New issue checklist
Task name
DownloadPipelineArtifact@2
Task version
2.198.0
Issue Description
Our pipeline has started failing to Download artifacts today for only one of the three jobs we run. All jobs run Ubuntu 20.04 and are running the same version of the task (2.198.0) yet one download fails and the other succeeds.
The jobs are:
Jobs 2 and 3 run in parallel.
The download artifact step always works in job 2 and always fails in job 3 with an error similar to this:
Note that the error still happens when hitting re-running failed jobs (meaning that job 3 is running by itself and no longer in parallel with job 2).
The step is defined in yaml as
Idk if it's relevant but it's weird to me that the logs seems to say that it downloads the artifact three times (may just be automatic retries, or it could genuinely be trying to download three times at once leading to the exception).
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu 20.04
Relevant log output
2024-10-18T01:01:48.1228874Z ##[section]Starting: Download Artifact: web-library
2024-10-18T01:01:48.1239540Z ==============================================================================
2024-10-18T01:01:48.1239729Z Task : Download Pipeline Artifacts
2024-10-18T01:01:48.1239833Z Description : Download build and pipeline artifacts
2024-10-18T01:01:48.1239951Z Version : 2.198.0
2024-10-18T01:01:48.1240059Z Author : Microsoft Corporation
2024-10-18T01:01:48.1240162Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/download-pipeline-artifact
2024-10-18T01:01:48.1240347Z ==============================================================================
2024-10-18T01:01:48.5003431Z Download from the specified build: #29147
2024-10-18T01:01:48.5012319Z Download artifact to: /home/vsts/work/1/s/frontend
2024-10-18T01:01:49.0092512Z Using default max parallelism.
2024-10-18T01:01:49.3305795Z Using default max parallelism.
2024-10-18T01:01:49.3315173Z Max dedup parallelism: 192
2024-10-18T01:01:49.3315468Z DomainId: 0
2024-10-18T01:01:49.5066287Z ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:01:49.5220111Z Hashtype: Dedup1024K
2024-10-18T01:01:49.5468277Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:01:49.5486666Z Minimatch patterns: []
2024-10-18T01:01:52.7566041Z Filtered 212943 files from the Minimatch filters supplied.
2024-10-18T01:01:52.7937816Z Downloaded 0.0 MB out of 1,491.6 MB (0%).
2024-10-18T01:01:57.7209800Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:01:57.7211262Z Minimatch patterns: []
2024-10-18T01:01:57.8023489Z Downloaded 18.2 MB out of 1,491.6 MB (1%).
2024-10-18T01:02:00.4086215Z Filtered 212943 files from the Minimatch filters supplied.
2024-10-18T01:02:00.4356045Z Downloaded 0.0 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:02.8219624Z Downloaded 0.4 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:05.3630500Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:02:05.3631217Z Minimatch patterns: []
2024-10-18T01:02:05.4342391Z Downloaded 7.5 MB out of 1,491.6 MB (1%).
2024-10-18T01:02:07.8217646Z Downloaded 45.4 MB out of 1,491.6 MB (3%).
2024-10-18T01:02:08.8305651Z Filtered 212943 files from the Minimatch filters supplied.
2024-10-18T01:02:08.8600550Z Downloaded 0.0 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:10.4782365Z Downloaded 2.4 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:12.8315497Z Downloaded 17.7 MB out of 1,491.6 MB (1%).
2024-10-18T01:02:13.8563909Z Downloaded 17.7 MB out of 1,491.6 MB (1%).
2024-10-18T01:02:15.4806993Z Downloaded 17.7 MB out of 1,491.6 MB (1%).
2024-10-18T01:02:16.3305178Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:02:16.3305905Z Minimatch patterns: []
2024-10-18T01:02:17.8401104Z Downloaded 55.6 MB out of 1,491.6 MB (4%).
2024-10-18T01:02:18.8531063Z Filtered 212943 files from the Minimatch filters supplied.
2024-10-18T01:02:18.8587161Z Downloaded 0.0 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:18.8789027Z Downloaded 0.0 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:20.4834981Z Downloaded 1.4 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:22.8395907Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:23.8592996Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:23.8800270Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:25.4828261Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:27.8352090Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:28.8554510Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:28.8757898Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:30.4855085Z Downloaded 4.1 MB out of 1,491.6 MB (0%).
2024-10-18T01:02:31.3926950Z ApplicationInsightsTelemetrySender correlated 2 events with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97
2024-10-18T01:02:31.3978588Z ##[error]The process cannot access the file '/home/vsts/work/1/s/frontend/node_modules/js-yaml/bin/js-yaml.js' because it is being used by another process.
2024-10-18T01:02:31.5266723Z ##[section]Finishing: Download Artifact: web-library
Full task logs with system.debug enabled
(Fyi this log only shows the first instance of the exception otherwise it goes past the max limit of this field)
Repro steps
The text was updated successfully, but these errors were encountered: