You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Which platform are you using? (ex: Windows, Mac, Linux)
Linux (running on a k8s pod having image peterdavehello/azcopy)
What command did you run?
az copy [src blob with sas] [destination blob with sas] --overwrite=prompt --from-to=BlobBlob --s2s-preserve-access-tier=false --check-length=false --include-directory-stub=false --s2s-preserve-blob-tags=true --recursive=true --log-level=ERROR
What problem was encountered?
I am trying to copy across region and subscription, there is one container having lot of folder each containing a single file.
The size of file is small (around 2-3 Kbs) but the number of such folders are huge (more than 4-5M)
also using export AZCOPY_CONCURRENCY_VALUE=2000 as suggested here
It is taking lot of time, any solution to speed up this?
How can we reproduce the problem in the simplest way?
Have you found a mitigation/solution?
The text was updated successfully, but these errors were encountered:
Speeding up AzCopy, especially when dealing with a large number of small files across regions and subscriptions, can be challenging due to the nature of the operation and the limitations of network latency and bandwidth. Can you check if azcopy is able to effectively utilize available bandwidth?
Also, try upgrading to the latest version and see if it improves performance.
Which version of the AzCopy was used?
10.24.0
Which platform are you using? (ex: Windows, Mac, Linux)
Linux (running on a k8s pod having image peterdavehello/azcopy)
What command did you run?
az copy [src blob with sas] [destination blob with sas] --overwrite=prompt --from-to=BlobBlob --s2s-preserve-access-tier=false --check-length=false --include-directory-stub=false --s2s-preserve-blob-tags=true --recursive=true --log-level=ERROR
What problem was encountered?
I am trying to copy across region and subscription, there is one container having lot of folder each containing a single file.
The size of file is small (around 2-3 Kbs) but the number of such folders are huge (more than 4-5M)
also using
export AZCOPY_CONCURRENCY_VALUE=2000
as suggested hereIt is taking lot of time, any solution to speed up this?
How can we reproduce the problem in the simplest way?
Have you found a mitigation/solution?
The text was updated successfully, but these errors were encountered: