why AzcopyVersion 10.18.1 doesnot show detailed log even after defining log verbosity with --log-level INFO for the log file

[damison1]

Which version of the AzCopy was used? AzcopyVersion 10.18.1

Note: The version is visible when running AzCopy without any argument

Which platform are you using? (ex: Windows, Mac, Linux) LINUX

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

cat "/path/to/file.txt" | azcopy cp "https://[account].blob.core.windows.net/[container]/[path/to/blob]?[SAS]" --from-to PipeBlob --log-level

What problem was encountered?

No detailed log
$ cat 4fee856d-db1b-3347-6aec-14eb756b7486.log
2024/09/13 09:54:40 AzcopyVersion 10.18.1
2024/09/13 09:54:40 OS-Environment linux
2024/09/13 09:54:40 OS-Architecture amd64
2024/09/13 09:54:40 Log times are in UTC. Local time is 13 Sep 2024 09:54:40
2024/09/13 09:54:40 ISO 8601 START TIME: to copy files that changed before or after this job started, use the parameter --include-before=2024-09-13T09:54:35Z or --include-after=2024-09-13T09:54:35Z

How can we reproduce the problem in the simplest way?

Have you found a mitigation/solution?

When I run higher version 10.25.1, I can see detailed log like PUT operation and all but we don't want to upgrade to higher version and need detailed log with current azcopy version that we are using i.e. 10.18.1

[damison1]

cat "/path/to/file.txt" | azcopy cp "https://[account].blob.core.windows.net/[container]/[path/to/blob]?[SAS]" --from-to PipeBlob --log-level INFO

[vibhansa-msft]

10.18 is quite an older version. Kindly upgrade to the latest version and see if logging is working fine or not.

[damison1]

Hi @vibhansa-msft,

we don't want to upgrade azcopy version and keeping the same version is there any possible way that we can get detailed logs..
Let me know if we can define any environment variable or anything like this to get detailed azcopy log for 10.18.1 azcopy version

Command I run:
cat /path/to/file | azcopy cp "https://storageaccount.blob.core.windows.net/container/datavolume_0000.dat?SAS" --from-to PipeBlob --log-level INFO

Difference in logs
xbrhdph10101:HDB:ph1adm ~/.azcopy 68> cat 6cdcc629-3218-4547-4cca-8444ee306525.log
2024/09/13 09:38:45 AzcopyVersion 10.18.1
2024/09/13 09:38:45 OS-Environment linux
2024/09/13 09:38:45 OS-Architecture amd64
2024/09/13 09:38:45 Log times are in UTC. Local time is 13 Sep 2024 09:38:45
2024/09/13 09:38:45 ISO 8601 START TIME: to copy files that changed before or after this job started, use the parameter --include-before=2024-09-13T09:38:40Z or --include-after=2024-09-13T09:38:40Z

xbrhdph10101:HDB:ph1adm ~/.azcopy 63> cat 01a989f9-23f8-3c4b-5149-d9d25f35a5c2.log
2024/09/13 11:33:23 AzcopyVersion 10.25.1
2024/09/13 11:33:23 OS-Environment linux
2024/09/13 11:33:23 OS-Architecture amd64
2024/09/13 11:33:23 Log times are in UTC. Local time is 13 Sep 2024 11:33:23
2024/09/13 11:33:23 ISO 8601 START TIME: to copy files that changed before or after this job started, use the parameter --include-before=2024-09-13T11:33:18Z or --include-after=2024-09-13T11:33:18Z
2024/09/13 11:33:24 ==> REQUEST/RESPONSE (Try=1/44.8µs, OpTime=63.203037ms) -- RESPONSE STATUS CODE ERROR
PUT https://xbr0201backuptsi0001.blob.core.windows.net/xbr0201/datavolume_0000.dat?blockid=Me734ivkQFNBey2TBdCfPQAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%3D%3D&comp=block&se=2024-09-14T18%3A18%3A21Z&sig=-REDACTED-&sp=rwdlacup&spr=https&srt=sco&ss=b&sv=2021-06-08
Accept: application/xml
Content-Length: 8388608
Content-Type: application/octet-stream
User-Agent: AzCopy/10.25.1 azsdk-go-azblob/v1.3.1 (go1.22.4; linux)
X-Ms-Client-Request-Id: 0c57a556-1252-4d6d-6f96-4c274bd3c354
x-ms-version: 2023-08-03
RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Content-Length: 408
Content-Type: application/xml
Date: Fri, 13 Sep 2024 11:33:23 GMT
Server: Microsoft-HTTPAPI/2.0
X-Ms-Error-Code: AuthenticationFailed
X-Ms-Request-Id: 05939181-801e-0096-62d0-0588b6000000
Response Details: AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. Signature fields not well formed.</

[vibhansa-msft]

Try using log-level to debug instead of info and you may get more detailed logs.

[damison1]

Hi @vibhansa-msft,

Tried but it's also not working

cat "/path/to/file.txt" | azcopy cp "https://[account].blob.core.windows.net/[container]/[path/to/blob]?[SAS]" --from-to PipeBlob --log-level DEBUG

azthdbh20201:HDB:bh2adm ~/.azcopy 55> cat 457a6db6-768d-e643-47fa-279bd84afef3.log
2024/09/16 13:01:52 AzcopyVersion 10.18.1
2024/09/16 13:01:52 OS-Environment linux
2024/09/16 13:01:52 OS-Architecture amd64
2024/09/16 13:01:52 Log times are in UTC. Local time is 16 Sep 2024 13:01:52
2024/09/16 13:01:52 ISO 8601 START TIME: to copy files that changed before or after this job started, use the parameter --include-before=2024-09-16T13:01:47Z or --include-after=2024-09-16T13:01:47Z