falconeri_common::storage::s3 spams the log

[seamusabshere]

[2020-11-30T22:14:49Z TRACE falconeri_common::storage::s3] uploading /pfs/out/ to s3://mybucket/myfolder/
Nov 30 17:14:55 worker falconeri-production[myjob: Completed 256.0 KiB/272.1 MiB (858.2 KiB/s) with 1 file(s) remaining Completed 512.0 KiB/272.1 MiB (1.6 MiB/s) with 1 file(s) remaining Completed 768.0 KiB/272.1 MiB (2.4 MiB/s) with 1 file(s) remaining Completed 1.0 MiB/272.1 MiB (3.1 MiB/s) with 1 file(s) remaining Completed 1.2 MiB/272.1 MiB (3.8 MiB/s) with 1 file(s) remaining Completed 1.5 MiB/272.1 MiB (4.4 MiB/s) with 1 file(s) remaining Completed 1.8 MiB/272.1 MiB (5.0 MiB/s) with 1 file(s) remaining Completed 2.0 MiB/272.1 MiB (5.7 MiB/s) with 1 file(s) remaining Completed 2.2 MiB/272.1 MiB (6.3 MiB/s) with 1 file(s) remaining Completed 2.5 MiB/272.1 MiB (6.9 MiB/s) with 1 file(s) remaining Completed 2.8 MiB/272.1 MiB (7.6 MiB/s) with 1 file(s) remaining Completed 3.0 MiB/272.1 MiB (8.2 MiB/s) with 1 file(s) remaining Completed 3.2 MiB/272.1 MiB (8.8 MiB/s) with 1 file(s) remaining Completed 3.5 MiB/272.1 MiB (9.5 MiB/s) with 1 file(s) remaining Completed 3.8 MiB/272.1 MiB (10.1 MiB/s) with 1 file(s) remaining Completed 4.0 MiB/272.1 MiB (10.7 MiB/s) with 1 file(s) remaining Completed 4.2 MiB/272.1 MiB (10.9 MiB/s) with 1 file(s) remaining Completed 4.5 MiB/272.1 MiB (11.4 MiB/s) with 1 file(s) remaining Completed 4.8 MiB/272.1 MiB (12.0 MiB/s) with 1 file(s) remaining Completed 5.0 MiB/272.1 MiB (12.6 MiB/s) with 1 file(s) remaining Completed 5.2 MiB/272.1 MiB (13.2 MiB/s) with 1 file(s) remaining Completed 5.5 MiB/272.1 MiB (13.5 MiB/s) with 1 file(s) remaining Completed 5.8 MiB/272.1 MiB (14.0 MiB/s) with 1 file(s) remaining Completed 6.0 MiB/272.1 MiB (14.6 MiB/s) with 1 file(s) remaining Completed 6.2 MiB/272.1 MiB (15.1 MiB/s) with 1 file(s) remaining Completed 6.5 MiB/272.1 MiB (15.7 MiB/s) with 1 file(s) remaining Completed 6.8 MiB/272.1 MiB (16.1 MiB/s) with 1 file(s) remaining Completed 7.0 MiB/272.1 MiB (16.7 MiB/s) with 1 file(s) remaining Completed 7.2 MiB/272.1 MiB (17.1 MiB/s) with 1 file(s) remaining Completed 7.5 MiB/272.1 MiB (17.5 MiB/s) with 1 file(s) remaining Completed 7.8 MiB/272.1 MiB (18.1 MiB/s) with 1 file(s) remaining Completed 8.0 MiB/272.1 MiB (18.6 MiB/s) with 1 file(s) remaining Completed 8.2 MiB/272.1 MiB (19.1 MiB/s) with 1 file(s) remaining Completed 8.5 MiB/272.1 MiB (19.6 MiB/s) with 1 file(s) remaining Completed 8.8 MiB/272.1 MiB (20.2 MiB/s) with 1 file(s) remaining Completed 9.0 MiB/272.1 MiB (20.6 MiB/s) with 1 file(s) remaining Completed 9.2 MiB/272.1 MiB (21.1 MiB/s) with 1 file(s) remaining Completed 9.5 MiB/272.1 MiB (21.7 MiB/s) with 1 file(s) remaining Completed 9.8 MiB/272.1 MiB (22.1 MiB/s) with 1 file(s) remaining Completed 10.0 MiB/272.1 MiB (22.5 MiB/s) with 1 file(s) remaining Completed 10.2 MiB/272.1 MiB (23.0 MiB/s) with 1 file(s) remaining Completed 10.5 MiB/272.1 MiB (23.3 MiB/s) with 1 file(s) remaining Completed 10.8 MiB/272.1 MiB (23.8 MiB/s) with 1 file(s) remaining Completed 11.0 MiB/272.1 MiB (24.3 MiB/s) with 1 file(s) remaining Completed 11.2 MiB/272.1 MiB (24.6 MiB/s) with 1 file(s) remaining Completed 11.5 MiB/272.1 MiB (25.1 MiB/s) with 1 file(s) remaining Completed 11.8 MiB/272.1 MiB (25.6 MiB/s) with 1 file(s) remaining Completed 12.0 MiB/272.1 MiB (26.1 MiB/s) with 1 file(s) remaining Completed 12.2 MiB/272.1 MiB (26.5 MiB/s) with 1 file(s) remaining Completed 12.5 MiB/272.1 MiB (26.8 MiB/s) with 1 file(s) remaining Completed 12.8 MiB/272.1 MiB (27.3 MiB/s) with 1 file(s) remaining Completed 13.0 MiB/272.1 MiB (27.8 MiB/s) with 1 file(s) remaining Completed 13.2 MiB/272.1 MiB (28.0 MiB/s) with 1 file(s) remaining Completed

[emk]

@seamusabshere This is probably just a matter of passing better flags to aws s3 for now.