Better progress logger for stor #79
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace the existing progress logger with tqdm, a library that makes elegant progress bars that calculate rate of transfer, time remaining, and also look nice on the CLI (i.e., stays on one line and doesn't gum up the entire works). A future enhancement would be to make the progress logger configurable, but I figure for now just showing a better one is a bonus. How this was tested: 1. Tested stor cp -r and rm -r with various combinatiosn 2. Ensure passes all unit tests. Sem-Ver: feature
| @@ -74,6 +74,7 @@ | |||
| """ | |||
| import argparse | |||
| import copy | |||
| from collections import OrderedDict | |||
There was a problem hiding this comment.
AFAIK this import is unused
| logger = progress_logger | ||
|
|
||
| def add_result(self, result): | ||
| self.update_progress(num_objects=1, addl_bytes=os.path.getszie(result['dest'])) |
There was a problem hiding this comment.
Is this a typo? getszie
There was a problem hiding this comment.
hmmm... maybe I didn't let the s3 version bubble up long enough..
|
This functionality is semi-covered by the integration tests (which call copytree and friends). I think I can test it by checking for the calls to update_progress and/or grabbing the progress logger after running. |
|
A few typos to fix. Overall idea SGTM. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace the existing progress logger with tqdm, a library that makes
elegant progress bars that calculate rate of transfer, time remaining,
and also look nice on the CLI (i.e., stays on one line and doesn't gum
up the entire works). A future enhancement would be to make the progress
logger configurable, but I figure for now just showing a better one is a
bonus.
How this was tested:
I'm calling this a feature break: it does not break any existing APIs and the progress logger was never a public method exposed to users.
cc @kyleabeauchamp @wesleykendall @pkaleta @krhaas - if any of you have time to review that'd be awesome