Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ways to make tools faster for backups? #21

Open
gangamaha opened this issue Aug 17, 2018 · 0 comments
Open

Ways to make tools faster for backups? #21

gangamaha opened this issue Aug 17, 2018 · 0 comments

Comments

@gangamaha
Copy link

Hi,

While I am going to fix the endpoint enhancement PR for tool, we started using the tool with our patches in pre-prod environments and as per the logs it tooks the below times for backing up whisper file from one of our graphite nodes

2018-08-16-16:20:04;INFO;Starting backup of 142910 whisper files
2018-08-16-22:26:58;INFO;Backup complete

So approximately 6hrs in pre-prod and our prod environments are much bigger.
Cronjob uses these params
/usr/local/bin/whisper-backup --bucket $(name_ --prefix /data/graphite/whisper --endpoint $(endpoint) --processes 8 --quiet backup s3

In your experience how many processes you think the tool can use for backups and are there any more configurable options to make backups faster. I was going to do a tar of the same backup dir to have comparison but I believe tar is going to be cpu-intensive. Is it safer to have more processes to make it faster. If you like to know about our hardware please let me know

@gangamaha gangamaha changed the title Ways to make tools faster Ways to make tools faster for backups? Aug 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant