Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursive glob in gunicorn config can hang augur on large installs #2958

Open
GregSutcliffe opened this issue Dec 10, 2024 · 1 comment
Open
Assignees
Labels
bug Documents unexpected/wrong/buggy behavior

Comments

@GregSutcliffe
Copy link
Contributor

Description:
Having Facade data mounted inside the Augur tree (which is true in Docker) will cause Augur to hang on a large install

How to reproduce:

  1. Have a large install (I have 2Tb of clones)
  2. Mount it inside Augur (I'm using Docker and it's at /augur/facade)
  3. Start Augur - Gunicorn will hang for a long time (I gave up after several hours)

The culprit is here:

https://github.com/chaoss/augur/blob/a31a04cc9e22dbd157a01b3a3a498cd01cc44788/augur/api/gunicorn_conf.py#L22C1-L22C71

reload_extra_files = glob(str(Path.cwd() / '**/*.j2'), recursive=True)

That is a recursive glob, starting at Path.cwd(), looking for .j2 files. If that code hits a large number of files (such as Facade) then it will take an immense amount of time to process. Worse, it may actually find j2 files from outside Augur and load them, which is potentially a security risk.

As far as I can tell, the only j2 files are in augur/templates, so we should drop the recursive glob and just look there.

@GregSutcliffe
Copy link
Contributor Author

Alternatively, perhaps we could just delete it? A quick grep suggests reload_extra_files isn't used anywhere in the codebase

@sgoggins sgoggins added the bug Documents unexpected/wrong/buggy behavior label Dec 17, 2024
@sgoggins sgoggins self-assigned this Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Documents unexpected/wrong/buggy behavior
Projects
Status: No status
Development

No branches or pull requests

2 participants