Draft application upgrade to SQLAlchemy, Pytest #346
Conversation
Uses application factory pattern. Moves over to SQLAlchemy from MongoKit.
Removes BerkeleyDB for user authentication, instead using User model to store password. FTP PAM authentication methods will need to be updated to use SQL instead.
Adds a model attribute/DB column for whether CF Standard names are all compliant after Compliance Checker runs. Also updates NCEI archival scripts to utilize this column as part of the criteria prior to determining whether datasets should be archived by NCEI.
Removes a non-functioning dataset removal task which didn't properly get queued up due to Flask application context error. In theory even for removing many files from the web application from a delayed mode dataset the operation should not take much time and deletions are not a common occurrence. If needed can be fixed and re-added later.
Adds feature file for example scenarios with deployments. Tests are currently not implemented for these feature files but will likely be implemented later using pytest-bdd.
…ication_factory_fixes_rebase
Adds BDD testing steps for glider deployment creation, ensuring that a user can create a deployment successfully, have an email sent out to Glider DAC group, and ensure the proper filesystem directory and file structure is created for the dataset.
Allows config to work standalone, allowing some scripts which don't need Flask application context to work properly.
for more information, see https://pre-commit.ci
|
pre-commit.ci autofix |
for more information, see https://pre-commit.ci
|
TODO: re-enable unit, BDD tests for test runner. |
|
@ocefpaf, do you know why the unit tests might not be running under GHA? I adjusted the paths to reflect the altered directory hierarchy. EDIT: resolved, was only running against |
This reverts commit cb23313.
@benjwadams can we use |
|
Sure, I'll revert that. GHA doesn't seem to be properly installing |
I'm not familiar with that docker install steps you are using there. Could it be that the |
.github/workflows/pytest.yml
Outdated
| pull_request: | ||
| branches: | ||
| - main |
There was a problem hiding this comment.
This should allow this to run on all PRs.
| pull_request: | |
| branches: | |
| - main | |
| pull_request: |
There was a problem hiding this comment.
Didn't realize that was the syntax for it. Tests are now running properly locally so I'll see if applying this change works with GHA.
There was a problem hiding this comment.
BTW, I'm not a docker expert but I believe that "docker run" will create a new container and that is why the previous command doesn't persist to the next one. Maybe try to do everything in a single call?
docker run -u root glider-dac-build /bin/bash -c "pip install --no-cache -r /glider-dac/test_requirements.txt && pytest /glider-dac/glider_dac/tests"
There was a problem hiding this comment.
Yeah, this was it. Getting closer now, just need to fix a few paths for the tests.
Draft commit for migration to SQLAlchemy and moves application to application factory pattern for better reusability and testing.
Adds support for BDD testing via
pytest-bdd, see example feature files. Example tests extend #343 by implementing these tests.This is still a work in progress and not complete, but represents a large effort to modernize the DAC architecture.