Task: Set up pytest and integrate unit testing into PR workflow

[nlebovits]

Task Description

Configure the repository to use pytest for unit testing and integrate it into the pr_checks_backend workflow. Ensure that every PR impacting the data directory and its subdirectories triggers a unit testing check.

Steps

1. Install pytest as a development dependency.
2. Set up the repository for pytest:
   • Confirm that it discovers and runs all test files named test_*.py.
3. Integrate a unit testing step into the pr_checks_backend workflow:
   • Run pytest for PRs impacting the data directory and its subdirectories.
4. Configure the workflow to:
   • Report success when all tests pass.
   • Report failure when one or more tests fail.

Acceptance Criteria

• pytest is installed and configured in the repository.
• Existing tests run successfully using pytest.
• A unit testing check step using pytest is added to the pr_checks_backend workflow.
• Workflow triggers the unit testing step only for PRs impacting the data directory and its subdirectories.
• PR comments/reporting:
  • Success is reported when all tests pass.
  • Failure is reported when any test fails.

[rmartinsen]

I'd like to take a shot at this one. Seems important for work going forward.

[nlebovits]

@rmartinsen assigned! thanks for tackling--lmk if you run into issues

[rmartinsen]

This ticket is moving along and the tests run fine on the backend. The one issue I'm running into is with test_diff_backup.py. While these are certainly useful, they aren't really classic unit tests in that they aren't isolated from each other and hit external resources (slack, gcp, email etc.).

Would there be any issues with moving this out of tests and creating a simpler suite for this functionality? I'm not sure where it would go. Possibly data_utils?

[nlebovits]

Yeah, I think that's probably fine. Maybe @zigouras has opinions, since he's the one who wrote them? Fwiw, I'm not sure how switching to using hypertables will affect the data diff functionality. That's something I need to puzzle through--would be glad to get some help on it, if that's in your wheelhouse @rmartinsen.