Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate possible performance degradation following PR #706 #708

Open
lindsay-stevens opened this issue Jun 3, 2024 · 0 comments
Open

Comments

@lindsay-stevens
Copy link
Contributor

Software and hardware versions

pyxform v2.0.3 at 1599d1f, Python 3.10, 3.11, 3.12

Problem description

Changes in #706 seem to have resulted in rather slower test runs than before.

It's not clear whether this is due to slower execution of source or tests (or both).

Pre 706, actions, Post 706 actions runs were roughly:

  • macos: pre 2min 30s, post 2min 55s (+17%)
  • ubuntu pre 55s, post 1min 25s (+55%)
  • windows pre 2min 15s, post 3min 30s (+78%)

Also in the performance test in test_translations.py, test times increased since last run in this commit (Jan 2022):

  • 500 questions: pre 1s, post 3s (3x)
  • 1000 questions: pre 2s, post 10s (5x)
  • 2000 questions: pre 4s, post 30s (7.5x)

Possible reasons (scope of #706):

  • updated the Python versions from 3.7, 3.8, 3.9 to 3.10, 3.11, 3.12
  • updated lxml from 5.1.0 to 5.2.2 (also ruff but that's formatting only)
  • changes from % string formats to f-strings, or other format diffs

The performance test stats have additional factors that could explain the larger diffs:

  • Prior stats updated over 2 years ago (many code changes)
  • Laptop now 2 years crustier
  • Ran on different VM setup:
    • Half the vCPU and RAM allocated
    • Dynamic RAM activated
    • M.2 NVME -> SATA SSD
    • CentOS 7 -> Rocky 9

Steps to reproduce the problem

Run the test suite, note longer execution time as described above.

Expected behavior

Little to no performance change (or improvement??).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant