Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor benchmark handling of running mode #3964

Open
ajpotts opened this issue Jan 8, 2025 · 2 comments
Open

Refactor benchmark handling of running mode #3964

ajpotts opened this issue Jan 8, 2025 · 2 comments
Assignees

Comments

@ajpotts
Copy link
Contributor

ajpotts commented Jan 8, 2025

Examine the best way to handle running mode for the benchmarks.

@ajpotts
Copy link
Contributor Author

ajpotts commented Jan 8, 2025

@vasslitvinov reported the following concerns:

In the CLASS_SERVER mode, startup_teardown() indeeds starts a server, however that server is unused. Instead the tests connect to the server specified in ARKOUDA_SERVER_HOST. If there isn’t one, the tests hang.
In the GLOBAL_SERVER mode, I do not see that the server starts at all. Again, the tests hang. I think this is not the intended behavior.
benchmark.ini defines ARKOUDA_RUNNING_MODE=CLASS_SERVER whereas benchmark_v2/conftest.py defaults to GLOBAL_SERVER if ARKOUDA_RUNNING_MODE is unset. This is confusing. benchmark.ini also defines the host and the port, with benchmark_v2/conftest.py again providing the defaults, this time consistently.
My suggestion is to remove these settings from benchmark.ini, relying directly on benchmark_v2/conftest.py.

@ajpotts ajpotts self-assigned this Jan 8, 2025
@ajpotts ajpotts added this to the Update Benchmarks milestone Jan 8, 2025
@vasslitvinov
Copy link
Contributor

vasslitvinov commented Jan 8, 2025

Within Python, the information from startup_teardown() to connect() flows as a string with the host name. If two servers are launched, for example, on localhost, then it is easy to use one instead of the other. The issue is more obvious when servers are launched, for example, on two different compute nodes of a bigger system.

I also have a fear that we may accidentally overwrite the serverConnectionInfo file because two server launches use the same location, e.g., when launching a server per class. Not sure this is a real issue though, as hopefully we delete this file after we read it. (???)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants