Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

batch-scheduler: fix the number of cross-vm links computation #367

Merged
merged 2 commits into from
Jan 25, 2024

Conversation

csegarragonz
Copy link
Collaborator

This PR fixes a bug in the logic to calculate the number of cross-VM links.

Before, we calculated the number of cross-VM links as the product of the number of executors per VM.
This is only true if we have only 2 VMs.

Take for example the two following scheduling decision: [1, 1, 1, 1] and [2, 2].
By doing the product, the former has a number of cross-VMs of 1, and the latter of 4, which means that the planner would prefer the former over the latter.

In fact, the right algorithm to calculate the number of cross-VM links is to, for each host, add all non-local executors in different hosts, and times it for the number of multiple hosts. And divide by two at the end to remove duplicates.

The updated algorithm gives a number of cross-VM links of 6 and 4 respectively, making the planner prefer the latter over the former.

@csegarragonz csegarragonz marked this pull request as ready for review January 25, 2024 13:48
Copy link

codecov bot commented Jan 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d7d8e97) 81.79% compared to head (2dba1af) 81.87%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #367      +/-   ##
==========================================
+ Coverage   81.79%   81.87%   +0.08%     
==========================================
  Files         106      106              
  Lines        7997     8002       +5     
==========================================
+ Hits         6541     6552      +11     
+ Misses       1456     1450       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@csegarragonz csegarragonz merged commit 62c022d into main Jan 25, 2024
20 of 21 checks passed
@csegarragonz csegarragonz deleted the fix-cross-vms branch January 25, 2024 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant