Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent creation of unnecessary multiple pod vms on failure #1842

Closed
kartikjoshi21 opened this issue May 16, 2024 · 2 comments · Fixed by #1854
Closed

Prevent creation of unnecessary multiple pod vms on failure #1842

kartikjoshi21 opened this issue May 16, 2024 · 2 comments · Fixed by #1854
Assignees

Comments

@kartikjoshi21
Copy link
Contributor

The CreateVM api call currently exhibits a continuous retry behavior upon encountering failures during VM creation. This leads to multiple unintended VMs being created in the resource group, even when retries are unnecessary, such as when the image is unavailable. This process persists until available network addresses are exhausted, resulting in unnecessary resource creation and potential network conflicts. This bug is opened to discuss implementing a backoff mechanism to prevent this behavior and ensure more reliable VM creation.

@kartikjoshi21
Copy link
Contributor Author

cc: @mkulke @surajssd @bpradipt

@mkulke
Copy link
Contributor

mkulke commented May 16, 2024

The busy loop in CAA for "create vm" should include an exponential backoff mechanism. Beyond that we don't need any heuristics, IMO. This should be applicable to all cloud providers, busy loops calling CSP apis might lead to rate limits and impact a user's account in unfortunate ways.

@kartikjoshi21 kartikjoshi21 self-assigned this May 27, 2024
kartikjoshi21 added a commit to kartikjoshi21/cloud-api-adaptor that referenced this issue May 27, 2024
Add check to verify if podvm image is present

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
kartikjoshi21 added a commit to kartikjoshi21/cloud-api-adaptor that referenced this issue May 27, 2024
Add check to verify if podvm image is present

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
kartikjoshi21 added a commit to kartikjoshi21/cloud-api-adaptor that referenced this issue Jun 10, 2024
Delete network interface synchronously if vm creation is failed

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
kartikjoshi21 added a commit to kartikjoshi21/cloud-api-adaptor that referenced this issue Jun 10, 2024
Delete network interface synchronously if vm creation is failed

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
kartikjoshi21 added a commit to kartikjoshi21/cloud-api-adaptor that referenced this issue Jun 21, 2024
Delete network interface synchronously if vm creation is failed

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
mkulke pushed a commit to mkulke/cloud-api-adaptor that referenced this issue Jul 4, 2024
Delete network interface synchronously if vm creation is failed

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
Vaibhav-Nazare pushed a commit to Vaibhav-Nazare/cloud-api-adaptor that referenced this issue Jul 6, 2024
Delete network interface synchronously if vm creation is failed

Fixes: confidential-containers#1842
Signed-off-by: Kartik Joshi <[email protected]>
Signed-off-by: Vaibhav-Nazare <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants