Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed autoscaling bug and updated hpa sample #270

Closed
wants to merge 3 commits into from

Conversation

asteurer
Copy link
Contributor

No description provided.

@asteurer
Copy link
Contributor Author

@bacongobbler @calebschoepp I removed the requirements for a minimum pod count, but I'm running into an issue with spec.replicas. When I don't set the replica field, all resources are created; however, I can't port-forward the service, which is how I'm testing the scaling capabilities. Any thoughts?

Copy link
Contributor

@michelleN michelleN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asteurer is there an issue or thread I can look at to understand the bug?

@michelleN
Copy link
Contributor

@asteurer is there an issue or thread I can look at to understand the bug?

nvm found #24 !

@@ -5,7 +5,7 @@ metadata:
spec:
image: ghcr.io/spinkube/spin-operator/cpu-load-gen:20240311-163328-g1121986
executor: containerd-shim-spin
enableAutoscaling: true
replicas: 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bacongobbler says:

Shouldn't this be removed if both minReplicas and maxReplicas are set?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

} else {
replicas = ptr(app.Spec.Replicas)
}
replicas := app.Spec.Replicas
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bacongobbler says:

related to the above comment RE: min/maxReplicas, shouldn't this be nil if either of those are set?

If only minReplicas is set (is that valid?), should this be set to minReplicas as a starting value?
If only maxReplicas is set, should this be set to 1 (or possibly nil, assuming we can scale to zero)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -99,6 +91,9 @@ type SpinAppStatus struct {

// Represents the current number of active replicas on the application deployment.
ReadyReplicas int32 `json:"readyReplicas"`

// Selector for the underlying pods
Selector string `json:"selector"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@endocrimes would we have to make this optional to support runtimes where this selector is meaningless? Or is setting it to the empty string sufficient.

@calebschoepp
Copy link
Contributor

@bacongobbler @calebschoepp I removed the requirements for a minimum pod count, but I'm running into an issue with spec.replicas. When I don't set the replica field, all resources are created; however, I can't port-forward the service, which is how I'm testing the scaling capabilities. Any thoughts?

Maybe you can't port forward the service because the replicas defaults to zero, so there are no pods, so there is nothing to port forward. It's a bit circular haha.

You could try playing with KEDA scaling on some external factor like messages in a queue. That way to generate load you just put a bunch of messages in a queue and then it would scale it up from zero.

@asteurer
Copy link
Contributor Author

asteurer commented Jul 11, 2024

Maybe you can't port forward the service because the replicas defaults to zero, so there are no pods, so there is nothing to port forward. It's a bit circular haha.

Shouldn't there be at least 1 pod since we set a minReplicas? If so, I'm not clear on how we would reference the HPA in the SpinApp deployment.

Also, k scale spinapp does work with the updated minimum pod count logic

@calebschoepp
Copy link
Contributor

Shouldn't there be at least 1 pod since we set a minReplicas? If so, I'm not clear on how we would reference the HPA in the SpinApp deployment.

Also, k scale spinapp does work with the updated minimum pod count logic

This looks relevant.

Copy link
Contributor

@calebschoepp calebschoepp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new comments from me. I just want you to address @bacongobbler's comments

@@ -5,7 +5,7 @@ metadata:
spec:
image: ghcr.io/spinkube/spin-operator/cpu-load-gen:20240311-163328-g1121986
executor: containerd-shim-spin
enableAutoscaling: true
replicas: 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

} else {
replicas = ptr(app.Spec.Replicas)
}
replicas := app.Spec.Replicas
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asteurer
Copy link
Contributor Author

There's a bug in SpinKube that is preventing metrics from being properly emitted from Spin applications, which is preventing autoscaling, and is making it impossible for me to test. Kate Goldenring is working with jsturtevant (slack handle) from MS to get this fixed.

@asteurer asteurer closed this by deleting the head repository Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants