Skip to content
This repository has been archived by the owner on Jan 8, 2024. It is now read-only.

AWS ECS releaser reports empty target group as healthy #4900

Open
pcjun97 opened this issue Sep 7, 2023 · 0 comments
Open

AWS ECS releaser reports empty target group as healthy #4900

pcjun97 opened this issue Sep 7, 2023 · 0 comments
Labels

Comments

@pcjun97
Copy link

pcjun97 commented Sep 7, 2023

Describe the bug
The releaser for AWS ECS plugin checks for unhealthy targets in a target group, but reports healthy if there is no target in the target group at all. The deploy step does not wait for the tasks to be in "running" state before moving to the release step, which at that point in time the tasks (which are in "pending" state) has yet to be registered as targets in the new target group.

Steps to Reproduce
Run waypoint deploy with aws-ecs apps.

project = "example"

app "httpbin" {
  build {
    use "docker-pull" {
      image = "kennethreitz/httpbin"
    }
  }

  deploy {
    use "aws-ecs" {
      ...
    }
  }
}

Despite the ECS task is still starting up and the target group being empty, the release step will reports all targets are healthy:

» Releasing...

» Performing operation locally
-> Running release v383
-> Initializing release...
-> Release initialized
-> Checking that all targets are healthy...
-> All targets are healthy!
-> Modifying load balancer to introduce new target group <REDACTED>
-> Finished ECS release

Expected behavior
The releaser should treat a target group with no targets as "unhealthy", and report that accordingly.

Waypoint Platform Versions
Additional version and platform information to help triage the issue if
applicable:

  • Waypoint CLI Version: 0.11.3
  • Waypoint Server Platform and Version: 0.11.3
  • Waypoint Plugin: aws/ecs

Additional context
Relevant code snippet:

// Check each target to see if any one of them isn't healthy, before we
// route 100% of traffic to it!
for _, targetHealthDescription := range targetHealth.TargetHealthDescriptions {
log.Debug("checking target health", "target", targetHealthDescription.Target.Id)
// Possible states are: initial, healthy, unhealthy, unused, draining,
// and unavailable.
if *targetHealthDescription.TargetHealth.State != "healthy" {
return nil, errors.Errorf("target (id: %s) is not healthy - will "+
"only release when all targets in group (ARN: %q) are healthy",
*targetHealthDescription.Target.Id, target.TargetGroupArn)
}
}

@pcjun97 pcjun97 added the new label Sep 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant