Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: ensure we watch health probes in dnsrecord controller #308

Merged
merged 1 commit into from
Nov 22, 2024
Merged

Conversation

maksymvavilov
Copy link
Contributor

@maksymvavilov maksymvavilov commented Nov 15, 2024

Changes:

  • Watch probes in the dnsrecords reconciler
  • Refactor the receivedPrematurely to consult a list of probes (and not to retrieve probes twice I'm not passing them around)
  • If we have an operator running without probes-enabled flag set to true we now will ignore healthcheck spec. Before we would be waiting for probes to be created (they will not be created)
  • Added test case to capture premature reconciliation case

@maksymvavilov maksymvavilov linked an issue Nov 15, 2024 that may be closed by this pull request
@maksymvavilov maksymvavilov force-pushed the gh-264 branch 2 times, most recently from 6fe3e24 to 8430c7c Compare November 15, 2024 14:51
logger.V(1).Info("unexpected object type", "error", fmt.Sprintf("%T is not a *v1alpha1.DNSHealthCheckProbe", o))
}

records := &v1alpha1.DNSRecordList{}
Copy link
Collaborator

@maleck13 maleck13 Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here we could do a get based on the owner ref on the probe? We only have one probe here and that probe can only belong to one record right? So :

  • get fresh copy probe from cluster
  • check status healthy against the one received in the watch function
  • If different than the fresh copy, get dnsrecord based on ownerref of probe and requeue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fetching records in the watch at all now. Using owner reference for it

@@ -446,7 +505,7 @@ func setStatusConditions(record *v1alpha1.DNSRecord, hadChanges bool, notHealthy
setDNSRecordCondition(record, string(v1alpha1.ConditionTypeReady), metav1.ConditionTrue, string(v1alpha1.ConditionReasonProviderSuccess), "Provider ensured the dns record")

// probes are disabled or not defined, or this is a wildcard record
if record.Spec.HealthCheck == nil || strings.HasPrefix(record.Spec.RootHost, v1alpha1.WildcardPrefix) {
if record.Spec.HealthCheck == nil || strings.HasPrefix(record.Spec.RootHost, v1alpha1.WildcardPrefix) || !probesEnabled {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

starting to think we might remove the probes enabled flag soon

@maleck13 maleck13 changed the title watch health probes in dnsrecord controller Bug: ensure we watch health probes in dnsrecord controller Nov 19, 2024
}

probes := &v1alpha1.DNSHealthCheckProbeList{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we defined this early would it help with readability. IE jusst pass in the empty list of probes to updateStatus? as passing in nil is a bit obscure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defined up top and passing it always into the update status func

Copy link
Collaborator

@maleck13 maleck13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good tried a few scenrios DNSRecord and Policy ended up as expectd

status:
  conditions:
  - lastTransitionTime: "2024-11-20T07:14:32Z"
    message: DNSPolicy has been accepted
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-11-21T08:15:08Z"
    message: DNSPolicy has been successfully enforced
    reason: Enforced
    status: "True"
    type: Enforced
  - lastTransitionTime: "2024-11-22T11:20:05Z"
    message: 'DNSPolicy has encountered some issues: not all sub-resources of policy
      are passing the policy defined health check. Not healthy DNSRecords are: external-t1c
      external-t1d '
    reason: Unknown
    status: "False"
    type: SubResourcesHealthy
  observedGeneration: 1
  recordConditions:
    t1c.cb.hcpapps.net:
    - lastTransitionTime: "2024-11-22T11:20:03Z"
      message: 'Not healthy addresses: [af2b62688c5454c35a1827a82e7d7c68-1089732621.eu-west-1.elb.amazonaws.com]'
      observedGeneration: 1
      reason: HealthChecksFailed
      status: "False"
      type: Healthy
    t1d.cb.hcpapps.net:
    - lastTransitionTime: "2024-11-22T11:20:03Z"
      message: 'Not healthy addresses: [af2b62688c5454c35a1827a82e7d7c68-1089732621.eu-west-1.elb.amazonaws.com]'
      observedGeneration: 1
      reason: HealthChecksFailed
      status: "False"
      type: Healthy
  totalRecords: 2
status:
  conditions:
  - lastTransitionTime: "2024-11-20T07:14:32Z"
    message: DNSPolicy has been accepted
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-11-21T08:15:08Z"
    message: DNSPolicy has been successfully enforced
    reason: Enforced
    status: "True"
    type: Enforced
  - lastTransitionTime: "2024-11-22T14:06:42Z"
    message: All sub-resources are healthy
    reason: SubResourcesHealthy
    status: "True"
    type: SubResourcesHealthy
  observedGeneration: 1
  totalRecords: 2

@maleck13 maleck13 added this pull request to the merge queue Nov 22, 2024
Merged via the queue into main with commit 495efc9 Nov 22, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Watch probes in dnsrecord reconciler
2 participants