Internal Encryption fixes #860

KauzClay · 2023-01-31T22:47:09Z

Changes

🐛 Fixes #862

Unencrypt request back to envoy for domainmappings when internal encryption is enabled
net-contour currently sets the protocol as h2 for services with ports named http2 when internal encryption. This change will set the protocol as h2c if the kingress has domainmapping labels.
This isn't the perfect fix, but it does at least make domainmappings work properly when using internal encryption. I hope this is enough for the beta stage of these features.

🐛 internal encryption breaks http01 challenges

Don't encrypt traffic to http challenge endpoint, use http instead
The current implementation tries to use tls to connect to the http01 challenge endpoint when internal encryption is enabled, auto-tls is enabled, and httpOption is redirected,
This path is currently hidden by flawed cert-manager behavior that is being addressed here. So long as this fix is included before or in the same release as the cert-manager changes, I don't think anyone will run into this.

Release Note

Fixes issue where DomainMappings do not become ready when Internal Encryption is enabled.

Docs

N/A

knative-prow · 2023-01-31T22:47:11Z

@KauzClay: The label(s) kind/<kind> cannot be applied, because the repository doesn't have them.

In response to this:

Changes

unencrypt call back to envoy in domainmappings when internal encryption is enabled

this is a hack as of now, not a good idea to rely on rewrite hosts to know something is domainmapping

/kind

Fixes # knative/serving#13659

Release Note
TBD
Docs
N/A

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2023-01-31T22:53:24Z

Codecov Report

Merging #860 (6dfcff6) into main (6e15300) will increase coverage by 0.08%.
The diff coverage is 91.66%.

❗ Current head 6dfcff6 differs from pull request most recent head d854098. Consider uploading reports for the commit d854098 to get more accurate results

@@            Coverage Diff             @@
##             main     #860      +/-   ##
==========================================
+ Coverage   93.99%   94.08%   +0.08%     
==========================================
  Files           7        7              
  Lines         783      794      +11     
==========================================
+ Hits          736      747      +11     
  Misses         27       27              
  Partials       20       20

Impacted Files	Coverage Δ
pkg/reconciler/contour/contour.go	`85.00% <85.71%> (+0.38%)`	⬆️
pkg/reconciler/contour/resources/httpproxy.go	`97.81% <100.00%> (+0.04%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

KauzClay · 2023-02-06T16:08:24Z

/assign @dprotaso

KauzClay · 2023-02-06T16:13:24Z

/assign @evankanderson

Since you were also involved on some of the previous internal-encryption PRs

KauzClay · 2023-02-06T22:57:24Z

pkg/reconciler/contour/contour.go

@@ -332,3 +333,11 @@ func (r *Reconciler) lbStatus(ctx context.Context, vis v1alpha1.IngressVisibilit
 	}
 	return
 }
+
+func isDomainMapping(ing *v1alpha1.Ingress) bool {
+	_, hasDmUIDLabelKey := ing.Labels[serving.DomainMappingUIDLabelKey]


I had to bring in a fair amount of serving code just to use these labels. Maybe there is a better way to import it that doesn't bring in so much?

What you probably want to do is to either copy the constants here, or move them into networking. When you have net-contour import serving directly, it makes it hard to cut a release, because the net-contour package can't be cut until the serving package is tagged.

Alternatively, you could look at the OwnerReferences field to see whether the owner is a Knative Route or a DomainMapping.

The third option would be to add some explicit field to the KIngress indicating which type it is.

The fourth option would be to look at the spec.rules -- if there's a single rule with visibility: externalIP, that must be a DomainMapping, because a Route always has an internal mapping: https://github.com/knative/serving/blob/main/pkg/reconciler/route/resources/ingress.go#L141

ah okay, thanks for explaining that about importing serving.

I didn't even think about the owner reference, I think I will go for that. It seems the least intrusive but still pretty explicit. Then I can avoid having duplicated constants or adding more fields to the kingress.

DomainMapping just sets HostRewrite on the KIngress - https://github.com/knative/networking/blob/2473e65d69206ed544f32e47f994cf3635bebdbe/pkg/apis/networking/v1alpha1/ingress_types.go#L208-L212

You could look at that spec property and the target k8s service (it pointing to a contour's envoy service)

ah, well maybe not, the type definition for domainmappings is in the serving repo.

I think you can define a constant for DomainMapping here, or add a helper in networking to determine if a KIngress represents a DomainMapping.

It might be more robust, though, to look for the rewriteHeader field and the presence of knative.dev/networking/pkg/http/header.OriginalHostKey as a field in AppendHeaders.

okay, I went with the check for rewriteheader + originalhost key, and put in a comment explaining it a bit.

dprotaso · 2023-02-07T18:18:52Z

This PR makes me realize that we need a special consideration for traffic going through external envoy => internal envoy and ensuring that hop is encrypted - right now we're setting it to h2c.

I wonder if TLS passthrough makes more sense.

evankanderson · 2023-02-07T18:36:15Z

I wonder if TLS passthrough makes more sense.

I like that, except that we have no KIngress construct to describe this (and it's not clear that we should).

KauzClay · 2023-02-07T21:13:34Z

/test integration-tests

* unencrypt call back to envoy in domainmappings when internal encryption is enabled * identify domainmapping kingress via presence of Rewrite host and k-original-host header

KauzClay · 2023-02-14T17:41:52Z

Hey @evankanderson @dprotaso , do you have any other thoughts on this one?

To me it sounds like any actions regarding Dave's comment

This PR makes me realize that we need a special consideration for traffic going through external envoy => internal envoy and ensuring that hop is encrypted - right now we're setting it to h2c.

probably should happen in another PR. Is that what you had in mind?

dprotaso · 2023-02-21T15:56:18Z

probably should happen in another PR. Is that what you had in mind?

Yeah let's make an new issue and do it as a follow up

/lgtm
/approve

knative-prow · 2023-02-21T15:56:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso, KauzClay

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [KauzClay,dprotaso]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 31, 2023

KauzClay changed the title ~~[wip:] use h2c on domainmapping~~ [wip] use h2c on domainmapping Feb 1, 2023

knative-prow bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 1, 2023

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch 2 times, most recently from da86648 to f2e4f81 Compare February 1, 2023 18:27

knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 3, 2023

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch from 503a2ee to 00c84a9 Compare February 3, 2023 20:58

knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 3, 2023

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch from 00c84a9 to c743fa4 Compare February 3, 2023 21:13

KauzClay changed the title ~~[wip] use h2c on domainmapping~~ [wip] Internal Encryption fixes Feb 3, 2023

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch from 132b52b to 7657dfc Compare February 6, 2023 15:16

knative-prow bot assigned KauzClay Feb 6, 2023

KauzClay removed their assignment Feb 6, 2023

knative-prow bot assigned dprotaso Feb 6, 2023

knative-prow bot assigned evankanderson Feb 6, 2023

KauzClay commented Feb 6, 2023

View reviewed changes

KauzClay mentioned this pull request Feb 6, 2023

Implement e2e test for internal encryption knative/serving#13536

Closed

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 7, 2023

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch from 4c97bd6 to 6dfcff6 Compare February 7, 2023 22:15

knative-prow bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 7, 2023

KauzClay added 2 commits February 8, 2023 16:25

use h2c on domainmapping

57e5730

* unencrypt call back to envoy in domainmappings when internal encryption is enabled * identify domainmapping kingress via presence of Rewrite host and k-original-host header

don't encrypt http01 challenge traffic

d854098

KauzClay force-pushed the ck-internal-encryption-plaintext-loopback branch from b5ba695 to d854098 Compare February 8, 2023 21:26

KauzClay changed the title ~~[wip] Internal Encryption fixes~~ Internal Encryption fixes Feb 9, 2023

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 9, 2023

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 21, 2023

knative-prow bot merged commit e1d2bcc into knative-extensions:main Feb 21, 2023

This was referenced Feb 23, 2023

Enabling Internal Encryption breaks DomainMappings when using Contour #862

Open

Fix DomainMappings when internal-encryption is enabled #878

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Internal Encryption fixes #860

Internal Encryption fixes #860

KauzClay commented Jan 31, 2023 •

edited

Loading

knative-prow bot commented Jan 31, 2023

Changes

codecov bot commented Jan 31, 2023 •

edited

Loading

KauzClay commented Feb 6, 2023

KauzClay commented Feb 6, 2023

KauzClay Feb 6, 2023

evankanderson Feb 7, 2023

KauzClay Feb 7, 2023

dprotaso Feb 7, 2023

KauzClay Feb 7, 2023

evankanderson Feb 7, 2023

KauzClay Feb 8, 2023

dprotaso commented Feb 7, 2023

evankanderson commented Feb 7, 2023

KauzClay commented Feb 7, 2023

KauzClay commented Feb 14, 2023 •

edited

Loading

dprotaso commented Feb 21, 2023

knative-prow bot commented Feb 21, 2023

Internal Encryption fixes #860

Internal Encryption fixes #860

Conversation

KauzClay commented Jan 31, 2023 • edited Loading

Changes

knative-prow bot commented Jan 31, 2023

Changes

codecov bot commented Jan 31, 2023 • edited Loading

Codecov Report

KauzClay commented Feb 6, 2023

KauzClay commented Feb 6, 2023

KauzClay Feb 6, 2023

Choose a reason for hiding this comment

evankanderson Feb 7, 2023

Choose a reason for hiding this comment

KauzClay Feb 7, 2023

Choose a reason for hiding this comment

dprotaso Feb 7, 2023

Choose a reason for hiding this comment

KauzClay Feb 7, 2023

Choose a reason for hiding this comment

evankanderson Feb 7, 2023

Choose a reason for hiding this comment

KauzClay Feb 8, 2023

Choose a reason for hiding this comment

dprotaso commented Feb 7, 2023

evankanderson commented Feb 7, 2023

KauzClay commented Feb 7, 2023

KauzClay commented Feb 14, 2023 • edited Loading

dprotaso commented Feb 21, 2023

knative-prow bot commented Feb 21, 2023

KauzClay commented Jan 31, 2023 •

edited

Loading

codecov bot commented Jan 31, 2023 •

edited

Loading

KauzClay commented Feb 14, 2023 •

edited

Loading