Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support configuring RESTAPI to GRPC via HTTPFilterPolicy #616

Closed
wants to merge 3 commits into from
Closed

Support configuring RESTAPI to GRPC via HTTPFilterPolicy #616

wants to merge 3 commits into from

Conversation

hanxiantao
Copy link
Contributor

add grpcJsonTranscoder plugin
fix #605

@codecov Codecov
Copy link

codecov bot commented Jun 29, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 88.88889% with 2 lines in your changes missing coverage. Please review.

Project coverage is 89.72%. Comparing base (1d5db16) to head (db6bb02).
Report is 110 commits behind head on main.

Files with missing lines Patch % Lines
types/plugins/grpc_json_transcoder/config.go 77.77% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #616      +/-   ##
==========================================
+ Coverage   89.70%   89.72%   +0.01%     
==========================================
  Files         109      111       +2     
  Lines        5216     5234      +18     
==========================================
+ Hits         4679     4696      +17     
- Misses        362      363       +1     
  Partials      175      175

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hanxiantao hanxiantao marked this pull request as draft June 29, 2024 10:17
@hanxiantao
Copy link
Contributor Author

@spacewander After using the grpc_json_transcoder, it appears to load by default
htnn/gateway:e2e error info:

2024-06-30T00:30:00.840780Z warning envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138 gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) 0.0.0.0_10000: Proto constraint validation failed (field: "descriptor_set", reason: is required):
thread=18
2024-06-30T00:30:01.899953Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:02.898713Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:03.899237Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected

istiod error info:

2024-06-30T00:32:44.401660Z info ads ADS: "10.244.0.15:48202" default-istio-7b5bfb6c49-d6nn8.e2e-another-35 terminated
2024-06-30T00:33:48.712678Z info delta ADS: "10.244.0.8:51874" istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-1 terminated
2024-06-30T00:33:49.104493Z info delta ADS: new delta connection for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36
2024-06-30T00:33:49.107514Z info delta CDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:21 removed:0 size:21.7kB cached:0/20 filtered:0
2024-06-30T00:33:49.110402Z info delta EDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:20 removed:0 size:3.2kB empty:4 cached:0/20 filtered:0
2024-06-30T00:33:49.110919Z info delta LDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:1 removed:0 size:5.0kB
2024-06-30T00:33:49.165421Z warn delta ADS:LDS: ACK ERROR istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36 Internal:Error adding/updating listener(s) 0.0.0.0_18000: Proto constraint validation failed (field: "descriptor_set", reason: is required):

@spacewander
Copy link
Member

@spacewander After using the grpc_json_transcoder, it appears to load by default htnn/gateway:e2e error info:

2024-06-30T00:30:00.840780Z warning envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138 gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) 0.0.0.0_10000: Proto constraint validation failed (field: "descriptor_set", reason: is required):
thread=18
2024-06-30T00:30:01.899953Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:02.898713Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:03.899237Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected

istiod error info:

2024-06-30T00:32:44.401660Z info ads ADS: "10.244.0.15:48202" default-istio-7b5bfb6c49-d6nn8.e2e-another-35 terminated
2024-06-30T00:33:48.712678Z info delta ADS: "10.244.0.8:51874" istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-1 terminated
2024-06-30T00:33:49.104493Z info delta ADS: new delta connection for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36
2024-06-30T00:33:49.107514Z info delta CDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:21 removed:0 size:21.7kB cached:0/20 filtered:0
2024-06-30T00:33:49.110402Z info delta EDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:20 removed:0 size:3.2kB empty:4 cached:0/20 filtered:0
2024-06-30T00:33:49.110919Z info delta LDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:1 removed:0 size:5.0kB
2024-06-30T00:33:49.165421Z warn delta ADS:LDS: ACK ERROR istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36 Internal:Error adding/updating listener(s) 0.0.0.0_18000: Proto constraint validation failed (field: "descriptor_set", reason: is required):

We need to provide a placeholder config for http filter in HTTPFilterConfigPlaceholder. The placeholder config will be validated but won't be used.

I am not sure how deep Envoy will validate the config in https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/grpc_json_transcoder/v3/transcoder.proto#extensions-filters-http-grpc-json-transcoder-v3-grpcjsontranscoder, maybe we can pass a fake pb filename if it doesn't validate the content.

@hanxiantao
Copy link
Contributor Author

@spacewander使用grpc_json_transcoder后，似乎默认加载htnn/gateway:e2e错误信息：

2024-06-30T00:30:00.840780Z warning envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138 gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) 0.0.0.0_10000: Proto constraint validation failed (field: "descriptor_set", reason: is required):
thread=18
2024-06-30T00:30:01.899953Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:02.898713Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2024-06-30T00:30:03.899237Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected

istiod 错误信息：

2024-06-30T00:32:44.401660Z info ads ADS: "10.244.0.15:48202" default-istio-7b5bfb6c49-d6nn8.e2e-another-35 terminated
2024-06-30T00:33:48.712678Z info delta ADS: "10.244.0.8:51874" istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-1 terminated
2024-06-30T00:33:49.104493Z info delta ADS: new delta connection for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36
2024-06-30T00:33:49.107514Z info delta CDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:21 removed:0 size:21.7kB cached:0/20 filtered:0
2024-06-30T00:33:49.110402Z info delta EDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:20 removed:0 size:3.2kB empty:4 cached:0/20 filtered:0
2024-06-30T00:33:49.110919Z info delta LDS: PUSH request for node:istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system resources:1 removed:0 size:5.0kB
2024-06-30T00:33:49.165421Z warn delta ADS:LDS: ACK ERROR istio-ingressgateway-56d5bdc4f5-5kjqx.istio-system-36 Internal:Error adding/updating listener(s) 0.0.0.0_18000: Proto constraint validation failed (field: "descriptor_set", reason: is required):

我们需要在 HTTPFilterConfigPlaceholder 中为 http 过滤器提供占位符配置。该占位符配置将被验证但不会被使用。

我不确定 Envoy 将如何深入验证https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/grpc_json_transcoder/v3/transcoder.proto#extensions-filters-http-grpc-json-transcoder-v3-grpcjsontranscoder中的配置，如果它不验证内容，也许我们可以传递一个假的 pb 文件名。

Thank you for your help. I've seen similar examples in other code sections, and I will give it a try following your suggestions.

@github-actions GitHub Actions
Copy link

👋 Hi, I detected conflicts against the base branch 🙊
You'll want to sync 🔄 your branch with upstream!

@github-actions GitHub Actions
Copy link

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added stale and removed stale labels Aug 22, 2024
@github-actions GitHub Actions
Copy link

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added the stale label Sep 22, 2024
@github-actions GitHub Actions
Copy link

This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot closed this Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
conflict stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support configuring RESTAPI to GRPC via HTTPFilterPolicy
2 participants
@hanxiantao @spacewander