The request will be triggered from time to time after the request enters apisix, and the request will be sent to the back end with a delay of 1-2s

[sandy420]

The following is the configuration of my APISIX:
APISIX version: v2.10.1

etcd:
host:
- http://10.x.x.x:2379
- http://10.x.x.x:2379
- http://10.x.x.x:2379
user: "xxxxx"
password: "xxxxxx"
apisix:
enable_server_tokens: false
stream_proxy:
only: false
tcp:
- 15028
- 14029
- 25028
- 24029
- 10050
- 4433
- 10040
- 20161
admin_key:
- name: XXXX
key: xxxxxxxxxxxxxxxxxxxxx # using fixed API token has security risk, please update it when you deploy to production environment
role: admin
node_listen: 80
allow_admin:
- 10.0.0.0/8
- 172.16.0.0/16
- 127.0.0.1
router:
http: 'radixtree_host_uri'
ssl:
listen_port: 443
enable_http2: false
ssl_protocols: "TLSv1 TLSv1.1 TLSv1.2 TLSv1.3"
ssl_ciphers: "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES
128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA"plugins:

• proxy-rewrite
• prometheus
• log-rotate
• redirect
• node-status
• server-info
• proxy-mirror
• echo
• ip-restriction
• basic-auth
• kafka-logger
  plugin_attr:
  prometheus:
  export_uri: /apisix/prometheus/metrics
  enable_export_server: true
  export_addr:
  ip: 0.0.0.0
  port: 9091
  log-rotate:
  interval: 21600
  max_kept: 120
  nginx_config:
  worker_rlimit_nofile: 65535
  event:
  worker_connections: 65535
  http_end_configuration_snippet: |
  reset_timedout_connection off;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 128k;
  client_body_buffer_size 2048k;
  proxy_intercept_errors on;
  large_client_header_buffers 8 32k;
  gzip off;
  gzip_min_length 1k;
  gzip_buffers 4 16k;
  gzip_http_version 1.0;
  gzip_comp_level 9;
  gzip_vary on;
  gzip_types
  text/plain
  application/x-javascript
  text/css
  application/javascript
  application/json
  application/xml
  text/javascript
  application/x-httpd-php
  image/jpeg
  image/gif
  image/png;
  proxy_next_upstream off;
  proxy_connect_timeout 60;
  proxy_send_timeout 300s;
  proxy_read_timeout 300s;
  proxy_buffers 64 32k; # getconf PAGESIZE
  proxy_buffer_size 1024k;
  proxy_busy_buffers_size 1024k;
  proxy_temp_file_write_size 1024k;
  proxy_max_temp_file_size 0;
  proxy_ignore_client_abort on;
  proxy_http_version 1.1;
  proxy_set_header Connection "";
  proxy_set_header Host $host;
  proxy_set_header traffic-source "INTERNAL";
  http:
  lua_shared_dict:
  prometheus-metrics: 100m
  real_ip_header: "X-Forwarded-For"
  real_ip_from:
  • 0.0.0.0/0
  • 'unix:'
    keepalive_timeout: 600s
    client_header_timeout: 600s
    client_body_timeout: 600s
    send_timeout: 100s
    access_log_format: "$time_iso8601|$msec|$status|$request_completion|$bytes_sent|$body_bytes_sent|$realip_remote_addr|$remote_addr|$http_x_forwarded_for|$remote_user|$h
    ost|$server_name|$server_port|$server_protocol|$scheme|$request_method|$request_length|$request_time|$request_uri|$uri|$content_length|$content_type|$http_referer|$http_user_agent|$http_app_jb|$http_client_info|$upstream_addr|$upstream_connect_time|$upstream_header_time|$upstream_response_time|$upstream_status|$upstream_bytes_received|$upstream_cache_status|$upstream_http_content_type|$upstream_http_content_length|$upstream_http_content_disposition|$http_x_cat_parent_id|$upstream_scheme://$upstream_host$upstream_uri"

[tokers]

Please provide some steps so that we can reproduce your scenario.

[sandy420]

Please provide some steps so that we can reproduce your scenario.

And so, when the traffic request enters a normally deployed APISIX cluster (a cluster composed of 7 apisix hosts + 5 ETCD), then this request occasionally has a 1-2 second delay. Our request volume is 10,000 + QPS, but This kind of abnormal request is very rare, and it occurs 1-3 times every few minutes, which causes a very bad user experience for financial services.