Skip to content

Commit

Permalink
Integrating hloader (#55)
Browse files Browse the repository at this point in the history
Status codes



make map before adding data



Update examples



Bump hloader



Support multi-parch hloader

Signed-off-by: Raul Sevilla <[email protected]>
  • Loading branch information
rsevilla87 authored Jan 25, 2024
1 parent ce598f8 commit 353b12e
Show file tree
Hide file tree
Showing 9 changed files with 167 additions and 76 deletions.
40 changes: 22 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,31 @@ OCP Ingress performance ultimate tool!

## Reference

Ingress-perf configuration is defined in a YAML file, holding an array of the following structure. [Examples directory](./config)

| Field Name | Type | Description | Default Value |
|------------------|------------------|----------------------------------------------------------------------------------------------------------|---------------|
| `termination` | `string` | Defines the type of benchmark termination. Allowed values are `http`, `edge`, `reencrypt` and `reencrypt`. | N/A |
| `connections` | `int` | Defines the number of connections per client. | `0` |
| `samples` | `int` | Defines the number of samples per scenario. | `0` |
| `duration` | `time.Duration` | Defines the duration of each sample. | `""` |
| `path` | `string` | Defines the scenario endpoint path, for example: `/1024.html`, `/2048.html`. | `""` |
| `concurrency` | `int32` | Defines the number of clients that will concurrently run the benchmark scenario. | `0` |
| `tool` | `string` | Defines the tool to run the benchmark scenario. | `""` |
| `serverReplicas` | `int32` | Defines the number of server (nginx) replicas backed by the routes. | `0` |
| `tuningPatch` | `string` | Defines a JSON merge tuning patch for the default `IngressController` object. | `""` |
| `delay` | `time.Duration` | Defines a delay between samples. | `0s` |
| `warmup` | `bool` | Enables warmup: indexing will be disabled in this scenario. | `false` |
| `requestTimeout` | `time.Duration` | Request timeout | `1s` |
| `procs ` | `int` | Number of processes to trigger in each of the client pods | `1` |
Ingress-perf configuration is defined in a YAML file, holding an array of the following structure. [Examples directory](./examples)

| Field Name | Type | Description | Default Value | Tools |
|------------------|------------------|---------------------------------------------------------------------------------------------|---------------|------------------|
| `termination` | `string` | Benchmark termination. Allowed values are `http`, `edge`, `reencrypt` and `reencrypt`. | N/A | `wrk`,`hloader` |
| `connections` | `int` | Number of connections per client. | `0` | `wrk`,`hloader` |
| `samples` | `int` | Number of samples per scenario. | `0` | `wrk`,`hloader` |
| `duration` | `time.Duration` | Duration of each sample. | `""` | `wrk`,`hloader` |
| `path` | `string` | Scenario endpoint path, for example: `/1024.html`, `/2048.html`. | `""` | `wrk`,`hloader` |
| `concurrency` | `int32` | Number of clients that will concurrently run the benchmark scenario. | `0` | `wrk`,`hloader` |
| `tool` | `string` | Tool to run the benchmark scenario. | `""` | `wrk`,`hloader` |
| `serverReplicas` | `int32` | Number of server (nginx) replicas backed by the routes. | `0` | `wrk`,`hloader` |
| `tuningPatch` | `string` | Defines a JSON merge tuning patch for the default `IngressController` object. | `""` | `wrk`,`hloader` |
| `delay` | `time.Duration` | Delay between samples. | `0s` | `wrk`,`hloader` |
| `warmup` | `bool` | Enables warmup: indexing will be disabled in this scenario. | `false` | `wrk`,`hloader` |
| `requestTimeout` | `time.Duration` | Request timeout | `1s` | `wrk`,`hloader` |
| `procs` | `int` | Number of processes to trigger in each of the client pods | `1` | `wrk`,`hloader` |
| `keepalive` | `bool` | Use HTTP keepalived connections | `true` | `hloader` |
| `requestRate` | `int` | Number of requests per second | `0` (unlimited) | `hloader` |
| `http2` | `bool` | Use HTTP2 requests, when possible | `false` | `hloader` |

## Supported tools

- wrk: HTTP benchmarking tool. https://github.com/wg/wrk
- wrk: HTTP benchmarking tool. https://github.com/wg/wrk. amd64 and arm64
- hloader: https://github.com/rsevilla87/hloader

## Running

Expand Down
1 change: 1 addition & 0 deletions containers/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ FROM registry.access.redhat.com/ubi8/ubi:latest
RUN dnf install -y iproute procps-ng
COPY --from=builder /wrk/wrk /usr/bin/wrk
COPY json.lua json.lua
RUN curl -sS -L https://github.com/rsevilla87/hloader/releases/download/v0.2.0/hloader-Linux-v0.2.0-$(arch | sed s/aarch64/arm64/).tar.gz | tar xz -C /usr/bin/
62 changes: 62 additions & 0 deletions examples/hloader.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# vim: expandtab shiftwidth=2 softtabstop=2

# First sample is set as warmup and also will tune the default ingress-controller to place the routers on infra nodes
- termination: http
connections: 20
samples: 1
duration: 1m
path: /1024.html
concurrency: 1
tool: hloader
serverReplicas: 90
requestTimeout: 10s
tuningPatch: '{"spec":{"tuningOptions": {"threadCount": 8}, "nodePlacement": {"nodeSelector": {"matchLabels": {"node-role.kubernetes.io/infra": ""}}}, "replicas": 2}}'
warmup: true

- termination: http
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 1
tool: hloader
serverReplicas: 90
delay: 10s
requestTimeout: 10s
procs: 2

- termination: edge
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 1
tool: hloader
serverReplicas: 90
delay: 10s
requestTimeout: 10s
procs: 2

- termination: reencrypt
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 1
tool: hloader
serverReplicas: 90
delay: 10s
requestTimeout: 10s
procs: 2

- termination: passthrough
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 1
tool: hloader
serverReplicas: 90
delay: 10s
requestTimeout: 10s
procs: 2
42 changes: 0 additions & 42 deletions config/standard.yml → examples/standard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,45 +56,3 @@
serverReplicas: 90
delay: 10s
procs: 2

- termination: http
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 20
tool: wrk
serverReplicas: 90
delay: 10s

- termination: edge
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 20
tool: wrk
serverReplicas: 90
delay: 10s

- termination: reencrypt
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 20
tool: wrk
serverReplicas: 90
delay: 10s

- termination: passthrough
connections: 200
samples: 2
duration: 2m
path: /1024.html
concurrency: 20
tool: wrk
serverReplicas: 90
delay: 10s


1 change: 1 addition & 0 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
Warmup: false, // Disable warmup by default
RequestTimeout: time.Second,
Procs: 1,
Keepalive: true,
PrometheusMetrics: prometheusQueries,
}
if err := unmarshal(&defaultCfg); err != nil {
Expand Down
6 changes: 6 additions & 0 deletions pkg/config/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ type Config struct {
RequestTimeout time.Duration `yaml:"requestTimeout" json:"requestTimeout"`
// Prometheus metrics
PrometheusMetrics map[string]string `json:"-"`
// RequestRate defines the amount of requests to run in parallel
RequestRate int `yaml:"requestRate" json:"requestRate"`
// Keepalive use keepalived connections
Keepalive bool `yaml:"keepalive" json:"keepalive"`
// Use HTTP2 protocol, if possible
HTTP2 bool `yaml:"http2" json:"http2"`
}

var prometheusQueries = map[string]string{
Expand Down
1 change: 1 addition & 0 deletions pkg/runner/exec.go
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ func exec(ctx context.Context, tool tools.Tool, pod corev1.Pod, result *tools.Re
}

func normalizeResults(result *tools.Result) {
result.StatusCodes = make(map[int]int64)
for _, pod := range result.Pods {
result.TotalAvgRps += pod.AvgRps
result.StdevRps += pod.StdevRps
Expand Down
55 changes: 55 additions & 0 deletions pkg/runner/tools/hloader.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Copyright 2023 The ingress-perf Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package tools

import (
"encoding/json"
"fmt"
"strconv"

"github.com/cloud-bulldozer/ingress-perf/pkg/config"
)

type hLoader struct {
cmd []string
res PodResult
}

func init() {
toolMap["hloader"] = HLoader
}

func HLoader(cfg config.Config, ep string) Tool {
newHLoader := &hLoader{
cmd: []string{"hloader", "-u", ep,
"-c", strconv.Itoa(cfg.Connections),
"-d", fmt.Sprint(cfg.Duration),
"-r", strconv.Itoa(cfg.RequestRate),
"-t", fmt.Sprint(cfg.RequestTimeout),
fmt.Sprintf("--keepalive=%v", cfg.Keepalive),
fmt.Sprintf("--http2=%v", cfg.HTTP2),
},
res: PodResult{},
}
return newHLoader
}

func (w *hLoader) Cmd() []string {
return w.cmd
}

func (w *hLoader) ParseResult(stdout, _ string) (PodResult, error) {
return w.res, json.Unmarshal([]byte(stdout), &w.res)
}
35 changes: 19 additions & 16 deletions pkg/runner/tools/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,22 +33,24 @@ type Tool interface {
}

type PodResult struct {
Name string `json:"pod"`
Node string `json:"node"`
InstanceType string `json:"instanceType"`
AvgRps float64 `json:"rps"`
StdevRps float64 `json:"rps_stdev"`
StdevLatency float64 `json:"stdev_lat"`
AvgLatency float64 `json:"avg_lat_us"`
MaxLatency float64 `json:"max_lat_us"`
P90Latency float64 `json:"p90_lat_us"`
P95Latency float64 `json:"p95_lat_us"`
P99Latency float64 `json:"p99_lat_us"`
HTTPErrors int64 `json:"http_errors"`
ReadErrors int64 `json:"read_errors"`
WriteErrors int64 `json:"write_errors"`
Requests int64 `json:"requests"`
Timeouts int64 `json:"timeouts"`
Name string `json:"pod"`
Node string `json:"node"`
InstanceType string `json:"instanceType"`
AvgRps float64 `json:"rps"`
StdevRps float64 `json:"rps_stdev"`
StdevLatency float64 `json:"stdev_lat"`
AvgLatency float64 `json:"avg_lat_us"`
MaxLatency float64 `json:"max_lat_us"`
P90Latency float64 `json:"p90_lat_us"`
P95Latency float64 `json:"p95_lat_us"`
P99Latency float64 `json:"p99_lat_us"`
HTTPErrors int64 `json:"http_errors"`
ReadErrors int64 `json:"read_errors"`
WriteErrors int64 `json:"write_errors"`
Requests int64 `json:"requests"`
Timeouts int64 `json:"timeouts"`
AvgThgoughputBps int64 `json:"avg_throughput_bps"`
StatusCodes map[int]int64 `json:"status_codes"`
}

type Result struct {
Expand All @@ -72,5 +74,6 @@ type Result struct {
Timeouts int64 `json:"timeouts"`
Version string `json:"version"`
InfraMetrics map[string]float64 `json:"infra_metrics"`
StatusCodes map[int]int64 `json:"status_codes"`
ClusterMetadata
}

0 comments on commit 353b12e

Please sign in to comment.