Skip to content

OSIS Initial Performance Analysis

Jump to bottom
Sai Tharun Thodupunuru edited this page Sep 6, 2021 · 31 revisions

Objective

Checking the performance behavior of the OSIS service on scality cloud. As OSIS is not configured through NGINX, only one node will be configured in vCloudDirector.

  • Tenant Flow: Query/Create/List Tenant

  • User Flow: Query/Get/Create/List User

  • Credential Flow: Query/Get/Create/List Credentials

Test Cases:

  • Base Case:

    • Simple flow → Query/Create/List Tenant, Query/Get/Create/List User, Query/Get/Create/List Credentials

  • Medium load:

    • 1 client (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

    • 2 clients (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

    • 4 clients (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

  • Ramp Up test - Stress failure test

    • Tenant Flow, User Flow, Credential Flow
Clients (threads) Loop Count(number of times to execute the test per thread) Rampup time: (Ramp-Up Period tells JMeter how long to delay before starting the next user) (in seconds)
128 10 300
256 10 300
512 10 300
128 10 600
256 10 600
512 10 600

Metrics to capture:

  • Elapsed time: Measures the elapsed time from just before sending the request to just after the last chunk of the response has been received,

  • Latency: Measures the latency from just before sending the request to just after the first chunk of the response has been received,

  • Median: Number which divides the samples into two equal halves,

  • 90% Line (90th Percentile): The elapsed time below which 90% of the samples fall

  • 95% Line (95th Percentile): The elapsed time below which 95% of the samples fall

  • Throughput: Calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. The formula is: Throughput = (number of requests) / (total time).

Storage Node metrics using RING grafana dashboards(http:///stats/):

  • Docker Image size

  • RAM/CPU/Disk usage

  • Report numbers to Product for tooling adjustments.

Test Results

1. Base Case:

Simple flow → Query/Create/List Tenant, Query/Get/Create/List User, Query/Get/Create/List Credentials

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Create Tenant 1 1095 1095 1095 1095 1095 1095 1095 0.000% 0.91324
List Tenants 1 204 204 204 204 204 204 204 0.000% 4.90196
Query Tenants 1 205 205 205 205 205 205 205 0.000% 4.87805
Create User 1 1432 1432 1432 1432 1432 1432 1432 0.000% 0.69832
List Users 1 292 292 292 292 292 292 292 0.000% 3.42466
Query Users 1 266 266 266 266 266 266 266 0.000% 3.75940
Get User 1 236 236 236 236 236 236 236 0.000% 4.23729
Create Credential 1 281 281 281 281 281 281 281 0.000% 3.55872
List Credentials 1 319 319 319 319 319 319 319 0.000% 3.13480
Query Credentials 1 278 278 278 278 278 278 278 0.000% 3.59712
Get Credential 1 237 237 237 237 237 237 237 0.000% 4.21941
TOTAL 11 440 278 1095 1095 1432 204 1432 0.000% 2.23532

2. Medium load

i. 1 client (threads)

a.(loop count) 100 x Tenant Flow, User Flow, Credential Flow

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Create Tenant 100 228 214 252 294 385 192 813 0.000% 0.38506
List Tenants 100 222 216 251 265 276 196 279 0.000% 0.38624
Query Tenants 100 203 198 221 228 262 185 296 0.000% 0.38622
Create User 100 386 374 464 494 564 292 565 0.000% 0.38603
List Users 100 210 206 239 244 281 185 307 0.000% 0.38630
Query Users 100 232 228 259 279 284 206 353 0.000% 0.38635
Get User 100 208 203 225 243 282 189 312 0.000% 0.38639
Create Credential 100 246 229 281 295 345 202 1253 0.000% 0.38644
List Credentials 100 227 223 256 269 287 200 294 0.000% 0.38643
Query Credentials 100 218 212 245 262 283 192 291 0.000% 0.38647
Get Credential 100 222 213 253 273 366 195 427 0.000% 0.38650
TOTAL 1100 237 217 304 373 478 185 1253 0.000% 4.19926

b.(loop count) 1000 x Tenant Flow, User Flow, Credential Flow

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Create Tenant 1000 216 207 241 259 308 185 1446 0.000% 1.34937
List Tenants 1000 283 263 375 440 599 195 874 0.000% 1.35041
Query Tenants 1000 237 224 292 331 439 185 612 0.000% 1.35033
Create User 1000 299 283 352 378 490 240 1719 0.000% 0.98528
List Users 1000 299 292 365 385 489 195 922 0.000% 0.98535
Query Users 1000 207 202 229 241 266 182 563 0.000% 0.98540
Get User 1000 205 201 226 240 261 181 422 0.000% 0.98575
Create Credential 1000 229 221 254 272 310 196 1549 0.000% 0.49581
List Credentials 1000 1209 1226 1896 2125 2515 220 3220 0.000% 0.49534
Query Credentials 1000 289 288 350 373 425 193 493 0.000% 0.49531
Get Credential 1000 287 285 350 375 436 191 576 0.000% 0.49528
TOTAL 11000 342 247 410 1138 1856 181 3220 0.000% 2.91225
Clone this wiki locally