Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TaoBench Issue on Cloud Instance Run #60

Open
SiyFeng opened this issue Sep 24, 2024 · 1 comment
Open

TaoBench Issue on Cloud Instance Run #60

SiyFeng opened this issue Sep 24, 2024 · 1 comment

Comments

@SiyFeng
Copy link

SiyFeng commented Sep 24, 2024

I have successfully installed TaoBench on both client and server instance on Alicloud. The whole run seams no issue showed. But the final result (total_qps) has the high probability to be zero.
Command:
./benchpress_cli.py run tao_bench_autoscale -i '{"num_clients": 1}'
Logs:
Results Report:
{
"benchmark_args": [
"--num-servers=0",
"--memsize=0",
"--fast-threads-ratio=0.75",
"--slow-to-fast-ratio=3",
"--interface-name=eth0",
"--port-number-start=11211",
"--warmup-time=0",
"--test-time=720",
"--server-hostname=",
"--num-clients=1",
"--clients-per-thread=0",
"--bind-cpu=1",
"--bind-mem=1",
"--real"
],
"benchmark_desc": "Spawns multiple Tao benchmark servers depending on the CPU core count. After executing this job, please see the tail of benchpress.log for instructions on starting clients. MAKE SURE to start clients within 1 minute.",
"benchmark_hooks": [
"tao_instruction: None",
"copymove: {'is_move': True, 'after': ['benchmarks/tao_bench/server*.csv', 'tao-bench-server-*.log']}"
],
"benchmark_name": "tao_bench_autoscale",
"machines": [
{
"cpu_architecture": "x86_64",
"cpu_model": "AMD EPYC 9T24 96-Core Processor",
"hostname": "iZuf64r6bitq56bglblqkrZ",
"kernel_version": "5.15.0-117-generic",
"mem_total_kib": "259220836 KiB",
"num_logical_cpus": "64",
"os_distro": "ubuntu",
"os_release_name": "Ubuntu 22.04.4 LTS"
}
],
"metadata": {
"L1d cache": "1 MiB (32 instances)",
"L1i cache": "1 MiB (32 instances)",
"L2 cache": "32 MiB (32 instances)",
"L3 cache": "128 MiB (4 instances)"
},
"metrics": {
"fast_qps": 0,
"hit_ratio": 0.0,
"num_data_points": 0,
"role": "server",
"score": 0.0,
"slow_qps": 0,
"spawned_instances": 1,
"successful_instances": 1,
"total_qps": 0
},
"run_id": "ffb66251",
"timestamp": 1727162932
}

@excelle08
Copy link
Contributor

Hi SiyFeng,

Could you check server_1.csv in your benchmark_metrics_ffb66251 folder to see the time-series QPS and hit ratio data? If the hit ratio did not end up reaching something greater than 0.88, the benchmark was not warmed up and the result parser would report zero because it thought there was no valid data points. In this case please increase warmup_time to ensure the benchmark can warm up.

Another consideration is that TaoBench is latency sensitive, so if the ping latency between the server and the client is high, the benchmark will also yield lower results than the server is expected to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants