How can the settings be adjusted to improve CPU utilization during MTEB execution?

[eric-yq]

Hello team, I am using following command to run task on 8 vCPU cloud servers
mteb run -m BAAI/bge-large-en-v1.5 -t Banking77Classification

But the CPU usage is different,

• c7a.2xlarge: SMT disabled, nearly 100%
• c7i.2lxlarge: SMT enabled, arount 50%

Can you guide me how to increase the CPU usage on c7i.2lxlarge(SMT enabled) server to 100%?

Thanks

[KennethEnevoldsen]

Converting this to a conversation. As it is a usage question.

Hi @eric-yq

I believe increasing utilization is a question of increasing model utilization of the CPU. mteb provides a reference implementation of models in https://github.com/embeddings-benchmark/mteb/tree/main/mteb/models.

However, using the python API you can flexibly provide any implementation of a model

model = ...
# add you model specific optimizations here, e.g. quantization, move to gpu, distribute across multiple cpus using torch 

tasks = mteb.get_tasks(tasks = ["Banking77Classification"])
benchmark = mteb.MTEB(tasks = ...) 
benchmark.run(model)

For reference on how to speed up sentence transformer on CPU, see this

[eric-yq]

I found that on Graviton4 instance(ARMv9, Neoverse-V2), it shows much do_wait which will slower the performance much.
On Intel CPU instance, AVX512 is used t accelerate the performance.