Ideal Large Language Model host system

[arjacy]

Hi everyone.
I'm not up to date, and also inexperienced, so please forgive my ignorance.
Llama.cpp is a really impressive and useful system, but it does not use all cores available, even on a very large system (100+ cores).
I am asking you all, why is this the case? Have you all evaluated and/or considered using the SequenceL interpreting compiler to generate massively thrraded cpp, or, is the real bottleneck memory bandwidth?
Thanks everyone and thanks Grigory!
-Richard