Computer resources consuming

[ekorolevanyrun]

Hi!

I am using ursadb for searching malware samples with yara rules, and my goal is make ursadb work as faster as possible. However I'm facing with 12-15 seconds queries when make select in gigabyte sized datasets (there are approximately 40-50 datasets). Also I noticed that ursa consumes only 10-15% cpu of a single processor thread, 4-5 MB/sec of disk (measured using iotop).

So my question is: is it possible to boost ursadb searching and make to use ursadb max of computer resources?

We have optane ssd, I assume the searching speed should be higher.

---

I have try to decrease datasets size
I set config parameters (database_workers) to different values

It didn't help

[msm-code]

Hi, are you still interested in having this fixed?

Ursadb should be either using 100% CPU, or 100% waiting disk. Even if you're using SSD, in most cases it should be blocked on SSD. 12 second queries when making select in gigabyte sized datasets is extremely suspicious. It should take miliseconds, seconds at best.

Is your dataset characteristic in any way? For example files are large, or there's a lot of very small files? Is there a way for me to reproduce the issue?