Replace sirupsen/logrus with a more performant logger

[tzaffi]

Problem

In working on #128 , I ran some performance tests and noticed that log-level had a significant impact. logrus has been observed to be slow in go-algorand and there is an internal issue (2479) to replace it and a poc branch using zerolog.

Problematic experiment

Using the Justfile command

❯ just conduit-bootstrap-and-go 300

to bootstrap testnet and run a postgresql exporter against it for 300 seconds. I ran it a number of times against both the original pipeline and the new one. Here are the experimental results:

Log Level	Reps	Original rounds/300 sec (logs/round)	Pipelining rounds/300 sec (logs/round)	Pipelining v Original (%)
TRACE	3	3718 (7.0)	3509 (14.0)	-5.6%
INFO	2	4578.5 (3.0)	4423.5 (3.0)	-3.4%

So comparing the results within each column we can see:

• 19% performance decline in the Original algorithm when going from 3 → 7 logs/round
• 21% performance decline in the Pipelining algorithm when going from 3 → 14 logs/round

The sample was very noisy but it looks like each log per round is costing around 1-5 % hit in terms of performance.

Action Items

1. A POC PR which let's us validate the performance implications of swapping out logrus for zerolog
2. Decide if we want to allow "plug and play" logger capability (we have heard one community member make this suggestion)
3. Switch all logs in conduit to the new logger (possibly via an interface if we opt for the "plug and play" approach)

More links

• cce's noop logger
• Will's epic on go-algorand
• Google's intro of go 1.21 structured logging and a nice overview of logging in go

Dependencies

None

Urgency

Medium - as we're currently working on improving Conduit's performance, this seems like a useful avenue to persue.