Speeding up initial synchronisation

[wout]

First of all, thank you for building Carp! Already it feels a lot more stable than db-sync. I'm looking forward to using it for my upcoming project on Cardano.

Since last Tuesday, the node is doing the initial sync and it's steady but slow. I've got a 16 vCPU / 32GB server and I can see not much of the CPU being used most of the time.

Is there a way to assign more resources to Carp to speed up the process? I'm running everything through docker-compose.

[SebastienGllmt]

We'll be updating to Oura which has faster N2C syncing so it get a bit faster for mainnet and much faster for testnet. Other than that, you can also check to see if you really need all the tasks you have in your execution plan. The more tasks you remove, the faster it will get

We also have two Catalyst proposals that you may be interested for some other ideas we have to speed things up:

1. Funding continued work on Carp link
2. Fast re-indexable block format so Carp can quickly re-index data from blocks you've already synchronized link

[wout]

I'm syncing with the default tasks, so I'll sit this one out and fine-tune it later. Thanks again!

PS: I will vote.

[pablosymc]

@SebastienGllmt i'm assuming the slow sync is more related to the database operations rather than pulling data from Oura yes? I noticed that all database indexes are created at time of the migration, before we start syncing. Would deferring some of the index creation (until the tip is reached for example) speed up sync time?