Initial wallet sync is slow

[Technetium1]

• Version: Sugarchain Yumekawa version v0.16.3.35-b88f6d2 (64-bit)
• OS: Windows
• CPU: Ryzen 3900XT
• Net: 500 Mbit down / 25 Mbit up
• Possibly related issue: Question: increase HTTP THREADS? #94
• Possibly relevant code:

  sugarchain/src/httpserver.h

  Lines 12 to 14 in 7db380d

  	static const int DEFAULT_HTTP_THREADS=4;
  	static const int DEFAULT_HTTP_WORKQUEUE=128; // was 16 // FIXME.SUGAR // for huge RPC calling
  	static const int DEFAULT_HTTP_SERVER_TIMEOUT=30;

I would like to know if there is a way to get a better speed for the initial synchronization. I've been at it for 2.5 days and it's still only getting around ~5.5% (maximum) of the chain per hour. There is next to no CPU utilization even with the process at real-time priority, so it seems that it's limited by the number of peers. I have had somewhere between 3 and 8 connected continuously.

There doesn't seem to be an explanation of all the possible parameters in sugarchain.conf, unless I'm missing the obvious. I would very much appreciate a link if it exists somewhere, and it would make sense to add the link to this popup:

Is there a possibility of an alternative version of this for those without a bandwidth cap or limited CPU power? https://github.com/sugarchain-project/sugarchain/blob/master-v0.16.3/doc/reduce-traffic.md

[decryp2kanon]

There is next to no CPU utilization even with the process at real-time priority, so it seems that it's limited by the number of peers.

actually it takes long to download header. you can make it faster, but it cause too much data traffics. increasing peers is not helped.

IBD is two part, first download headers. second sync headers with network. during sync there is a bottleneck PoW check. however we should not do bypass this verification. but somehow you can bypass this PoW check with comment out the source code.

the reason of slow IBD is two.

1. yespower PoW check is much slower than sha256 because its multi threading resistance against GPU and ASIC.
2. sugar has many historical blocks. its much longer chain than even BTC.

shorting IBD time is our one of main plan. welcome any suggestions.

[Technetium1]

Taking all of that information into account makes me believe #177 is related. What do you mean specifically by it cause too much data traffics? Is this because the size of headers is large, or because there are many requests? Is there a chance to bypass PoW check initially and queue it to be checked in the background after IBD is completed?

An idea for increasing the speed of gathering historical blocks... Allow nodes to optionally send compressed chunks of historical blocks if the request is for a range reasonably far back in the chain. Would it not be reasonable to compress 10-50k (more? 100k?) blocks at a time to save bandwidth at the cost of CPU? Surely some users would prefer to have a choice in saving bandwidth at a cost of CPU, especially with the multithreading of modern systems and the increase in data cap costs. It should certainly checksum the result of the received data; to reduce the need for a later request in case of failure from of any inconsistencies in transmission. If PoW is still done on the compressed data, there should be no problem sending it as large chunks.

I think that any reduction in traffic required is a victory, I only wish I were able to offer a proof of concept.

[decryp2kanon]

What do you mean specifically by it cause too much data traffics?

#78

we disabled the more getheaders due to too much 15x data traffic. i guess its an issue on BTC source code. they need to optimize this.

check this too
#122

[decryp2kanon]

#207