Beyond a certain limit blockstat crashes with access violation

[AlexHeylin]

Firstly - thanks for this very useful project!

This is almost certainly the same issue as #3
In my environment over 31 files causes blockstat to run for a while then crashes (appcrash logged) without any output, not even an error. Given the huge file count difference between my issue and #3 I suspect this memory allocation related, and tied to number of blocks or something similar.

To aid tracking, here's redacted output of the 31 file run which is the largest that works for me in this directory

Comparing Mode
Fsinfo D:\ clustersize 65536 clusters 824056832
Files:
<redacted list of 31 *.vbk files>

Sharing:
	- 1 x 	 1901140836352 bytes 1813069 mb
	- 30 x 	 183042048 bytes 174 mb
	- 31 x 	 186974208 bytes 178 mb
	- 32 x 	 25296896 bytes 24 mb
	- 33 x 	 200802304 bytes 191 mb
	- 34 x 	 90243072 bytes 86 mb
	- 35 x 	 274137088 bytes 261 mb
	- 36 x 	 238682112 bytes 227 mb
	- 37 x 	 48627712 bytes 46 mb
	- 38 x 	 125829120 bytes 120 mb
	- 39 x 	 23789568 bytes 22 mb
	- 40 x 	 153485312 bytes 146 mb
	- 41 x 	 190906368 bytes 182 mb
	- 42 x 	 39714816 bytes 37 mb
	- 43 x 	 252051456 bytes 240 mb
	- 44 x 	 10092544 bytes 9 mb
	- 45 x 	 103809024 bytes 99 mb
	- 155 x 	 103809024 bytes 99 mb
	- 156 x 	 5242880 bytes 5 mb
	- 157 x 	 142606336 bytes 136 mb
	- 158 x 	 22675456 bytes 21 mb
	- 159 x 	 17170432 bytes 16 mb
	- 160 x 	 7340032 bytes 7 mb
	- 161 x 	 38797312 bytes 37 mb
	- 162 x 	 28311552 bytes 27 mb
	- 163 x 	 29687808 bytes 28 mb
	- 164 x 	 10027008 bytes 9 mb
	- 165 x 	 6422528 bytes 6 mb
	- 166 x 	 35651584 bytes 34 mb
	- 167 x 	 10813440 bytes 10 mb
	- 168 x 	 138018816 bytes 131 mb
	- 169 x 	 10551296 bytes 10 mb
	- 170 x 	 49283072 bytes 47 mb
	- 171 x 	 65011712 bytes 62 mb
	- 172 x 	 28639232 bytes 27 mb
	- 173 x 	 11206656 bytes 10 mb
	- 174 x 	 5177344 bytes 4 mb
	- 175 x 	 7405568 bytes 7 mb
	- 176 x 	 12845056 bytes 12 mb
	- 177 x 	 72679424 bytes 69 mb
	- 178 x 	 15138816 bytes 14 mb
	- 179 x 	 25165824 bytes 24 mb
	- 180 x 	 12582912 bytes 12 mb
	- 181 x 	 67829760 bytes 64 mb
	- 182 x 	 18153472 bytes 17 mb
	- 183 x 	 67108864 bytes 64 mb
	- 184 x 	 393216 bytes 0 mb
	- 185 x 	 7995392 bytes 7 mb
	- 188 x 	 4980736 bytes 4 mb
	- 189 x 	 3407872 bytes 3 mb
	- 190 x 	 10289152 bytes 9 mb
	- 191 x 	 32702464 bytes 31 mb
	- 192 x 	 24117248 bytes 23 mb
	- 193 x 	 19726336 bytes 18 mb
	- 194 x 	 18022400 bytes 17 mb
	- 195 x 	 24969216 bytes 23 mb
	- 196 x 	 23265280 bytes 22 mb
	- 197 x 	 20250624 bytes 19 mb
	- 198 x 	 17498112 bytes 16 mb
	- 199 x 	 15138816 bytes 14 mb
	- 200 x 	 72679424 bytes 69 mb
	- 201 x 	 25427968 bytes 24 mb
	- 202 x 	 39845888 bytes 38 mb
	- 203 x 	 35061760 bytes 33 mb
	- 204 x 	 16384000 bytes 15 mb
	- 205 x 	 589824 bytes 0 mb
	- 206 x 	 12976128 bytes 12 mb
	- 207 x 	 27131904 bytes 25 mb
	- 208 x 	 7733248 bytes 7 mb
	- 209 x 	 14417920 bytes 13 mb
	- 210 x 	 40763392 bytes 38 mb
	- 211 x 	 3801088 bytes 3 mb
	- 212 x 	 78512128 bytes 74 mb
	- 213 x 	 19988480 bytes 19 mb
	- 214 x 	 16318464 bytes 15 mb
	- 215 x 	 2818048 bytes 2 mb
	- 216 x 	 32833536 bytes 31 mb
	- 217 x 	 16777216 bytes 16 mb
	- 218 x 	 13107200 bytes 12 mb
	- 219 x 	 12189696 bytes 11 mb
	- 220 x 	 32374784 bytes 30 mb
	- 221 x 	 26738688 bytes 25 mb
	- 222 x 	 12058624 bytes 11 mb
	- 223 x 	 7340032 bytes 7 mb
	- 224 x 	 34340864 bytes 32 mb
	- 225 x 	 5505024 bytes 5 mb
	- 226 x 	 25755648 bytes 24 mb
	- 227 x 	 18415616 bytes 17 mb
	- 228 x 	 63504384 bytes 60 mb
	- 229 x 	 13565952 bytes 12 mb
	- 230 x 	 21364736 bytes 20 mb
	- 231 x 	 23920640 bytes 22 mb
	- 232 x 	 8388608 bytes 8 mb
	- 233 x 	 16842752 bytes 16 mb
	- 234 x 	 1703936 bytes 1 mb
	- 235 x 	 6684672 bytes 6 mb
	- 236 x 	 131072 bytes 0 mb
	- 237 x 	 17825792 bytes 17 mb
	- 238 x 	 7274496 bytes 6 mb
	- 239 x 	 26214400 bytes 25 mb

Total Savings 483011919872 (460636 mb)
Total Fragments Over All Files 497064

I don't know if it makes much difference, but it's quite likely a good number of these are not using block-clone

Here's the event Windows logs on app crash

Faulting application name: blockstat.exe, version: 0.0.0.0, time stamp: 0x58f0d6c3
Faulting module name: ntdll.dll, version: 10.0.17763.1192, time stamp: 0x5854f5da
Exception code: 0xc0000005
Fault offset: 0x000000000001250d
Faulting process id: 0x21e0
Faulting application start time: 0x01d6430d6a45bba6
Faulting application path: C:\Users\Administrator\Desktop\blockstat\blockstat.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 75c241d1-43dc-41d4-ab02-fb6975d2a34c
Faulting package full name: 
Faulting package-relative application ID: 

Normally I'd aim to resolve this by sending you a PR, but C++ is a language I'm really not good at, so I might need some help with this one please.

Thanks

[tdewin]

Could you maybe check with -v verbose to see if it always crashes on the same file. Also, could you let me know if the memory keeps growing after let's say 1 minute? Normally it does consume a lot of memory but that should be done at the very beginning as it allocates a big table for counting references

I'm more worried that it is some kind of corruption you are hitting in the memory (but I guess a memtest might be too much to ask? )