feat(io): compare actual size in CheckIsFileSizeSame

[c3y1huang]

Which issue(s) this PR fixes:

Issue longhorn/longhorn#4105

What this PR does / why we need it:

longhorn/longhorn-engine#1003 (comment)

Special notes for your reviewer:

Additional documentation or context

[codecov]

Codecov Report

Attention: Patch coverage is 62.50000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 78.93%. Comparing base (d78642c) to head (949a952).

Files	Patch %	Lines
io/file.go	62.50%	3 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
- Coverage   79.21%   78.93%   -0.29%     
==========================================
  Files          32       32              
  Lines        1506     1519      +13     
==========================================
+ Hits         1193     1199       +6     
- Misses        200      204       +4     
- Partials      113      116       +3     

Flag	Coverage Δ
unittests	78.93% <62.50%> (-0.29%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[c3y1huang]

I was expecting this TestCopyFile test case to fail after this PR due to this and this, but it didn't. I am not sure why. @derekbit any idea?

[c3y1huang]

I cannot see how copySparseData in the unit test handles sparse data differently from io.Copy.

Achieves by ready and writing by intervals.

[derekbit]

I was expecting this TestCopyFile test case to fail after this PR due to this and this, but it didn't. I am not sure why. @derekbit any idea?

IIUC, the test case creates a temp file with a string "content" and truncate the file to 20 bytes, so the file content is
'c' 'o' 'n' 't' 'e' 'n' 't' '\x00' .... '\x00'. Then, copy the content to the destination file.
Can you elaborate more on that why you think the test case should fail?

[derekbit]

LGTM

[c3y1huang]

IIUC, the test case creates a temp file with a string "content" and truncate the file to 20 bytes, so the file content is
'c' 'o' 'n' 't' 'e' 'n' 't' '\x00' .... '\x00'. Then, copy the content to the destination file.
Can you elaborate more on that why you think the test case should fail?

I’m a bit confused. I was expecting it to fail because we are now comparing actual sizes, and io.Copy doesn't support sparse file copy?

longhorn/longhorn-engine#1003 (comment)

[derekbit]

IIUC, the test case creates a temp file with a string "content" and truncate the file to 20 bytes, so the file content is
'c' 'o' 'n' 't' 'e' 'n' 't' '\x00' .... '\x00'. Then, copy the content to the destination file.
Can you elaborate more on that why you think the test case should fail?

I’m a bit confused. I was expecting it to fail because we are now comparing actual sizes, and io.Copy doesn't support sparse file copy?

longhorn/longhorn-engine#1003 (comment)

I see.
The size of the sparse file is only 20 bytes. A file is composed of blocks, with each block typically being 512 bytes or 4096 bytes. Although the file's apparent size is 20 bytes, it actually occupies 8 blocks on the disk. This results in a total disk usage of 8 blocks x 512 bytes (disk block size). Therefore, the file is not sparse.

Note: Blocks is the number of blocks allocated on the disk, and the disk block size is 512 bytes. IO Block is the block size of the filesystem.

# echo content > test
# truncate --size 20 test
# stat test
  File: test
  Size: 20        	Blocks: 8          IO Block: 4096   regular file
Device: fd00h/64768d	Inode: 3407888     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-07-01 15:09:03.514369106 +0000
Modify: 2024-07-01 15:10:02.550371344 +0000
Change: 2024-07-01 15:10:02.550371344 +0000
 Birth: 2024-07-01 15:09:03.514369106 +0000

You can try

• Write string "content"
• Truncate the file to 4097 bytes
  Then, the file should be sparse. You can see the actual size is 8 blocks x 512 bytes (disk block size), but the apparent size is 4097.

# echo content > test
# truncate --size 4097 test
# stat test
  File: test
  Size: 4097      	Blocks: 8          IO Block: 4096   regular file
Device: fd00h/64768d	Inode: 3407888     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-07-01 15:09:03.514369106 +0000
Modify: 2024-07-01 15:16:08.726461833 +0000
Change: 2024-07-01 15:16:08.726461833 +0000
 Birth: 2024-07-01 15:09:03.514369106 +0000

[c3y1huang]

I see. The size of the sparse file is only 20 bytes. A file is composed of blocks, with each block typically being 512 bytes or 4096 bytes. Although the file's apparent size is 20 bytes, it actually occupies 8 blocks on the disk. This results in a total disk usage of 8 blocks x 512 bytes (disk block size). Therefore, the file is not sparse.

Note: Blocks is the number of blocks allocated on the disk, and the disk block size is 512 bytes. IO Block is the block size of the filesystem.

# echo content > test
# truncate --size 20 test
# stat test
  File: test
  Size: 20        	Blocks: 8          IO Block: 4096   regular file
Device: fd00h/64768d	Inode: 3407888     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-07-01 15:09:03.514369106 +0000
Modify: 2024-07-01 15:10:02.550371344 +0000
Change: 2024-07-01 15:10:02.550371344 +0000
 Birth: 2024-07-01 15:09:03.514369106 +0000

You can try

• Write string "content"
• Truncate the file to 4097 bytes
  Then, the file should be sparse. You can see the actual size is 8 blocks x 512 bytes (disk block size), but the apparent size is 4097.

# echo content > test
# truncate --size 4097 test
# stat test
  File: test
  Size: 4097      	Blocks: 8          IO Block: 4096   regular file
Device: fd00h/64768d	Inode: 3407888     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-07-01 15:09:03.514369106 +0000
Modify: 2024-07-01 15:16:08.726461833 +0000
Change: 2024-07-01 15:16:08.726461833 +0000
 Birth: 2024-07-01 15:09:03.514369106 +0000

Thank you for the explanation, it's clear to me now.

[derekbit]

LGTM