Question on --device parameter

[ubenmackin]

My BTRFS setup is RAID10, so I don't have a single device tied to my BTRFS array. I wasn't sure what I am supposed to pass in this case. Do I just pass in any device that is part of the array for the file/folder I am trying to dedupe?

For example, if I want to dedupe everything in '/mnt/ddimages', and that is part of a BTRFS pool that is comprised of 4 disks:

/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd

Would I use the command:

dduper --device /dev/sda --dir /mnt/ddimages

Will that work as expected? Or will it somehow only dedupe the files that BTRFS has stored on /dev/sda? Do I then need to run the command 4 times, once for each device?

I guess my question is why do we need to specify --device at all? Isn't that something that can be determined based on the mount that holds the file/folders specified?

[Lakshmipathi]

Hi @ubenmackin

TBH, I don't know the expected results :P I opened this bug #12 to validate the behaviour with RAID setup. Let me check them and update by this weekend.

I guess my question is why do we need to specify --device at all? Isn't that something that can be determined based on the mount that holds the file/folders specified?

Right, this is the ideal case, just passing the mount and let the tool to determine the devices. I need to re-work on this tool.

[Lakshmipathi]

I did quick test on RAID10. Below are the results. In short, you can use any one of the disk with --device option. That should work. Please check with test environment before running on critical data.

$ sudo btrfs fi show /mnt
Label: none  uuid: 78e7c726-b7dc-41b2-9538-840c36ac757e
	Total devices 4 FS bytes used 2.25MiB
	devid    1 size 512.00MiB used 123.19MiB path /dev/loop13
	devid    2 size 512.00MiB used 123.19MiB path /dev/loop14
	devid    3 size 512.00MiB used 123.19MiB path /dev/loop15
	devid    4 size 512.00MiB used 123.19MiB path /dev/loop16


$ sudo btrfs fi du /mnt/f1 /mnt/f2
     Total   Exclusive  Set shared  Filename
   2.00MiB     2.00MiB       0.00B  /mnt/f1
   2.00MiB     2.00MiB       0.00B  /mnt/f2


## copied two files to mount point
$ sudo cp /tmp/s1 /mnt/f2
$ sudo cp /tmp/s1 /mnt/f1


$ df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop13     1.0G  7.5M  900M   1% /mnt

$sudo ./dduper --device /dev/loop13 --dir /mnt/ 
Perfect match :  /mnt/f1 /mnt/f2
************************
Dedupe completed for /mnt/f1:/mnt/f2
Summary
blk_size : 4KB  chunksize : 2048KB
/mnt/f1 has 1 chunks
/mnt/f2 has 1 chunks
Matched chunks: 1
Unmatched chunks: 0
Total size(KB) deduped: 2048
dduper took 1.083340496999881 seconds


$ sync
$ df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop13     1.0G  5.5M  902M   1% /mnt


$sudo btrfs fi du /mnt/f1  /mnt/f2
     Total   Exclusive  Set shared  Filename
   2.00MiB       0.00B     2.00MiB  /mnt/f1
   2.00MiB       0.00B     2.00MiB  /mnt/f2

Try running it with dry-run first:

dduper --device /dev/sda --dir /mnt/ddimages --dry-run

hopefully it show display matched chunks. You can also use --analyze option on specific files like

dduper --device /dev/sda --files /mnt/ddimages/filename1 /mnt/ddimages/filename2 --analyze