Allow rsync to flush written data before returning

[terrorobe]

rsync currently does not flush any written data to disk before exiting. This leaves a time window where data can be lost on the destination in case of storage problems or host crashes.

Would you be open to PRs adding a --flush and/or --flush-after option that instructs rsync to flush the written data after each file and after the transfer of all files respectively? The latter option would have the upside of adding little IO overhead for data heavy rsync runs while still ensuring that the copied data is durable on the receiving side.

I saw that you wrote up a patch that implements the first approach a few years ago but unfortunately it never ended up being merged.

[WayneD]

I haven't really wanted to add a --make-rsync-go-really-really-super-extra-slow option, but since it's such a simple change, I'll give it another look soon.

[terrorobe]

For modern storage hardware the performance impact should be negligible, modern SSDs have write latencies in the 50-100 microsecond range.

Opening the files with O_(D)SYNC would probably have the biggest performance overhead; the amount of work flushing the files after the transfer using f(data)sync needs to do depends on transfer speed between the rsync processes and flush interval of the VFS layer; using f(data)sync on all changed files after the transfer is finished should mostly exhibit syscall overhead, not touching the storage for the majority of operations (again, dependent on transfer speed and flush interval).

I think most rsync users would be happy to be able to pick the performance tradeoff in favor of data durability, since I assume quite a few of them use rsync in situations where data durability is required and rsync's exit codes are taken as gospel. This includes this website right here ;)

[WayneD]

I added the --fsync option, promoting it from patches with a few improvements.

[terrorobe]

Thank you! 🙇