Fix backup last mysql binlog

[terranus]

Database name

MySQL

Pull request description

Last binlog wasn't backup at all by skipping at 'until check'. Last binlog may contain up to several hours of modified data. So there can be data loss on PiTR restore to the latest time point.

Describe what this PR fix

Fix backing up latest (aka current) MySQL binlog. Backing up only if:

• backing up for the first time
• binlog size is greater then saved size at wal-g cache

Please provide steps to reproduce (if it's a bug)

Just run

wal-g binlog-push

Please add config and wal-g stdout/stderr logs for debug purpose

If you can, provide logs

# WALG_LOG_LEVEL=DEVEL wal-g binlog-push
[...]
DEBUG: 2024/01/18 13:42:19.742645 Testing... binlog.000001
DEBUG: 2024/01/18 13:42:19.742661 Skip binlog binlog.000001 (archived binlog check)
DEBUG: 2024/01/18 13:42:19.742665 Testing... binlog.000002
DEBUG: 2024/01/18 13:42:19.742673 Skip binlog binlog.000002 (archived binlog check)
DEBUG: 2024/01/18 13:42:19.742681 Testing... binlog.000003
DEBUG: 2024/01/18 13:42:19.742684 Skip binlog binlog.000003 (archived binlog check)
DEBUG: 2024/01/18 13:42:19.742688 Testing... binlog.000004
DEBUG: 2024/01/18 13:42:19.742696 Skip binlog binlog.000004 (until check)
# ls -la /var/lib/mysql/binlog*
-rw-r----- 1 mysql mysql 1073606695 Jan 16 00:00 /var/lib/mysql/binlog.000001
-rw-r----- 1 mysql mysql        201 Jan 17 00:00 /var/lib/mysql/binlog.000002
-rw-r----- 1 mysql mysql        201 Jan 18 00:00 /var/lib/mysql/binlog.000003
-rw-r----- 1 mysql mysql       9283 Jan 18 13:24 /var/lib/mysql/binlog.000004
-rw-r----- 1 mysql mysql         64 Jan 18 00:00 /var/lib/mysql/binlog.index

[ostinru]

Current behavior reasoning

You are right - current binlog never backed up and this can lead to lost transactions. However, in my practice, it is rarely the case for multi-node clusters: after switchover you will be able to backup binlogs from the new master and no transaction will be lost.
// I don't consider single-node cluster as HA-clusters, and data lost in non-HA-clusters is Ok. Isn't it?

As for your patch...

I think that you can backup open binlog file in the middle of a transaction. And it should be proven that such binlogs will not break PiTR.

I know 3 ways to do PiTR for MySQL:

1. if you use 'Like a rock-star'[2] approach - MySQL will recognize [1] that this binlog wasn't closed properly and fix it.
2. if you use classic mysqlbinlog | mysql approach - you should also pass --force-if-open option to mysqlbinlog, and it seems[3] that it will skip truncated transaction.
3. if you use binlog-server approach - probably MySQL should handle such cases well. However, I don't know how it will work with wal-g when we will try to apply binlogs from multiple masters (and several of them will not be closed properly).

[1] https://github.com/mysql/mysql-server/blob/8.0/libbinlogevents/include/binlog_event.h#L248-L269
[2] https://lefred.be/content/howto-make-mysql-point-in-time-recovery-faster/
[3] https://github.com/mysql/mysql-server/blob/8.0/client/mysqlbinlog.cc#L3075-L3088

Summary

I think that this approach can work :) However, this should be well-tested before merging to master.

IMO, this feature should be hidden behind the flag (like wal-g binlog-push --force-if-open) to make conservative people happy.

PS: Will be glad to hear other wal-g MySQL maintainers' thoughts.