Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expire revisions on remote servers #37

Merged
merged 13 commits into from
Apr 11, 2024
Merged

expire revisions on remote servers #37

merged 13 commits into from
Apr 11, 2024

Conversation

dhnasa
Copy link
Collaborator

@dhnasa dhnasa commented Apr 14, 2023

depends on #62

closes #30, PL-131863

  • Change is documented in changelog
  • Support backup job migration across servers

  • Add tags {set, add, remove} subcommand

  • Add expire subcommand

  • logging: improve exception formatting

  • logging: add taskid

  • Add push and pull subcommand

  • Add server: selector to revision spec

  • Coordinate backups for the same job between servers
    The server with the largest number of local revisions is the leader.
    If the leader is offline another server will take over.

    This assumes that all servers share the same rng and thus
    schedule the backup at the same time. If this is not the case more
    backups than necessary may be created (but not less).

Security implications

  • Security requirements defined? (WHERE)
    • Minimise attack surface area: api access requires api key
    • Establish secure defaults: missing api key(s) prevent api access
    • Fail securely: invalid data does not crash the daemon (but may crash the backup process indefinitely)
    • Fail securely: network failure/unresponsive remote does not prevent backup (existing data remains valid until the remote server is removed from the config file)
    • Principle of Least privilege: only "dead" backups can be changed by remote servers
    • Don't trust services: malicious backy server can prevent new backups on other servers (but this is by design?, maybe add some plausibility checks e.g. no backups in the future? or timeouts when waiting for the leader to finish)
  • Security requirements tested? (EVIDENCE)
    • unit tests

@dhnasa dhnasa force-pushed the issue-30-server-migration branch 4 times, most recently from 7db7b46 to 5353686 Compare May 22, 2023 14:00
@dhnasa dhnasa mentioned this pull request Jun 19, 2023
3 tasks
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 3 times, most recently from fc5a31d to 247e352 Compare June 26, 2023 00:16
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 2 times, most recently from d82d828 to 705ca3d Compare July 13, 2023 02:15
@dhnasa dhnasa force-pushed the issue-30-server-migration branch from 705ca3d to 8241928 Compare October 9, 2023 22:47
@dhnasa dhnasa marked this pull request as ready for review October 9, 2023 22:52
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 4 times, most recently from 66bef16 to dc4fab5 Compare October 19, 2023 18:54
@dhnasa dhnasa force-pushed the issue-30-server-migration branch from dc4fab5 to bd80eca Compare November 6, 2023 13:42
@dhnasa dhnasa force-pushed the issue-30-server-migration branch from d1a4c51 to bb1bcad Compare November 17, 2023 21:43
@dhnasa dhnasa mentioned this pull request Nov 27, 2023
1 task
@dhnasa dhnasa force-pushed the issue-30-server-migration branch from bb1bcad to eaf19de Compare November 27, 2023 17:48
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 2 times, most recently from 2c3e6f2 to f452ead Compare December 8, 2023 14:59
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 2 times, most recently from 54cf114 to 6ab69ac Compare February 5, 2024 00:25
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 2 times, most recently from bb12f85 to 38594a6 Compare April 2, 2024 11:21
ctheune
ctheune previously approved these changes Apr 10, 2024
doc/man-backy.rst Outdated Show resolved Hide resolved
src/backy/api.py Outdated Show resolved Hide resolved
src/backy/api.py Outdated Show resolved Hide resolved
src/backy/client.py Outdated Show resolved Hide resolved
src/backy/client.py Outdated Show resolved Hide resolved
src/backy/api.py Outdated Show resolved Hide resolved
src/backy/api.py Outdated Show resolved Hide resolved
src/backy/api.py Outdated Show resolved Hide resolved
src/backy/api.py Show resolved Hide resolved
src/backy/daemon.py Outdated Show resolved Hide resolved
Johann Bahl added 9 commits April 11, 2024 13:16
Not adding migrations for rev files as there was no release between the
introduction of locations and this commit
The server with the largest number of local revisions is the leader.
If the leader is offline another server will take over.

This assumes that all servers share the same rng and thus
schedule the backup at the same time. If this is not the case more
backups than necessary may be created (but not less).
@dhnasa dhnasa force-pushed the issue-30-server-migration branch 2 times, most recently from e54e034 to 8538072 Compare April 11, 2024 12:00
@dhnasa dhnasa force-pushed the issue-30-server-migration branch from 8538072 to 6ccad75 Compare April 11, 2024 13:06
@ctheune ctheune merged commit 752c76f into main Apr 11, 2024
3 checks passed
@dhnasa dhnasa deleted the issue-30-server-migration branch April 11, 2024 17:00
dhnasa pushed a commit that referenced this pull request Jun 12, 2024
This was already the case (due to a bug) for a long time, but was
recently fixed in #37.
This commit reintroduces this behaviour.

The reasoning is that a backup which takes longer than 50% (grace
period) of the configured interval indicates a problem with the backup
server/network/etc...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support migration of backups on multiple servers
2 participants