Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(auto-balance): disk in pressure #1003

Merged
merged 3 commits into from
Jul 4, 2024

Conversation

c3y1huang
Copy link
Contributor

Which issue(s) this PR fixes:

Issue longhorn/longhorn#4105

What this PR does / why we need it:

Introduce file local sync to copy the file when there is another running replica on the same node so the data transfer doesn't need to go through TCP.

Special notes for your reviewer:

None

Additional documentation or context

None

shuo-wu
shuo-wu previously approved these changes Feb 5, 2024
Comment on lines +38 to +42
cli.BoolFlag{
Name: "sync-local",
Required: false,
Usage: "sync local replica",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Do we need this if we won't implement local sync for this debug cmdline?

@@ -65,6 +65,12 @@ message FilesSyncRequest {
repeated SyncFileInfo sync_file_info_list = 3;
bool fast_sync = 4;
int32 file_sync_http_client_timeout = 5;
FileLocalSync local_sync = 6;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I am thinking we can reuse the existing fields from_address and to_host for local sync. But a boolean field is required instead. I am fine with the current implementation anyway.

@innobead
Copy link
Member

Need to wait #1015. cc @FrankYang0529

Copy link

mergify bot commented Apr 18, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

1 similar comment
Copy link

mergify bot commented Apr 24, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch 9 times, most recently from 285c0e1 to e22874a Compare June 21, 2024 04:27
Copy link

mergify bot commented Jun 24, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

pkg/types/types.go Outdated Show resolved Hide resolved
pkg/sync/rpc/server.go Outdated Show resolved Hide resolved
Copy link
Member

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flow LGTM. However, we need to implement the sparse file copy rather than use io.Copy. The doc might be helpful.

@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch 5 times, most recently from f5ee325 to b482101 Compare July 2, 2024 04:55
@derekbit
Copy link
Member

derekbit commented Jul 2, 2024

@c3y1huang Is it ready?

longhorn/longhorn-4105

Signed-off-by: Chin-Ya Huang <[email protected]>
@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch from b482101 to a7dc671 Compare July 3, 2024 08:08
@c3y1huang
Copy link
Contributor Author

@derekbit , this is ready for review.

@c3y1huang c3y1huang requested a review from derekbit July 3, 2024 08:16
pkg/sync/rpc/server.go Outdated Show resolved Hide resolved
@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch from a7dc671 to 1c150ab Compare July 4, 2024 00:36
@c3y1huang c3y1huang requested a review from derekbit July 4, 2024 00:39
pkg/sync/rpc/server.go Outdated Show resolved Hide resolved
@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch from 1c150ab to e3ec778 Compare July 4, 2024 00:51
longhorn/longhorn-4105

Signed-off-by: Chin-Ya Huang <[email protected]>
longhorn/longhorn-4105

Signed-off-by: Chin-Ya Huang <[email protected]>
@c3y1huang c3y1huang force-pushed the feat-auto-balance-node-disks branch from e3ec778 to 828da65 Compare July 4, 2024 00:54
@c3y1huang c3y1huang requested a review from derekbit July 4, 2024 00:56
Copy link
Member

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekbit derekbit merged commit 9bf22f2 into longhorn:master Jul 4, 2024
10 checks passed
@c3y1huang c3y1huang deleted the feat-auto-balance-node-disks branch July 4, 2024 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants