Tracking issues of Remote file management

[Xuanwo]

This issue is used to track progress of yazi's remote file management.

Discuss at discord: https://discord.com/channels/1136203602898194542/1187669236672368681

Goal

Make yazi a super terminal file manager that can manage remote files.

First of all, users can define new shemas in config:

[schemas]
rules = [
  { name = "backup", type = "s3", bucket = "my-backup-bucket", ... },
  { name = "google", type = "gdrive", referesh_token = "my-token", ... },
  { name = "nextcloud", type = "webdav", endpoint = "http://my-nextcloud-server", ... }
]

Then, users can use yazi to manage them like local files:

# manage remote s3 bucket!
yazi backup://path/to/dir

# manage my gdrive
yazi google://root

# manage my nextcloud
yazi nextcloud://path/to/my/notes

Tasks

• Add shemas in config
• Add remote url scheme in URL
• Build opendal operator and handling io operations

[dyxushuai]

Amazing idea, really like it.

[dyxushuai]

I want to claim this.

[og900aero]

It would be nice if they were also available on the go, so these "drives" could also be called up with a keyboard shortcuts.

[dyxushuai]

It would be nice if they were also available on the go, so these "drives" could also be called up with a keyboard shortcuts.

Opendal already supports Go binding https://github.com/apache/opendal/tree/main/bindings/go

[sxyazi]

Can we implement it using the Go binding?

[Xuanwo]

What's the relation between keyboard shortcuts and go? I would rather directly integrate yazi with rust because the go binding is still in its early stages and it requires CGO and the c binding, which could complicate our cross-platform support efforts.

[danilevy1212]

Would SSH / SFTP / SCP be within scope of this feature? Currently, I use Dired (from emacs) for most of my file editing because of it's remote file management capabilities, through Tramp. If yazi supported this, I would totally switch over, since tramp can be quite slow, but I do have a whole lot of SSH hosts where I routinely do file operations on, so it's just too convenient to leave behind at the moment.

[Xuanwo]

Would SSH / SFTP / SCP be within scope of this feature?

SFTP, Yes.

Other protocols are considered as obsolete that not supported in opendal. See apache/opendal#2643

[danilevy1212]

Thank you! very much looking forward to this.

[Xuanwo]

Hello everyone, I have started my work at #1152.

Current progress: opendal has been integrated into yazi. I'm working out the layout of yazi to make it works correctly.

[uxfion]

Is it better to integrate through the plugin rclone? because rclone is mature and stable enough to avoid reinventing the wheel

[choovick]

Another user that was brought up on Norton Commander then Total Commander then Midnight Commander. Shell link is a must for me as well, with this feature MC and be uninstalled. MC URI schema sh://user@host_or_ip[:port]/path

[kkumlien]

Can't even start using Yazi without fully integrated, remote file management MC-style (or similar). Keep up the good work!

[planetrobbie]

Same this is a pretty useful feature that I use everyday with MC.

[BLucky-gh]

+1 on the rclone suggestion. Most people who use a lot of remote storages would already know rclone and it's config, and would have some stuff already configured, making it trivial to copy over the config to yazi, or even have yazi import it, on top of the fact that it's already battle tested and mature, reducing maintenance burden on yazi devs

[Nicknamely]

I wanted to share that I have successfully set up rclone with OneDrive on my machine using the VFS feature. This allows me to access files from my OneDrive without downloading them entirely. The files appear in my file manager, and even though they are not physically downloaded, I can see them, and their metadata is available through the VFS cache.

I understand that with the --vfs-cache-mode writes option, files are downloaded on-demand when accessed (in the background) and cached locally for performance, but not necessarily stored permanently unless explicitly specified.

My question is, since Yazi has a preview feature for files, does it download the file in order to preview it? As I know that files are automatically downloaded when accessed via the VFS, I'm curious if Yazi's preview mechanism behaves similarly or if it leverages any additional caching or methods to provide previews without fully downloading the file.

Overview of the Setup

Rclone Configuration

rclone --vfs-cache-mode writes --vfs-cache-max-age 24h --vfs-cache-max-size 10G mount OneDrive-Personal: ~/Documents/OneDrive-personal/

• VFS Mode: writes (Files are cached and written to the remote storage when modified)
• Cache Max Age: Files will stay in cache for a maximum of 24 hours before being evicted.
• Cache Max Size: The cache will be limited to 10GB.

Auto Start

Instead of using a system service (and dealing with systemd ), I have hyprland start it.

exec-once = rclone --vfs-cache-mode writes --vfs-cache-max-age 24h --vfs-cache-max-size 10G mount OneDrive-Personal: ~/Documents/OneDrive-personal/ &