[FR] Add library documentation on how to use QUIC

[pataquets]

As per uplink cp --help --advanced, looks like QUIC is already implemented. However, there is neither information or examples on how to use it from Golang. I've looked at the docs, the wiki and the code walkthrough example and found nothing. Is there anything left to check which I've missed?
Adding an example and/or docs would be great.

[pataquets]

Pinging @mniewrzal as commiter of walkthrough. Also, as I've seen elsewhere, this issue also needs "documentation" tag.
If any usable info is posted here, I can attempt to document it ("attempt" due to my limited understanding).

[jtolio]

Hey! I just read through your rclone ticket for background. I'd love to know more about why you're interested in forcing QUIC for your client. Why are raising ulimits a problem? Otherwise, I agree that uplink can be pretty stressful on networks, but there may be some other options available to you that would already work with Uplink.

For more context - we've actually removed the quic flag from the Uplink CLI as part of a broader effort to improve the overall transport layer. https://github.com/storj/storj/blob/main/docs/blueprints/noise.md is now enabled in the latest releases, for instance (and has a lot of good background on what worked and didn't work with QUIC for us). You may also be interested in https://github.com/storj/storj/blob/main/docs/blueprints/tcp-fastopen.md and proposed changes https://review.dev.storj.io/c/storj/storj/+/9933/2/docs/blueprints/tcp-fastopen.md. In short, the transport protocol is currently undergoing some changes, and so there isn't really a stable API to offer.

If what you want is to avoid using the standard TCP congestion controller, you may benefit from the STORJ_SOCKET_LOWPRIO_CTL environment variable, used only on Linux here: https://github.com/storj/common/blob/main/socket/background_linux.go and documented here: https://pkg.go.dev/storj.io/common/socket#BackgroundDialer. This of course requires that the Uplink has been configured to use this background dialer. The way to configure the Uplink for this is changing (https://review.dev.storj.io/c/storj/uplink/+/9859/1/config.go), and may ultimately go away entirely.

Ultimately, we are still researching whether we can use QUIC. It might become the default, or it might go away. I'd love to understand more about what you're trying to solve for, so perhaps we can have something more guaranteed to offer you!

[pataquets]

Thanks for the thorough explanation, @jtolio.

First, my needs currently (at least, as I've scoped in this issue) is mainly for client only, heavy upload workloads. AFAIK, StorJ has this redundancy upload/download penalty and it might consume quite a lot of bandwidth, specially while uploading (rclone for continuous backup scenarios). Also, for my usage pattern, I consider this traffic non-interactive, as opposed to other kinds (SSH, web browsing, streaming) and I want StorJ to always yield to interactive use.

With the above network loads, I prefer to use UDP wherever possible. I'm not interested in any of QUIC features other than it being UDP and congestion control. In most of my scenarios and requirements, UDP gives several advantages:

• UDP/QUIC Traffic is low priority than TCP and it can yield if available bandwidth/throughput suddenly changes. I'm aware of the speed/retry tradeoff, but I'm absolutely fine with it.
• Although solutions exist to locally alleviate this (low priority sockets), they only work in the local host scope and in my case I want this to scale to a LAN with multiple hosts sharing the same uplink, such as shared home and small office scenarios.
• Middleboxes/routers/CPE would also bear way less load in terms of traffic handling and connection keeping, which might be a lifesaver in some scenarios, specially if you can't do anything about it. Even cheap boxes will be able to handle it better, or in the worst case, just drop traffic.
• Somewhat related to the above, limited hardware clients (read RPi's and the likes) would also handle better the loads, I guess.

All the above benefits will have varying impacts on different scenarios, which can also quickly change (laptop moving from home to small office or vice versa). So, explicitly deprioritizing StorJ traffic as much as possible to make it non-interactive and least-effort to all the hops

For instance, I'm currently hosting some distros ISO mirrors via torrent, which in the beginning were affecting interactive traffic and network equipment. Problems went away as soon as I restricted my sharing to just UDP and now I don't notice anything.

Note: I'm currently not participating as a storage provider, nor considered QUIC applicability here, but I'm pretty sure that most of the above would apply in similar scenarios. Not sure, thou.