Reducing the role of RGB Node

[dr-orlovsky]

Orignaly, RGB Node was created with the idea of performing P2P communications for client-side-validated data. However, with the later distinction of data transfer from the core of RGB protocol, and support for multiple data transfer solutions (LN/Storm, relays etc) the need in the dedicated node got much reduced.

Currently, to do nearly anything related to RGB, like validate a consignment or pepare a payment, one needs to run RGB Node, which manages stash. This puts high resource requirements, which, in case of mobile or hardware wallets may be too demanding. At the same time, some of the operations carried by RGB Node (like consignment validation) doens't require access to the stash - while others can be performed just by accessing a disk-stored stash.

TL;DR: in most of the cases there is no need for a runtime which is constantly run in memory. A git-like solution working with disk storage in some directory may be enough to perform all the tasks which are needed for mobile, hardware wallets and, in many cases, desktop.

The only situation where a constantly running runtime is needed is a server to do all tasks remotely - for instance to reduce consuming of mobile traffic in roaming when large consignments are involved they may be processed by a server. However this server may be a tiny layer of networking on top of the same library which is used on mobile.

The poposal

Move most of the RGB Node functionality into RGB standard library and make it not requiring networking, permanent runtime, multithreading etc; giving much flexibility to wallet and software developers (like gitlib). Transform RGB node into a small server which uses this library and provides networking RPC API to it.

[cryptoquick]

This is an excellent idea, and it's basically proved out by the work we've done in bitmask-core. I'd love to be able to deprecate portions of the RGB sections if we can get the same functionality in the RGB std lib.

[dr-orlovsky]

Ok so stored will not be replaced with another project, it will get deprecated, and for storage operations rgb-std methods will get a path to a data directory where a git-based storage will be used. Is my understanding correct now @dr-orlovsky?

Correct except that stored will not be "depricated" but just not used in RGB (but still can be used in other projects if needed).

Why can't we keep using sled to save data?

I haven't said anything about the format of the on-disk data storage. This in fact can be anything - from what we have today (sled) to LLDB or anything else which we think would work. The considerations I have on this matter are:

1. I prefer not do write a lot of new storage-related code, so yes sled is the choice
2. It has to be not a C library but rather something rust native
3. It has to work well with WASM (?)
4. It has to work on mobile
5. It has to work on embeded platforms

Has a migration for existing nodes been considered? If so, has the time to write this migration been considered when you said that's "not a huge work"?

I was assuming we continue using the same storage format, since I do not see clear reason to change it at least in parallel with moving from RGB Node to git-like approach. But if you see a reason to do that that has to be estimated separately, and it will not be related to the topic. Also it has to be better on the criteria 1-5 above than the current solution.

Talking instead about RGB-WG/rgb-node#222, I think that in the long term it will still be necessary

I am not getting what do you mean. What will be necessary? Stopping the thread? But there would be no threads in the library; all threads will be managed by a wallet. You need to stop it - you stop it.

[zoedberg]

I was assuming we continue using the same storage format

Happy to hear this. Sorry for my misunderstading

I am not getting what do you mean. What will be necessary? Stopping the thread? But there would be no threads in the library; all threads will be managed by a wallet. You need to stop it - you stop it.

Yes, in the future we will need to stop spawned threads for lnp-node and storm-node (that, I guess for their nature must remain daemons). As discussed in RGB-WG/rgb-node#222 and also in a dev call there is no simple way to stop a spawned thread in Rust and for that we will need a different thread managing architecture

[dr-orlovsky]

This is unrelated to the proposed changes to RGB Node, right?

As for LNP and Storm the new architecture is outthere already: check github.com/rust0amplify/io-reactor. It provides a way to shutdown the thread and launch it over again later. It is also capapble of managing thousands of connections on the same thread in a non-blocking way without use of rust async, i.e. much more lightweight

[zoedberg]

This is unrelated to the proposed changes to RGB Node, right?

Yes, it's unrelated. Talking about proposed changes I'd say that if doing this refactor along with the consesus breaking changes helps reducing your work in a significant way I'm ok with it, if instead it's not so time-impacting I'd try to split work in 2 releases in order to facilitate wallets/libraries developers life

As for LNP and Storm the new architecture is outthere already

Wow, this is great news! In which release do you think we're going to be able to change the runtime architecture in lnp-node and storm-node?

[dr-orlovsky]

In which release do you think we're going to be able to change the runtime architecture in lnp-node and storm-node?

I'd like to see it happening in v0.10, which now for LNP/Storm would be able to happen independently from RGB release (i.e. later).

IO reactor is production-ready (it was tested even outside of LNP/BP projects), so it is just a question of switching to it.