SignalDB's local-first (lofi) story?

[beorn]

I'm looking for a local-first (synced local/remote) database and I like SignalDB, but I'm not sure about how to think about it in terms of a local-first scenario.

If you have an app with multiple users and a lot of data (that doesn't fit in memory):

• each time the client connects it needs to exchange only the changes with the server (currently it seems everything is downloaded?)
• you probably don't want to keep the entire database in-memory (but instead in a local db like sqlite or pglite in wasm)
• you probably want to only sync parts of the data (rows/columns) depending on the user and their permissions

There are some notes on the RxDB example on how SignalDB and RxDB work together and compliment each other, but it seems like there's a lot of overlap in the features they provide.

I was thinking perhaps SignalDB could be extended to become a local-first database with a locally cached db that can be queried, and this again is partially synced with a remote database. Does this seem out-of-scope? In this kind of scenario what would be a good role for SignalDB to play?

Some comparable technologies:

• zerosync
• powersync
• electric-sql
• instantdb

[maxnowack]

Currently, SignalDB requires keeping the data of all collections in memory. This limitation will be addressed with the implementation of #397, which will allow asynchronous fetching of collection data. For now, SignalDB is better suited for smaller applications that don't need to handle large amounts of data. As you mentioned in #1008, it’s possible to use the AutoFetchCollection to pull data from asynchronous storage into SignalDB's memory, providing a workaround in the meantime.

When the client connects, syncing all the data is not required. You can selectively sync parts of the data or fetch only the changes based on a timestamp. This can be accomplished with a more advanced pull method for the SyncManager. Although the documentation currently lacks detailed examples, this flexibility is available within SignalDB.

Regarding the RxDB bridge, it was initially a solution for syncing/replication before SignalDB had its own implementation. While it’s still possible to use it, as you noted, there’s significant overlap in their features.

I was thinking perhaps SignalDB could be extended to become a local-first database with a locally cached db that can be queried, and this again is partially synced with a remote database.

That’s exactly the vision for SignalDB – becoming a local-first database, with efficient syncing and a smooth developer experience. There’s more work to be done, but we're steadily moving toward that goal. Contributions are always welcome!