Multiple text types per doc + multiple editors in the UI

[jasonm]

Hello!

I'm thinking of an app with a single Yjs doc that contains multiple text types inside that doc, and where the editors (e.g. slate-yjs) for those multiple text types are all displayed on the page at the same time. I think that this doesn't currently work, but have some ideas about how it could. I wonder if anyone here has feedback on these ideas. I'm quite new to Yjs and Slate, so any advice is appreciated - perhaps I'm way off-base with some of these suggestions. I'll try to implement them over the next week or so when I get a bit of time.

Context

Consider the "Managing multiple collaborative documents in a shared type" example on the Yjs docs: Here, there are buttons to switch between e.g. Document 0, Document 1, etc. What if, instead of allowing a user to switch docs, all docs were drawn in a list? As I understand it, the cursors would not work because the multiple editors would all try to use the same key(s) in the awareness CRDT.

The editor plugins I have tried (slate, remirror, tiptap) all seem to assume that there's only one editor per doc, and that they can therefore set their cursor information into the awareness crdt without any "namespacing" to identify the text type. The issue comes when multiple users have their cursors in different text types within the same doc: all editor component instances read from the same awareness CRDT, despite the fact that only one of them should.

Problem

For example, consider a doc with two text types, each under their own key:

doc: { textA, textB }

Let's say there is a UI which shows two text editors on the screen: one for textA, and another for textB. Let's also say there are 2 users collaborating: user1 and user2.

When both users are both focused into textA this work fine -- the editor sets awareness about their cursor position, and each user reads the other user's cursor position from awareness and draws the other user cursor into textA.

But if user1 is working in textA and user2 is working in textB, then both users will still communicate their cursor positions into an un-namespaced awareness key, and user1 will attempt to draw user2's cursor into textA using the position information from user2's presence in textB (and vice-versa).

Solution 0: Separate subdocs

I originally started down a path of trying to address this by putting separate text types into wholly separate docs:

doc: { subdoc1: { text: textA }, subdoc2: { text: textB } }

This works! Each piece of awareness would be correctly scoped to the appropriate text, because there is only 1 text type per doc.

However, using subdocuments, every user would subscribe to every subdocument. It's somewhat inefficient due to the lack of subdocument multiplexing in providers (such as described in https://discuss.yjs.dev/t/multiple-room-sync-subdocument/403 ). I'm mostly thinking about y-websocket here. There is some recent work on this in forks, eg DAlperin/y-websocket@f2f1cd8 -- so, perhaps the subdoc approach is viable.

Other solutions

However, can we do this without subdocs? I think we can, by enriching the structure of the awareness crdt.

When considering solutions, there's an interesting question: can a user be present in multiple texts at the same time? For a browser-based editor that draws multiple text editors on the screen, I think the answer can safely be "no" -- when user1 moves their focus from textA to textB, it is acceptable to say that they are no longer present in textA. However, perhaps there are other scenarios where this is not the case.

With that in mind, I can think of two approaches to a solution. One depends on that assertion that "a user may be present in at most one text type per doc at a time." The other solution does not.

Solution 1: awarenessPath in awareness crdt

If a user may only be present in at most one text type per doc at a time (i.e. user1 can be in textA or textB but not both) then we can continue to have an editor component own a set of top-level keys in the awareness, but then include some identifier of the "current" text type as another key in the awareness:

Example of a diff to slate-yjs cursor/awareness with this approach:
https://gist.github.com/jasonm/d1f3c3d4d7a7c231a804f78bf89c2a03/revisions

(I called this "awarenessPath", thinking of it as a path to the text type within the top-level document, e.g. users.0.notes.5... but perhaps some guid would be better.)

Solution 2: awarenessPath to namespace multiple presences-per-user within awareness crdt

If, on the other hand, a user may be present in multiple text types per doc at a time (i.e. user1 can be in both textA and textB at the same time) then we can take a slightly different approach. The editor component still needs to receive some identifier of the text type it is bound to, but then we can use that identifier as a key in the awareness to namespace the cursor details that are relevant to that text:

Example of a diff to slate-yjs cursor/awareness with this approach:
https://gist.github.com/jasonm/3af9670547a54024941f305250e4d0fe/revisions

WDYT?

[jasonm]

While I'm thinking about this... there's some related thinking about how different libraries which use the awareness CRDT (like slate-yjs) could be used together (or even have their awarenesses composed?) in applications with a rich document structure. From a note on this discuss.yjs.dev thread post:

I'm (...) thinking more about how multiple awareness consumers (mouse X/Y, tree/block presence like Notion, text edit cursors, etc.) can share the awareness keyspace. (...) really this could generalize to some lightweight standard where various libraries who participate in the awareness CRDT agree on how to share the keyspace safely.

Also, perhaps this is very much an application-level concern rather than a library concern… I think if libraries expose a way for the application to provide some sort of awarenessRootKey [to namespace their usage of the awareness CRDT] then the responsibility of structuring the awareness CRDT can rest wholly with the application.

[jasonm]

I forked the repo and made a PR for illustration: https://github.com/jasonm/slate-yjs/pull/1/files?w=1

and made an example app and recorded a short video where I describe how I use the namespaced awareness in that example app:
https://www.loom.com/share/9826ef69d39b4f0fa541309efea72bf1

This implementation follows the "Solution 1" in my original discussion post. I also tried the "Solution 2" approach. It also works, but required a little more code to explicitly manage blur/focus and is unnecessary for this use case of text inputs where a given client will have at most one text editor active at a time.

I'm curious what others, including @BitPhinix, think of this approach. Thanks for any feedback!

I don't love the fact that some of my changes to use an awarenessPath key require knowledge of that key in the library (slate-yjs here) and some of the changes happen in my application. I think it could make sense to offer an API that pushes all of this responsibility onto the application author. One way could be accepting an optional filterAwareness function into useCursor, and then using that predicate inside the useCursor hook implementation where it filters other awarenesses to compute the cursors:

// Inside my editor wrapper component, where `currentPath` is some
// application-specific context identifier, like "which text region is this?"
const filterAwareness = (a) => {
  return a.awarenessPath === currentPath;
};

const { decorate } = useCursors(editor, { filterAwareness });

[jasonm]

Ah interesting, I see there is some functionality in Tiptap/y-prosemirror along these lines: yjs/y-prosemirror#19

[BitPhinix]

Thanks for this exploration 🙏, I'll add the concept of a awarenessPath/key in v4 👍

[DAlperin]

@jasonm. I noticed you referenced my y-websocket fork that supports sub-documents (I have made a whole bunch of stability fixes since that commit if you are interested). I am currently working on an application where I think the best way forward on a particular feature is to bring native subdocument support into slate-yjs. Would anyone be interested in maybe chatting sometime on bringing sub document support to slate-yjs?

I have an extremely hacky (do not use this, it's pretty broken) example of how one might integrate this with an editor at a high level https://github.com/DAlperin/y-prosemirror

@BitPhinix @jasonm

[DAlperin]

every user would subscribe to every subdocument

The nice thing about the multiplexing protocol I have been working on is that you can choose which subdocuments to subscribe to, so given your example of

doc: { subdoc1: { text: textA }, subdoc2: { text: textB } }

When the root document first gets loaded, the doc looks something like this

doc: { subdoc1: { ... }, subdoc2: { ... } }

where the client is aware that subdoc1 and subdoc2 exist but their contents won't be loaded until the client explicitly calls load() on them.

ydoc.on('subdocs', ({added}) => {
  added.forEach(subdoc => {
    subdoc.load()
  });
})

For example this loads all the documents as the client becomes aware of them but you can see how it would be trivial to do that conditionally.

[jasonm]

@DAlperin Cool! Thanks for the ping. I think that subdocument multiplexing support across the ecosystem of providers (e.g. y-websocket) and consumer components like editors is very interesting. When you write about the multiplexing protocol you're working on, is that at the provider level?

I've actually veered away from using subdocument in my current project for now as it doesn't seem to be required. I've also been using TipTap (a prosemirror wrapper) which supports multiple editors in one doc by using a field argument passed to the constructor of their editor's Collaboration plugin to distinguish between them - this may be useful as a reference in your y-prosemirror work?

[DAlperin]

@jasonm

When you write about the multiplexing protocol you're working on, is that at the provider level?

Pretty much, yeah. y-protocols defines a binary protocol which each provider then implements over its own transport layer (i.e. websocket or webrtc, etc). I have added on a new "message type" onto the protocol that allows for the client and server to specify the guid of the document in question so you can address a theoretically infinite number of sub documents. Currently only my fork of y-websocket implements this protocol but I have to imagine that adding it to other providers should be trivial.

field argument passed to the constructor of their editor's Collaboration plugin to distinguish between them

Ooh thats very interesting, thanks for pointing me in that direction. I am curious what your use case for attaching multiple editors to one document is?

As awesome as it looks I don't quite think it's an option for my project since each editor logically needs to be it's own document since one document can be embedded in hundreds of different places around the application. What the user sees on a given screen should look like a single document (hopefully) but could actually be made up of any given number of sub documents.

[DAlperin]

field argument passed to the constructor of their editor's Collaboration plugin

Out of curiosity how does this work with the collaboration cursor? A cursory skim of the docs doesn't make that clear. Have you had any luck with it?

[DAlperin]

Sorry to keep pinging just wanted to point to my y-prosemirror fork which is now (as far as I can tell) entirely spec compliant when it comes to sub documents. I have also added a number of guards against several footguns (i.e. if you accidentally have multiple docs using the same state key, ignore messages that originate from a doc with a different guid as yours)

[jasonm]

Out of curiosity how does this work with the collaboration cursor?

Hm, reviewing my code, looks like I'm not actually using this. While my app (I'm prototyping/evaluating Yjs as a possible new tech for our app at my day job, Forum) has multiple text types within a single doc, a given user is only present (awareness) in one of the types at a time, so I don't need to manage awareness for a single user into multiple text types at once. This kinda makes sense for the UX, where each text type has a corresponding textarea on the screen, and a user's cursor is really only active in one textarea at a time -- clicking from one textarea into into a second textarea should blur/unfocus the first one, and the user's presence/awareness really is only "in" one text type at a time.

However from the docs' description of the field argument of the CollaborationCursor extension (which is injected/provided into the TipTap wrapper of the prosemirror editor and manages mapping between the awareness data and drawing the cursor) and from my recollection, that field arg is used to select a "namespace" key within the awareness CRDT, so you could have one user publishing their presence for multiple text types. I could imagine a UX with two text areas on the screen, corresponding to two text types in a Y doc, where you might want to keep drawing a user's cursor into both text areas even though the user is only actively focused in one at a time. But, I'm not really sure why I'd want to do that 😄 hence moving away from needing this namespaced functionality.

fork which is now (as far as I can tell) entirely spec compliant when it comes to sub documents

Very cool! I'm keeping an eye on subdocument support - although I don't have a motivating use-case right now, it seems like a good part of an overall robust toolkit, and seems like it'll be useful eventually. I haven't done much benchmarking on client or server CPU utilization for Yjs processing in larger documents.

A digression about subdoc use cases...

In case you're curious about motivating use cases... the closest use case I can imagine, right now, is to support breakout rooms in our Forum classroom software. We have a main classroom, and then students frequently go into breakout groups to do group activities. During this mode, the teacher and TAs can both see an overview of activity in the breakouts (which students have asked questions, the grousp' progress through worksheets, etc.) and can choose to visit breakouts to answer questions. I could imagine structuring the breakouts as subdocs, so that each student group only connects to their own breakout's doc but teachers and TAs could subscribe across all subdocs to see rollup information.

On the other hand, if the CPU utilization of subdocs in this case is high enough to be taxing on students, it may be taxing to teachers as well. Perhaps it'd be more efficient to calculate summary stats / rollup data on a server and then mirror that into a single summary doc that teachers/TAs subscribe to. It really depends on the details of data size, change rates, etc. This is also getting way ahead of myself by imagining perf issues -- I'd of course want to build out a simple implementation and benchmark it before considering introducing the extra complexity of subdocs.

Now, on the other hand, I could imagine subdocs being more justified in a Notion/Roam-like application with a large amount of data that is sparsely connected and provides a UX where users edit a frequently-shifting combination of docs/subdocs.

Also, knowing that JupyterLab RTC is written in Yjs, I could imagine building some kind of large data analytics dashboard tooling that has similar usage patterns to the Notion/Roam example but where the actual data is JupyterLab notebook cells.

Now I'm really off-topic 😄 Anyways, thanks for the pointers to y-prosemirror and for your continued work on subdocs. While I'm not using it right now, it's great to see!

[DAlperin]

select a "namespace" key within the awareness CRDT, so you could have one user publishing their presence for multiple text types

This is (basically) how I currently do it with subdocs (though I'd like to revisit the api to make it a slightly less leaky abstraction), unfortunately I think we are stuck with namespaces since as far as YJS is concerned each subdocument is it's own document.

On the other hand, if the CPU utilization of subdocs in this case is high enough to be taxing on students, it may be taxing to teachers as well.

I have only done some cursory benchmarking but I don't think there is a performance hit from subdocs since yjs internally views sub documents as the same as regular documents with some extra transport layer magic which adds a few bytes in for subdocuments so the server can handle them.

The thing that immediately comes to mind is something like this:

The teachers and TA load the root document which has x amount of documents as subnodes so they can see them all

• Teacher/TA (doc 1)
  • Breakout 1 (doc 1-a)
  • Breakout 2 (doc 1-b)
  • Breakout 3 (doc 1-c)

Then each breakout room loads their document as the root document. Subdocs don't have a concept of 'parent' so much as the root node has the concept of which other docs it includes as subdocs, what that means is that each individual room does not have to be aware of the concept of subdocs at all so the performance is unaffected.

• Breakout 1 (doc 1-a)
• Breakout 2 (doc 1-b)
• Breakout 3 (doc 1-c)

large amount of data that is sparsely connected and provides a UX where users edit a frequently-shifting combination of docs/subdocs

This is where the rabbit hole started for me :)

If you'd ever like to chat about how you could implement the breakout room feature you were describing let me know, I love helping out ed-tech wherever possible (I'm still a HS student so my open source contribution and closed feedback for edtech systems still feels kinda personal :) (my email is dov (at) dov (dot) dev)

Good luck!