feat(56/STATUS-COMMUNITIES): initial draft #567

rymnc · 2023-01-27T07:28:27Z

Part of the effort described in waku-org/pm#12

Porting the status community spec to vac research (app-protocol):
vacp2p/research#166

Companion PR - #569

rymnc · 2023-01-27T07:31:09Z

content/docs/rfcs/53/README.md

+# Design Requirements
+
+Due to the nature of organizations, the following requirements are necessary for the design of organization chats -
+
+1. The Organization owner is trusted.
+2. The Organization owner can add or remove peers from the Organization.
+3. The Organization owner can add or remove channels.
+4. The peers in the Organization can send/receive messages to the channels which they have access to.
+5. Organizations may be encrypted or unencrypted (public).
+6. An Organization is uniquely identified by a public key.
+7. The public key of the Organization is shared out of band.
+8. The metadata of the Organization can be found by listening on a content topic derived from the public key of the Organization.
+9. Peers part of an Organization run their own infrastructure.


@felicio can you confirm if these assumptions are OK?

LGTM, just,

The Organization owner is trusted.

I'd replace org for community.

The Organization owner can add or remove peers from the Organization.

Specifically, those are kick and ban ops (ref https://discord.com/channels/624205794384281629/730410574428504166/1020292913768443985).

The Organization owner can add or remove channels.

And edit ofc, but dunno how taxative we wanna be here.

Organizations may be encrypted or unencrypted (public).

Might be better to primarily distinguish them as private and public (ref https://discord.com/channels/624205794384281629/1048212980904501329/1052603998714220564).

Also, and maybe just as an fyi, public communities are "de facto unencrypted" atm. Since they're encrypted with a symmetric key, which is however obtainable by anyone (ref. https://discord.com/channels/864066763682218004/865466680129880074/1058202405583474688). If you'd be questioning this, I'd probably agree with you.

Peers part of an Organization run their own infrastructure.

Meaning running Store or Relay protocols, for example? Web clients don't do that for instance.

Also, admin role, in addition to the owner, def at https://discord.com/channels/624205794384281629/697843216048717905/1022799536495796224.

The peers in the Organization can send/receive messages to the channels which they have access to.

Members may be more suitable naming.

Given op thread at https://discord.com/channels/864066763682218004/1064649971497971862/1065146990540902400, do you think the following is worth mentioning here too:

the periodic publishing

the clock

the token-gating

waku protocols in use

back-ups

notifications

Thanks for your feedback.
There are a couple things we need to discuss though -

Since status communities "implements" this spec, the implementation details of web clients will be omitted, as this only serves as a way on how communities work.

unencrypted vs encrypted communities: as per my understanding of the message you shared, it is possible to derive the key of public communities. Is there a reason we're encrypting messages from public communities at all?

Based on naming in other specs, let's try to stick to "peers" instead of "members". No strong opinion here, but can change it if it's required.

I agree that we need to mention periodic publishing, clock, waku protocols in use, and backups. I believe token gating is still being workshopped, so a new PR to this rfc can discuss token gating and relevant ideas/issues there. I'll add it to future work for now.

Imo we don't need to specify how notifications are delivered.

Addressed some of your concerns in 6352e9f

Roger that.

Addressed some of your concerns in 6352e9f

Nice. Would you agree to move "To store and retrieve messages for offline devices" line to "SHOULD" section too instead of "MAY"? Without it, users wouldn't be able to fetch historical messages, for example.

Is there a reason we're encrypting messages from public communities at all?

I don't have a better answer, I'm sorry. @fryorcraken, please do you?

Done, moved WAKU2-STORE to the "SHOULD" section

Is there a reason we're encrypting messages from public communities at all?

I don't have a better answer, I'm sorry. @fryorcraken, please do you?

Sorry, I don't know either, would need to ask @cammellos

Historical reasons, it was always a requirement for waku-1/whisper that messages have to be encrypted when sent over the network (there used to be support for unencrypted but it was dropped).
One benefit is that it provides some level of obscurity, since if you don't know/share the public key of the community, traffic can't be encrypted, but it's very bland and that's why we consider them unencrypted.

content/docs/rfcs/55/README.md

oskarth · 2023-02-06T10:31:16Z

content/docs/rfcs/56/README.md

+slug: 56
+title: 56/STATUS-COMMUNITIES
+name: Status Communities that run on Waku v2
+status: raw


This should be draft and correspond with actual code running Communities I think

Addressed in 4d11e7a. It is almost 1:1 with the implementation, just a few features which are too large to mention here can be present in follow-up rfcs (backup/sync protocol, archival protocol, etc)

oskarth · 2023-02-06T10:32:41Z

content/docs/rfcs/56/README.md

+
+1. The Community owner generates a public/private key pair.
+2. The Community owner configures the Community metadata, according to the wire format "CommunityDescription".
+3. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. 


More details on topic usage would be useful

Added in 4d11e7a

@felicio, can you confirm that the content topic usage is as mentioned in the commit? As per my investigation, this is how they're used.

Confirmed 👍, just

keccak-256 hash of the result MUST be used as the content topic

the hash is encoded as hex and what's used as a topic is this https://rfc.vac.dev/spec/23/#bridging-waku-v1-and-waku-v2 construct.

Code for ref,

https://github.com/status-im/status-go/blob/f6dc6f752a/wakuv2/common/topic.go#L66-L70

https://github.com/status-im/status-go/blob/f6dc6f752ac87ff3f05e235aec18ad2bc36da791/protocol/transport/topic.go#L20-L23

32 bytes and used as the content topic

Same as above would apply.

Great! thanks, I'll mention the hex-encoding

Addressed in bfd3e02

kaiserd

Thank you for the RFC ❇️
I left some in-line comments.

My take on the peers vs members discussion:

I'd use member when you refer to members of a status community, or members associated with a specific content topic. I'd use node/peer for the respective entities on the Waku relay layer.
I use node to refer to the own node, or some nodes from a bird-eye view; and peers to refer to nodes from the perspective of some node. E.g. a node is connected to peers; our node discovers peers, etc.

edit:
accidentally pressed approve. These were just intermediate comments, I'll do another review later.

content/docs/rfcs/56/README.md

kaiserd · 2023-02-08T08:35:34Z

content/docs/rfcs/56/README.md

+The messages in a channel are broadcasted to all the users in the channel.
+
+A regular group chat between two or more peers reduces to a 1:1 chat between each peer and the other peers.
+One mechanism for 1:1 chats is described in [55/STATUS-1TO1-CHAT](/spec/55/).


Imo, the difference between a group chat and a community should be made clearer.
I assume, the current explanation would confuse somebody new to Status communities.

Maybe:
The purpose of Status communities, as specified in this document, is allowing for large group chats.
Communities can have further substructure, e.g. specific channels.

Smaller group chats, on the other hand, are out of scope for this document and can be built over 55/STATUS-1TO1-CHAT. We refer to these smaller group chats simply as "group chats", to differentiate them from Communities.
[continue with your explanation]

Addressed in 72b8553

kaiserd · 2023-02-08T08:43:28Z

content/docs/rfcs/56/README.md

+
+A regular group chat between two or more peers reduces to a 1:1 chat between each peer and the other peers.
+One mechanism for 1:1 chats is described in [55/STATUS-1TO1-CHAT](/spec/55/).
+The key-exchange mechanism MUST be X3DH, as described in [53/WAKU2-X3DH](/spec/53/).


For 1:1 chat?
If it MUST be X3DH, why does it leave implicit SHOULD or MAY above:

One mechanism for 1:1 chats is described in

Maybe:
[...] for group chats based on 55/STATUS-1TO1-CHAT the key-exchange mechanism MUST be X3DH...

Addressed in 72b8553

content/docs/rfcs/56/README.md

kaiserd · 2023-02-08T09:19:11Z

content/docs/rfcs/56/README.md

+### Community Leave Flow
+
+1. A peer requests to leave a Community by sending a "CommunityRequestToLeave" message to the Community.
+2. The Community owner MAY accept or reject the request.


Why might a member not be allowed to leave 😅 ?

As per the current implementation, the owner has control over this process in the community.

kaiserd · 2023-02-08T09:20:57Z

content/docs/rfcs/56/README.md

+3. [53/WAKU2-X3DH-SESSIONS](/spec/54/) - To handle session keys
+4. [14/WAKU2-MESSAGE](/spec/14/) - To wrap community messages in a Waku message
+5. [13/WAKU2-STORE](/spec/13/) - To store and retrieve messages for offline devices
+


Note: We should link the scaling BCP RFC here once it is done.

kaiserd · 2023-02-08T09:22:27Z

content/docs/rfcs/56/README.md

+The peer MAY back up their local settings, by encrypting it with their public key, and sending it to a given content topic.
+The peer MAY then rely on this backup to restore their local settings, in case of a data loss.
+This feature relies on [13/WAKU2-STORE](/spec/13/) for storing and retrieving messages.
+


We have to analyse this (or at least keep this in mind) in regards to scaling.

kaiserd · 2023-02-08T09:26:28Z

content/docs/rfcs/56/README.md

+The clock used in the wire format refers to the Lamport timestamp of the message.
+The Lamport timestamp is a logical clock that is used to determine the order of events in a distributed system.
+This allows ordering of messages in an asynchronous network where messages may be received out of order.
+


Note: This might duplicate features of Waku and add unnecessary complexty. Depends on how Message ID evolves.

Co-authored-by: kaiserd <[email protected]>

…vents

content/docs/rfcs/56/README.md

fryorcraken · 2023-02-13T23:11:59Z

content/docs/rfcs/56/README.md

+
+The following Waku protocols MAY be used to implement Status Communities -
+
+1. [12/WAKU2-FILTER](/spec/12/) - Content filtering for resource restricted devices


Why is light push not mentioned here?

Addressed in 53a7692

content/docs/rfcs/56/README.md

fryorcraken · 2023-02-13T23:13:59Z

content/docs/rfcs/56/README.md

+This extends to banning and kicking members.
+4. The Community owner can add, edit and remove channels.
+5. Community members can send/receive messages to the channels which they have access to.
+6. Communities may be encrypted (private) or unencrypted (public).


My understanding is that public communities were dropped out. @John-44 can you confirm?

fryorcraken · 2023-02-13T23:15:51Z

content/docs/rfcs/56/README.md

+1. The Community owner generates a public/private key pair.
+2. The Community owner configures the Community metadata, according to the wire format "CommunityDescription".
+3. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. 
+the Community metadata SHOULD be encrypted with the public key of the Community. <!-- TODO: Verify this-->


Still a todo? Cc @felicio

The statement about metadata is true.

fryorcraken · 2023-02-13T23:16:36Z

content/docs/rfcs/56/README.md

+2. The Community owner configures the Community metadata, according to the wire format "CommunityDescription".
+3. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. 
+the Community metadata SHOULD be encrypted with the public key of the Community. <!-- TODO: Verify this-->
+The Community metadata MAY be sent during fixed intervals, to ensure that the Community metadata is available to members.


Would be good to define the current practice (what interval is used?) as this will help with the correctness of simulations.

I could not find instances of the community description being published periodically in status-go except when a change in the community description or control message happens.

https://github.com/status-im/status-go/blob/develop/protocol/messenger_communities.go#L191

As per @felicio, https://discord.com/channels/864066763682218004/1065146990540902400/1067769073678680126
It's broadcasted every hour.

content/docs/rfcs/56/README.md

fryorcraken · 2023-02-13T23:19:12Z

content/docs/rfcs/56/README.md

+
+1. The peer is invited to join a Community by the Community owner, by sending a "CommunityInvitation" message.
+2. A peer generates a public/private key pair.
+3. The peer decrypts the "CommunityInvitation" message, and verifies the signature.


back to my comment above. How does the peer decrypt the message? I guess not with the key it just generated.

Made it more clear in 3690bab

content/docs/rfcs/56/README.md

richard-ramos · 2023-02-14T00:15:43Z

content/docs/rfcs/56/README.md

+The content topic that Community channels/chats use MUST be the hex-encoded keccak-256 hash of the public key of the community concatenated with the chat id.
+
+```
+contentTopic = hex(keccak256(compressedPublicKey + chatId))


See prev comment

Addressed in 609fd24

richard-ramos · 2023-02-14T00:23:02Z

content/docs/rfcs/56/README.md

+2. The Community owner configures the Community metadata, according to the wire format "CommunityDescription".
+3. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. 
+the Community metadata SHOULD be encrypted with the public key of the Community. <!-- TODO: Verify this-->
+The Community metadata MAY be sent during fixed intervals, to ensure that the Community metadata is available to members.


I could not find instances of the community description being published periodically in status-go except when a change in the community description or control message happens.

content/docs/rfcs/56/README.md

richard-ramos · 2023-02-14T00:36:58Z

content/docs/rfcs/56/README.md

+
+### Community Ban Flow
+
+1. The Community owner adds a member to the ban list, revokes their grants, and publishes the updated Community metadata.


When an user is banned a re-key also happens if the community is encrypted. Should that be mentioned here?

Addressed in b333c9c

Co-authored-by: fryorcraken.eth <[email protected]> Co-authored-by: RichΛrd <[email protected]>

kaiserd

Thanks! LGTM ❇️

As for the 1:1 chat RFC, we should do a spec-parlance editing pass before moving to stable (future work).

feat(55/WAKU2-ORGANIZATION-CHATS): initial draft

9a499c7

rymnc changed the title ~~feat(55/WAKU2-ORGANIZATION-CHATS): initial draft~~ [WIP]: feat(55/WAKU2-ORGANIZATION-CHATS): initial draft Jan 27, 2023

rymnc commented Jan 27, 2023

View reviewed changes

rymnc added 2 commits January 27, 2023 13:43

fix(55/WAKU2-ORGANIZATION-CHATS): rename folder, omit cryptography

94b910e

fix(55/STATUS-COMMUNITIES): s/Organization/Communities

df9e554

rymnc changed the title ~~[WIP]: feat(55/WAKU2-ORGANIZATION-CHATS): initial draft~~ [WIP]: feat(55/STATUS-COMMUNITIES): initial draft Jan 31, 2023

felicio reviewed Jan 31, 2023

View reviewed changes

content/docs/rfcs/55/README.md Outdated Show resolved Hide resolved

rymnc mentioned this pull request Jan 31, 2023

Port 5/SECURE-TRANSPORT to vac research #569

Merged

rymnc changed the title ~~[WIP]: feat(55/STATUS-COMMUNITIES): initial draft~~ WIP: feat(55/STATUS-COMMUNITIES): initial draft Jan 31, 2023

rymnc added 4 commits February 1, 2023 10:13

fix(55/STATUS-COMMUNITIES): address comments

6352e9f

fix(55/STATUS-COMMUNITIES): move store to MUST use section

048ba99

fix: move to 56, add to index, copyright info

51f5ca1

fix: remove noise as a key-exchange candidate

2edf657

rymnc changed the title ~~WIP: feat(55/STATUS-COMMUNITIES): initial draft~~ WIP: feat(56/STATUS-COMMUNITIES): initial draft Feb 2, 2023

rymnc changed the title ~~WIP: feat(56/STATUS-COMMUNITIES): initial draft~~ feat(56/STATUS-COMMUNITIES): initial draft Feb 2, 2023

kaiserd self-requested a review February 2, 2023 15:47

rymnc added 3 commits February 3, 2023 14:18

fix(56/STATUS-COMMUNITIES): raw state, refs

d02ff45

feat(56/STATUS-COMMUNITIES): refer to backup mechanism

068491d

fix(56/STATUS-COMMUNITIES): better wording

d63b975

oskarth requested a review from cammellos February 6, 2023 09:42

oskarth reviewed Feb 6, 2023

View reviewed changes

rymnc mentioned this pull request Feb 6, 2023

raw RFC(standards): Status communities vacp2p/research#166

Closed

rymnc requested review from Samyoul and richard-ramos February 6, 2023 13:50

rymnc added 2 commits February 7, 2023 11:37

fix(56/STATUS-COMMUNITIES): specific usage of content topics

4d11e7a

fix(56/STATUS-COMMUNITIES): hex-encoded hash, and formula for clarity

bfd3e02

kaiserd approved these changes Feb 8, 2023

View reviewed changes

kaiserd self-requested a review February 8, 2023 09:36

fix(56/STATUS-COMMUNITIES): address semantic changes

12c97a5

Co-authored-by: kaiserd <[email protected]>

rymnc added 6 commits February 8, 2023 16:10

fix(56/STATUS-COMMUNITIES): clearer background

72b8553

fix(56/STATUS-COMMUNITIES): improved assumptions

64237af

fix(56/STATUS-COMMUNITIES): definition of content topic

30c465b

fix(56/STATUS-COMMUNITIES): clearer chat id

c0fc7cb

fix(56/STATUS-COMMUNITIES): add future work content topic usage for e…

fccad2d

…vents

fix(56/STATUS-COMMUNITIES): usage of members where applicable

6f7a44b

rymnc marked this pull request as ready for review February 8, 2023 11:31

fryorcraken reviewed Feb 13, 2023

View reviewed changes

richard-ramos reviewed Feb 14, 2023

View reviewed changes

rymnc and others added 5 commits February 14, 2023 14:15

fix(56/STATUS-COMMUNITIES): address semantic changes

d64aeb2

Co-authored-by: fryorcraken.eth <[email protected]> Co-authored-by: RichΛrd <[email protected]>

fix(56/STATUS-COMMUNITIES): topic usage

609fd24

fix(56/STATUS-COMMUNITIES): light node clarification

53a7692

fix(56/STATUS-COMMUNITIES): 1:1 chat for joining

3690bab

fix(56/STATUS-COMMUNITIES): community ban flow update

b333c9c

felicio mentioned this pull request Feb 20, 2023

17/URL data: draft status-im/specs#169

Closed

rymnc requested a review from felicio February 21, 2023 12:02

felicio approved these changes Feb 21, 2023

View reviewed changes

Merge branch 'master' into organization-spec

b251b64

kaiserd approved these changes Feb 21, 2023

View reviewed changes

rymnc merged commit 7b9950c into master Feb 21, 2023

rymnc deleted the organization-spec branch February 21, 2023 16:18

felicio mentioned this pull request Apr 4, 2023

feat(54/WAKU2-X3DH-SESSIONS): include topic usage #587

Merged


		The following Waku protocols MAY be used to implement Status Communities -

		1. [12/WAKU2-FILTER](/spec/12/) - Content filtering for resource restricted devices


		### Community Ban Flow

		1. The Community owner adds a member to the ban list, revokes their grants, and publishes the updated Community metadata.

feat(56/STATUS-COMMUNITIES): initial draft #567

feat(56/STATUS-COMMUNITIES): initial draft #567

Conversation

rymnc commented Jan 27, 2023 • edited Loading

rymnc Jan 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

felicio Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

felicio Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fryorcraken Feb 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaiserd left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaiserd left a comment

Choose a reason for hiding this comment

rymnc commented Jan 27, 2023 •

edited

Loading

rymnc Jan 27, 2023 •

edited

Loading

felicio Jan 31, 2023 •

edited

Loading

felicio Jan 31, 2023 •

edited

Loading

fryorcraken Feb 9, 2023 •

edited

Loading

kaiserd left a comment •

edited

Loading