Simplifies wasi-messaging interface with feedback (#24)

* feat(*): Updates messaging to support request-reply

This makes several updates to the messaging interface. Initially the
README said that this wasn't going to support request/reply, but based
on my reading of the Kafka, NATS, MQTT, and SQS APIs, this is a fairly
common pattern. Another piece of evidence here is what I've seen as a
wasmCloud maintainer from our users. Request/reply is one of the more
common things we see with a messaging service. Please note that this
doesn't _require_ the use of a reply-to topic, just exposes it for use.

I also did a few other changes here. First is that I added the topic to
the message. This was common across all systems and is often used by code
to select the appropriate logic to perform. I also removed the format
field as this didn't seem to be a common parameter across various services.
We could definitely add a content-type member to this record in the future
if needed, but I think much of that can be passed via the metadata field.

There are other things I might suggest some changes to, but I want to think
on them some more and open some issues to discuss them first

Signed-off-by: Taylor Thomas <[email protected]>

* feat(*): Updates interfaces to be more streamlined

This PR integrates various changes from talking to current users of
messaging in the community as well as conversations among the champions

Signed-off-by: Taylor Thomas <[email protected]>

* feat(*)!: Additional changes based on PR feedback

I also deleted the examples.md for now until we settle on the interface.
It will be easier to add back in once we have some real world examples
to point at

Signed-off-by: Taylor Thomas <[email protected]>

* feat(types): Updates the message type to have configurable fields

Also removes extensions as a guest configuration option (for now)

Signed-off-by: Taylor Thomas <[email protected]>

* feat(types): Renames guest config to just plain simple `config`

In many of the interfaces out there right now, we've moved more towards
just calling these things config

Signed-off-by: Taylor Thomas <[email protected]>

* feat(*): Additional changes to request/reply for streamlining

Also removes the channel parameter I forgot to remove in a previous
commit

Signed-off-by: Taylor Thomas <[email protected]>

* feat(request): Updates request-multi to support scatter/gather operations

One of the uses of request-multi is to support a scatter/gather operation.
In these cases, you might not know how many requests you are going to
receive, so you can't set expected replies. Generally these wait until
timeout and then return the results. This commit adds the ability to
support all the different use cases for request-multi

Signed-off-by: Taylor Thomas <[email protected]>

* ref(*): Simplifies interface and documents scope/portability

After a whole bunch of feedback from the community, I realized we were
still trying to make this interface too much. So I dramatically paired
back the interface to be purely focused on message passing. Further
features like ack/nack, guaranteed delivery, and so on are now out of
scope (see the README for full details).

This was partly inspired by a discussion in the CNCF Wasm WG around this
interface. To be perfectly frank, the level I paired this down to is
essentially the same level of guarantees offered by the wasmCloud
[messaging interface](https://github.com/wasmCloud/messaging). The main
reason being is that there are people actually using that interface for
real applications (with real host implementations). If we can come to
agreement on a simpler interface, it will be easier to add in functionality
such as the things I stripped out in this commit.

Please let me know any feedback you have around this, focusing on whether
or not this covers at least the most basic scenarios

Signed-off-by: Taylor Thomas <[email protected]>

* fixed typos, added subscribe, added messaging-request-reply world, and nits

Signed-off-by: danbugs <[email protected]>

* update auto-generated files and wit-abi-up-to-date version

Signed-off-by: danbugs <[email protected]>

* removed .idea/ and added to .gitignore

Signed-off-by: danbugs <[email protected]>

* updated README

Signed-off-by: danbugs <[email protected]>

* addressing first round of feedback: added type aliases, added remove-metadata method, added new-line at the end of files, improved argument names, removed .gitignore, improved documentation, changed get-subscriptions function name

Signed-off-by: danbugs <[email protected]>

* [pr feedback] added set-metadata method + revisited topic

Signed-off-by: danbugs <[email protected]>

* [pr feedback] changing signature of send

Signed-off-by: danbugs <[email protected]>

---------

Signed-off-by: Taylor Thomas <[email protected]>
Signed-off-by: danbugs <[email protected]>
Co-authored-by: Taylor Thomas <[email protected]>

.github/workflows/main.yml

@@ -13,5 +13,5 @@ jobs:
     - uses: actions/checkout@v2
     - uses: WebAssembly/wit-abi-up-to-date@v17
       with:
-        wit-bindgen: '0.18.0'
-        worlds: 'imports messaging'
+        wit-bindgen: '0.34.0'
+        worlds: 'imports imports-request-reply messaging-core messaging-request-reply'

README.md

@@ -2,55 +2,103 @@
 
 A proposed [WebAssembly System Interface](https://github.com/WebAssembly/WASI) API.
 
-### Current Phase
+## Table of Contents
+
+- [`wasi-messaging`](#wasi-messaging)
+  - [Table of Contents](#table-of-contents)
+  - [Current Phase](#current-phase)
+  - [Champions](#champions)
+  - [Phase 4 Advancement Criteria](#phase-4-advancement-criteria)
+  - [Introduction](#introduction)
+  - [Goals](#goals)
+  - [Portability criteria](#portability-criteria)
+  - [Dev notes](#dev-notes)
 
-`wasi-messaging` is currently in [Phase 1](https://github.com/WebAssembly/WASI/blob/main/Proposals.md#phase-1---feature-proposal-cg).
+## Current Phase
 
-### Champions
+`wasi-messaging` is currently in [Phase
+1](https://github.com/WebAssembly/WASI/blob/main/Proposals.md#phase-1---feature-proposal-cg).
+
+## Champions
 
 - [Dan Chiarlone](https://github.com/danbugs)
 - [David Justice](https://github.com/devigned)
 - [Jiaxiao Zhou](https://github.com/Mossaka)
 - [Taylor Thomas](https://github.com/thomastaylor312)
 
-### Phase 4 Advancement Criteria
+## Phase 4 Advancement Criteria
 
-`wasi-messaging` should have at least two implementations (i.e., from service providers, and or cloud providers), and, at the very minimum, pass the testsuite for Windows, Linux, and MacOS.
+For `wasi-messaging` to advance to Phase 4, it must have at least two independent implementations 
+for open-source message brokers (such as Kafka, NATS, MQTT brokers) and two for cloud service providers 
+(e.g., Azure Service Bus, AWS SQS).
 
-## Table of Contents
+## Introduction
 
-- [Introduction](#introduction)
-- [Goals](#goals)
-- [Non-goals](#non-goals)
+In modern software systems, different components or applications often need to communicate with each other 
+to exchange information and coordinate their actions. Messaging systems facilitate this communication by
+allowing messages to be sent and received between different parts of a system.
 
-### Introduction
+However, implementing message-based communication can be challenging. It requires dealing with the details
+of message brokers, such as connection management, channel setup, and message serialization. This complexity
+can hinder development and lead to inconsistent implementations.
 
-The messaging interfaces aim to provide a generic and flexible way for producers and consumers to communicate through message brokers. The `producer` interface allows producers to publish events to a specific channel in a broker, while the `consumer` interface allows consumers to subscribe to a channel and receive events through a push-based mechanism. The handler interface provides an on-receive function that can be used to process received events with full abstraction of the underlying broker implementation.
+The `wasi-messaging` interface is a purposefully small interface focused purely on message passing. It is
+designed to express the bare minimum of receiving a message and sending a message, along with an optional
+request/reply interface that allows for message-based APIs and/or RPC.
 
-### Goals
+By providing a standard way to interact with message brokers, the `wasi-messaging` interface aims to simplify
+this process, hiding the underlying complexity from the user. This aligns with the broader goals of WASI by
+promoting interoperability, modularity, and security in WebAssembly applications.
 
-The messaging interfaces aim to address the need for a standard way to handle message-based communication in modern software systems. In complex software systems, different components or even different applications need to communicate with each other to exchange information and coordinate their actions.
+## Goals
 
-However, implementing message-based communication can be challenging, as it requires dealing with the details of message brokers, such as connection management, channel setup, and message serialization. The messaging interfaces aim to simplify this process by providing a standard way to interact with message brokers, hiding the underlying complexity from the user.
+The primary goal of this interface is to focus on message passing. The only guarantee offered is 
+that publishing a message is handled successfully. No other guarantees are made about the delivery 
+of the message or being able to ack/nack a message directly. This minimalist approach provides the 
+most basic foundation of messaging, which can be expanded upon by future interfaces or proposals as 
+use cases emerge.
 
-This standardization can benefit various scenarios, such as 
+This simplicity allows:
+- **Ease of Integration**: Components can easily implement the message handler in this interface, 
+with details such as work dispatch and queuing handled behind the scenes, invisible to the business logic.
+- **Flexibility**: Anything that can send a message can easily be passed into a queue 
+(such as a Kafka stream or NATS JetStream) without the knowledge that it is being sent into a queue.
+- **Extensibility**: The paradigm can be expanded by future interfaces (like a queue-based work interface) to handle 
+more complex messaging scenarios. By focusing solely on message passing, the wasi-messaging interface simplifies the 
+development of interoperable WebAssembly modules that can communicate over various messaging systems without being 
+tied to any specific implementation.
 
-- Microservice architectures, where each microservice can communicate with other microservices using the messaging service interfaces. Similarly, applications that need to handle event-driven or streaming data can benefit from the push-based message delivery mechanism provided by the `consumer` and `handler` interfaces;
+## Portability criteria
 
-- Local use cases such as communication channels between online and offline browser-based Web applications and local WASI applications.
+The main portability criteria on which this should be judged is whether a component can receive and send a message 
+from all major messaging systems. This includes:
+- Message standards like MQTT and AMQP.
+- Specific technologies like NATS and Kafka.
+- Cloud provider implementations like Azure Service Bus and AWS SQS.
 
+This _does not_ mean it implements the full set of features of each of the messaging systems. In fact, it is expected 
+that most implementations will need to do work to adapt their system to this interface (e.g., in Kafka, you'd have 
+to mark the message as completed once the call to handle returns).
 
-Overall, the messaging interfaces aim to make it easier to build complex and scalable software systems by providing a common foundation for message-based communication.
+As mentioned above, this should still be completely compatible with any more advanced use cases of the various 
+message systems. For example, if you have a queue of work that is currently being handled by pre-existing software
+outside of Wasm components, a component could use this interface to publish messages that get ingested into this queue.
 
-### Non-goals
+Another way to state the portability criteria is that this implementation should not break the possibility of a 
+component consuming this interface to be integrated with a more advanced messaging use case.
 
-- The messaging service interfaces do not aim to provide advanced features of message brokers, such as broker clustering, message persistence, or guaranteed message delivery. These are implementation-specific details that are not addressed by the interfaces.
-- The messaging service interfaces do not aim to provide support for every possible messaging pattern or use case. Instead, they focus on the common use cases of pub-sub and push-based message delivery. Other messaging patterns, such as request-reply or publish-confirm-subscribe, are outside the scope of the interfaces.
-- The messaging service interfaces do not aim to provide a specific implementation of a message broker. Instead, they provide a standard way to interact with any message broker that supports the interfaces.
+## Dev notes
 
-### API walk-through
+To regenerate the `.md` files, run: 
+```sh
+wit-bindgen markdown ./wit/ -w imports --html-in-md
+wit-bindgen markdown ./wit/ -w imports-request-reply --html-in-md
+wit-bindgen markdown ./wit/ -w messaging-core --html-in-md
+wit-bindgen markdown ./wit/ -w messaging-request-reply --html-in-md
+```
 
-For a full API walk-through, see [wasi-messaging-demo](https://github.com/danbugs/wasi-messaging-demo).
+It would make sense for a lot of these functions to be asynchronous, but that is not currently natively supported in 
+the component model. Asynchronous support will be added as part of WASI Preview 3. When async support becomes 
+available, we plan to update the wasi-messaging interface to incorporate asynchronous patterns.
 
-> Note: This README needs to be expanded to cover a number of additional fields suggested in the
-[WASI Proposal template](https://github.com/WebAssembly/wasi-proposal-template).
+> **Note**: Ensure you have version 0.34.0 of `wit-bindgen` installed to avoid compatibility issues.