Is there any advice, doc, or examples available regarding interop with Node.js?

[rcmurray]

Is there any advice, documentation, or examples available regarding interop with Node.js?

One of our interfaces with \psi is an agent that sends and receives text-based information to and from \psi asynchronously (rather than in a request/reply pattern). We've currently got two ZeroMQ pub-sub interfaces working (one for each communication direction) but this currently (1) requires each publisher side to bind a port, preventing other processes from using it (e.g., other agents or clients on the same machine), and (2) requires the agent in its subscriber role to know the client's IP address. We see that ZeroMQ also has a client-server pattern in draft phase (https://rfc.zeromq.org/spec/41/), which seems like it would fit more naturally with our needs (asynchronous communication with only one interface instead of two), although (besides being in draft phase) this would require the agent/server to send each message to specific client routing IDs rather than publishing to all clients at once.

All of the rest of our agent's interfaces to clients are through a Node.js server, supporting asynchronous two-way communication and publishing to every client within a socket room. For these capabilities and for consistency, it would seem to be ideal if \psi could also connect through Node.js. We see that Node.js is mentioned as one of \psi's interop possibilities, but that's the extent of the documentation we've seen.

[AshleyF]

You're right that Node.js is mentioned in the interop docs but then all the code examples are in Python! Here's an example of how the same things are accomplished very similarly in Node.js/JavaScript using the ZeroMQ library:

To publish from \psi:

using (var p = Pipeline.Create())
{
    var gen = Generators.Range(p, 0, 1000, TimeSpan.FromMilliseconds(100));
    var sin = gen.Select(x => Math.Sin(x / 100.0));
    var mq = new NetMQWriter<double>(p, "sin-topic", "tcp://localhost:12345", JsonFormat.Instance);
    sin.PipeTo(mq);
    p.Run();
}

To subscribe from Node.js/JavaScript:

var zmq = require("zeromq"), sock = zmq.socket("sub");

sock.connect("tcp://127.0.0.1:12345");
sock.subscribe("sin-topic");

sock.on("message", function(topic, payload) {
    var data = JSON.parse(payload);
    var message = data.message;
    var originatingTime = data.originatingTime;
    console.log("Message: " + message + " (" + originatingTime + ")");
});

To publish from Node.js/JavaScript:

var zmq = require("zeromq"), sock = zmq.socket("pub");

sock.bindSync("tcp://127.0.0.1:34567");

setInterval(function() {
    var payload = {
        message: Math.random(),
        originatingTime: new Date().toISOString() };
    sock.send(["random-topic", JSON.stringify(payload)]);
}, 100);

And finally to subscribe from \psi:

using (var p = Pipeline.Create())
{
    var mq = new NetMQSource<double>(p, "random-topic", "tcp://localhost:34567", JsonFormat.Instance);
    mq.Do((m, e) => Console.WriteLine($"Message: {m} ({e.OriginatingTime})"));
    p.Run();
}

Hopefully this helps!

[rcmurray]

Thank you very much for attending to this and for your detailed example, Ashley. I probably wasn't quite clear enough in my question. I think this solution still requires binding a port on the Node.js server side and requires the Node.js server to track and subscribe to the specific IP address(es) of its client(s). To avoid those issues, I really should have said we would like to connect from \psi to Node.js using socket.io. We do have a ZeroMQ/NetMQ connection implemented already, but it suffers from these issues.

[AshleyF]

Ah, I see. Indeed the NetMQWriter and NetMQSource components support only pub/sub at the moment, which seemed to fit best with \psi's unidirectional streams. The writer does support multiple topics over one port, if that helps. NetMQWriter defaults to one topic, but calls to AddTopic(...) will create new receivers. For example:

var mq = new NetMQWriter(p, "tcp://localhost:12345", JsonFormat.Instance);
someStream.PipeTo(mq.AddTopic("some-topic"));
someOtherStream.PipeTo(mq.AddTopic("some-other-topic"));

I would imagine that your scenario may require extending or adding components. There is a socket.io package for .NET. If you went that route, I'd suggest following the pattern in the NetMQ* components; taking IFormatSerializer/Deserializer, etc. so that new transport components can play well with the other bits in Microsoft.Psi.Interop.

[rcmurray]

Thanks, Ashley. We'll see what we can do to add a socket.io package. It sounds like the serialization you mention will be key.