Remote Procedure Call RPC

Introduction

Remote Procedure Call (RPC) is a mechanism which allows a program to invoke a function that is defined in another address space. If a user, A, writes a function and shares it with another user B, then B can invoke the shared function in A's program. The return value of the function is retrieved by B, which can then be used in B's program.

RPC is implemented using the Source Academy Module feature. The module is named 'rpc' and exposes three functions, namely connect, share and executeAfter.

The RPC also allows sharing of values in Source programs. These include numbers, strings, booleans, null and undefined. The sharing of arrays of arbitrary nesting is also allowed, along with lists and pairs which resolve into arrays themselves.

User Documentation

To start off, the functions mentioned above must be imported from the module 'rpc' using Source's import statement. The behaviour of the aforementioned functions is described as follows: -

• share - Returns a promise that resolves to a token that uniquely identifies the shared data or function. share accepts data or functions as input and runs asynchronously. The obtained token, when shared to another user, can be supplied as input to the connect function to retrieve the shared data or function.

  • Parameter: input - Any value that occurs in a Source program.
  • Returns: A promise that resolves to a token that uniquely identifies the shared content.

• connect - Returns a promise that resolves to the content that is uniquely identified by the input token. The function throws an error if the input is invalid. connect executes asynchronously like share.

  • Parameter: token - A string token that uniquely identifies some shared content.
  • Returns: A promise that resolves into the data or function that is identified by the input token.
  • Throws: An error if the token is invalid.

• executeAfter - Executes a call-back function after a promise is resolved. The value that the promise resolves into is supplied as input to the call-back function. This function must be used to deal with the returned promises of connect and share. Using promises as normal values in a program without executeAfter can lead to undefined behaviour.

  • Parameter: promise - A promise that resolves into some data or function.
  • Parameter: call-back - A call-back function that executes after the promise is resolved. The resolved value is supplied as argument to this function.
  • Returns: The return value of the call-back, if any. Using this return value synchronously can lead to undefined behaviour.

The functions share and connect are the core of this module, and their asynchronous behaviour is achieved by using the executeAfter function.

The following are code examples of correct module usage: -

/* USER A */
const xs = list(1, 2, 3, 4, 5);
const token = share(xs);
executeAfter(token, display);

display('Sharing initiated');

/* 
A token 's' gets displayed in A's REPL after `Sharing initiated` is first printed. 
'Sharing initiated' is printed first since executeAfter needs to wait for the promise returned by share to get resolved.
*/

/* USER B */
// Assume the constant s is the token shared by A to B.
const dataPromise = connect(s);
executeAfter(dataPromise, display);

/* The list of numbers from 1 to 5 gets displayed in B's REPL */

The next example demonstrates sharing functions. Keep in mind that a function call times out after 15 seconds if the call hasn't been completed.

/* USER A */


const token = share(display);
executeAfter(token, display);

display('Sharing initiated');

/* 
A token 's' gets displayed in A's REPL after `Sharing initiated` is first printed. 
'Sharing initiated' is printed first since executeAfter needs to wait for the promise returned by share to get resolved.
*/

/* USER B */
// Assume the constant s is the token shared by A to B.

const functionPromise = connect(s);
executeAfter(functionPromise, displayOfA => {
    displayOfA("Hi there");
});

display("Printing to A's REPL");

/* 'Printing to A's REPL' gets printed in B's REPL first, following by 'Hi there' in A's REPL. */

The next example demonstrates the chaining of promises. Note that the return values of shared functions are also promises, so they must be used with executeAfter.

/* USER A */
const xs = list(1, 2, 3, 4, 5);

function sum(xs) {
    display("Calculating sum of list");
    return is_null(xs)
        ? 0
        : head(xs) + sum(tail(xs));
}

function share_and_display(x) {
    const token = share(x);
    executeAfter(token, display);
}

share_and_display(xs);
share_and_display(sum);

display('Sharing initiated');

/* 
Two tokens, s1 and s2 get displayed in A's REPL after `Sharing initiated` is first printed. 
'Sharing initiated' is printed first since executeAfter needs to wait for the promise returned by share to get resolved.
The token s1 corresponds to the list while s2 maps to the function.
*/

/* USER B */
// Assume the constants s1 and s2 are the tokens shared by A to B.

const dataPromise = connect(s1);
executeAfter(dataPromise, data => {
    const functionPromise = connect(s2);
    executeAfter(functionPromise, f => {
        const returnValue = f(data);
        executeAfter(returnValue, display);
    });
});

/* The value 15 gets printed in B's REPL. Before this, the display statement gets executed in the function sum and prints strings to A's REPL */

Examples of incorrect module usage are as follows: -

const token = share(data);
display(token);

const dataPromise = connect(s);
display(dataPromise);

The above examples lead to undefined behaviour since the display statements will be executed synchronously. However, the execution of these statements is almost always faster than the time taken for the promises to be resolved, so the defined constants have undefined values. Caveats: -

• Correct code may produce incorrect results if there are network related issues in either the client side or the server side.
• Higher order functions will produce undefined behaviour when shared. This is because higher order functions can involve sharing of functions from both sides, which can and will force clients to redefine standard function definitions with executeAfter being used to deal with arguments. We've settled for a trade-off that allows easy sharing, but of functions that don't take in and return other functions.

Prerequisites and Resources

This system requires an understanding and familiarity with a broad range of topics. The following is a non-exhaustive but comprehensive list of topics and resources that should help a developer get started:

Since this project is written in Typescript, familiarity with Typescript/Javascript (JS) is required.

Topic	Reference Resource
Source Academy Modules system	Modules Wiki
Remote Procedure Calls	Wikipedia Article
How Javascript is executed by the browser	How Javascript works in the browser
Asynchronous and Concurrent program execution	Video by Hussein Nasser
Javascript Event Loop	The JavaScript Event Loop explained
Firestore Realtime Database	Quickstart Guide
JavaScript Promises and async-await	Series of articles

Developer Guide

RPC is implemented using the Source Module system. The back-end functions are defined in src/bundles/rpc/index.ts. This includes the exposed functions described above, as well as several internal functions.

The system uses an intermediate real-time DB, Firestore, to facilitate storage and retrieval of shared content.

When a user shares some data or function, the shared content is marshalled and stored in the marshalled-data Firestore Collection as a new Document. The ID of the document is encoded and returned as a token to the user. Currently, the encode function is rudimentary and just returns the input string.

If the function connect is called with this obtained token as input, the collection marshalled-data is queried for a Document whose ID matches the decoded token. If a document is found, the marshalled content is retrieved from the database. The client side then unmarshalls the content and returns it in the form of a promise.

The realtime-DB only deals with marshalled data. Marshalling and unmarshalling are done on the client side before accessing the database.

Function Call Semantics

Remote functions are marshalled into a JSON that contains a uuid to uniquely identify the shared function. As a function is shared, a listener is added on the client side that listens to changes inside the Firestore Collection function-calls.

When a shared function is unmarshalled and returned, another function is obtained. This function accepts an arbitrary number of arguments. When called, it creates a new Document in the collection 'function-calls' to represent a new function call. The arguments are marshalled and added to the newly created Document.

If there is a new Document in function-calls that corresponds to the shared function, then the shared function is executed via the listener added earlier on the client side and the return value is updated in the Firestore Document.

Once the call is completed and the return value is updated, the Document in function-calls is updated again, after which the return value is retrieved in the form of a promise.

More information about the semantics, specifications and the functions can be found in the Semantics and Specifications section below.

Semantics and Specifications

The specifications as well as the underlying semantics can be found in this PDF

Applications and Uses

There are a number of possible scenarios where this module can be useful. A couple of ideas are listed here below:

• Peer-to-peer multiplayer gaming
• Running divide and conquer algorithms in a distributed manner
• Tutor-student solution sharing for class assignments

Further Work and Improvements

• Exploring other networking mechanisms for better performance. For example, webRTC can be explored for the communication between peers
• Work with interpreted source. As highlighted in the semantical specifications document, the current system doesn't work when Source is interpreted directly. An improvement could be to make it work with the interpreter.
• Real-time dashboard to visualise sharing and connecting. To allow users to keep track of shared functions and connected functions in realtime, a dashboard can be created using the tab feature of the modules system. Such a feature could display, in realtime, statistics of all remote function calls and their return values. This would not only give the user a more clear picture of the system but also aid debugging.