Server Architecture

BHoM library provides API hooks to both Speckle Works via the Speckle_Toolkit and 3D Repo via the 3DRepo_Toolkit as per the diagram above.

A number of considerations have to be taken into account in order to deliver a feasible server architecture that will support various user stories and technologies.

For instance:

• To support Bentley DGN file format, the file would need to be uploaded to 3D Repo which will then populate the relevant data structures in BHoM;
• 3D Repo currently does not support object definitions directly, instead, it stores meshes with metadata and transformation matrices above them;
• Web sockets should be used for event updates;
• However, direct requests should be used for actual data retrieval.

Server Integration

For an overall server architecture and requirements we are relying on "User Story 1.2" as a suitable example:

As an Estimator, I want to create a model view on the design scope so that I can bring it into an estimation software and create a cost estimate.

The main requirements are:

• Dashboard to determine the cost
• Track Changes between model Revisions
• Record 3D & Data changes in models

3D Repo API

As part of the AEC Delta Mobility project, we have designed a detailed REST API specification.

3D Repo’s API implementation allows access and interaction with the wealth of data stored server-side. In fact, the API is also utilised by 3D Repo’s online interface. BIM models can be queried for their metadata and filtered through various revisions over time. In this particular example, we will perform a GET Request for the 'Get all metadata' and 'Get all metadata by revision' scenarios respectively. Response to the API call will be a serialised JSON.

In the next phase of the project, 3D Repo will expose new end-points to retrieve individual meshes per object in order to provide direct conversion to BHoM library and further processing throughout.

Custom Dashboard

Visualising data becomes useful in decision making or for monitoring the progress of a model. Visualisation of the data itself can also be achieved through interactive dashboards using Power BI such as the example below. This is achieved using the same API.

3D Model

As an example, we are using two revisions of a rail tunnel which experiences bending as well as elevation changes throughout its length. This makes it an interesting use case for our demonstration of how the server architecture can be managed.

The desired task is to detect and visualise model changes in a 3D environment. In the case of 3D Repo, the model comparison is based solely on geometry. In other words, the underlying object IDs are not taken into account so that any two revisions regardless of file or data format can be compared easily. Geometrical model changes are visualised as follows:

• Green for volumetric additions
• Red for volumetric deletions

3D Diff (in 3D Repo)

The resulting visualisation is fully interactive and computed in real-time client side. The downside of this approach is that the entire revision and not purely a delta has to be transmitted over the internet.

Model Data

In addition, BIM 3D models contain metadata at an object level. Any custom and default data is exported from the authoring tool and accessed through a web browser or using APIs as shown below:

Power BI Query

In order to establish a live link between the server and Power BI, a new Query has to be created. In this case, data is obtained from the Web and with an href link (URL) as written in the API Docs. The Delta difference between revisions is achieved using the following GET Requests:

api1.www.3drepo.io/api/_teamspace_/_model_/revision/master/head/meta/all.json?key=_APIKey_
api1.www.3drepo.io/api/_teamspace_/_model_/revision/_rev_/meta/all.json?key=_APIKey_

To perform a client-side differencing, one can append 'Get all metadata by revision' query (revision 01) to the 'Get all metadata' table (the latest model revision/revision 02. When merging the results of both API queries into a single table (All Metadata), it is necessary to ensure that a custom field 'Revision' with the relevant value attributed to each row is added. This way it is possible to distinguish the revisions.

Calculated Value (client-side diffing)

It is also possible to perform client-side differencing by creating a new quick measure in your Power BI Fields to highlight the change between revisions. In this example, we calculate the Volume Difference of each element by querying 'Element:Volume' parameter value.

Volume Difference = 
VAR __BASELINE_VALUE =
	CALCULATE(
		SUM('All Metadata'[Element:Volume]),
		'All Metadata'[Revision] IN { "Rev01" }
	)
VAR __MEASURE_VALUE = SUM('All Metadata'[Element:Volume])
RETURN
	IF(NOT ISBLANK(__MEASURE_VALUE), __MEASURE_VALUE - __BASELINE_VALUE)

---

3D Diff (in Speckle)

Revision_A:

Revision_B:

Diffing between Revision_A and Revision_B:

The diffing response (between Revision_A and Revision_B) as specified in the REST API has been implemented here. Calling http://{server.com}/api/streams/{streamId}/delta/{streamId} produces the following response:

{
	"success": true,
	"revision_datetime": "2/4/2020, 4:15:41 PM",
	"autor": {
		"name": "Paul",
		"surname": "Poinet",
		"_id": "5dfa34188b094663fcd94381",
		"email": "[email protected]",
		"company": "UCL"
	},
	"delta": {
		"created": [530 items],
		"deleted": [],
		"common": [3401 items]
	},
	"revision_A": {
		"id": "Zt0Jg36lp",
		"updatedAt": "2/4/2020, 3:08:00 PM",
		"sender": "Grasshopper"
	},
	"revision_B": {
		"id": "F9UVibS-C",
		"updatedAt": "2/4/2020, 3:08:00 PM",
		"sender": "Grasshopper"
	}
}

Endpoint call from the .NET client in Rhino:

---

SpeckleViz (Project Architecture)

SpeckleViz has been implemented to help users getting a better understanding of the data flow of a specific Speckle Project across users, streams and documents. As SpeckleViz directly reflects how your Streams are organized and relate to each other within a same Speckle Project, it will make more sense if your Streams have been created, named and organized in a meaningful way.

Where to find the graph

• You can simply find the graph at the bottom of your Speckle Project's page.
• Depending on how many Streams are contained within the Project and how many users have read/write access, the graph might take a few seconds to generate itself. So please be patient if you see a blank canvas at first.

---

Overview

The SpeckeViz interface is divided into 2 main parts: the control interface (composed of the main toolbar, the time range slider and the tag query selector) and the graph canvas itself. Each of these is further explained throughout the next sections.

---

Toolbar

To interact with the graph, your main weapon of choice is the toolbar, located above the interface. Each command is further explained throughout the next sections.

---

How to read the graph

• Circle nodes represent Senders 🚀 and Receivers 📡 (a.k.a local clients)
• Square nodes represent Streams 🛰️📦 (a.k.a a box of data on the cloud)
• 🚀--->🛰️📦 This pattern represents data that has been shared to a Stream by a user through a Sender
• 🛰️📦--->📡 This pattern represents data that has been retrieved by a user from a Stream through a Receiver
• 🚀--->🛰️📦--->📡 This pattern is the sum of the two patterns above...
• Generally, both nodes and edges are coloured according to their createdAt timestamps (dark blue for the newest created, light grey for the oldest). The edge's thickness corresponds to the number of objects being transferred through the respective stream.

---

• The pink clusters group the local clients per software file, or document. Below, a Grasshopper session contains two Speckle Receivers and one Speckle Sender. For now, each type of document is represented by a logo, such as:
  • Grasshopper: 🦗
  • Rhino3D: 🦏
  • Dynamo: 🔧
  • GSA: 💎

---

• The blue clusters group the local clients per user. The user's initials and company are exposed. Below, someone from UCL instantiated 5 Speckle Receivers and 3 Speckle Senders.

---

How to interact with the graph

• You can drag nodes around! Just hold left click on a node and drag it around.

---

• You can zoom in and zoom out by using the scroll wheel, and pan by holding left click on the background canvas and moving your mouse around it. If your graph disappears, you can still click on the GPS button situated on the toolbar, which will re-center it!

---

• As mentioned above, the clients are grouped either by Document GUID or User GUID. You can switch between the two by using the toggle switch situated on left side of the toolbar.

---

• Use the slider on the right side of the toolbar to expand/collapse the clusters. This might be useful to disentangle the wires and zoom in on a particular cluster.

---

• Right clicking on a stream will trigger a context menu from which you can:
  • view the stream within the viewer app
  • view the stream within the admin app
  • view the stream's raw data
  • view the stream's related client's raw data

---

• Use the slider below the toolbar to select and highlight a specific timeframe of your project!

When dragging the slider, a badge below it will update and inform you on the number of streams comprised within the specified time range. Clicking on the button itself will open the viewer interface within which the selected streams will be aggregated.

---

• If you modify your project in any way (by adding/removing streams to it, for example), you might want to hit the refresh button situated on the toolbar as the graph won't listen for changes and therefore won't update automatically.

---

• It is possible to anchor/release the graph by clicking on these icons situated on the toolbar.

---

• You can query streams by tag(s) (which you previously set through the management interface) with the search bar situated below the time range slider.

---

• The graph's layout and edges display can be customized through the toolbar buttons EDGES DISPLAY and GRAPH LAYOUT. Regarding the edges display, you can choose between Line, Arc, Diagonal Horizontal, Diagonal Vertical and Diagonal Smart. You can also set the overall graph's layout as Free, Horizontal or Vertical

---

• Finally, you can save your graph as .PNG and share it with your team!

---

Example

This video showcases how the graph can be deployed onto a design data set, from design to fabrication stages.