Skip to content

Server Architecture

Paul Poinet edited this page Feb 5, 2020 · 31 revisions

diagram

  • Rhomberg DGN file to 3D Repo then to BHoM

  • 3D Repo currently does not support object definitions directly, instead, it stores meshes with metadata

  • Web sockets for event updates

  • But direct requests for actual data retrieval

Implementation of User Story 1.2

  • Dashboard to determine the cost
  • Track Changes between model Revisions
  • Record 3D & Data changes in models

Power BI API Diagram

Custom Dashboard

Visualising your data comes in handy in making decisions or for monitoring the progress of your model. Visualise your project’s data through interactive dashboards using Power BI. Dashboard

3D Model

Tunnel - Rev02

Detect and visualise model changes in a 3D environment. Model comparison is based solely on geometry and object IDs are not taken into account. Geometrical model changes are visualised as following:

  • Green for anything Added
  • Red for anything Deleted

3D Diff (in 3D repo)

3D Diff

Model Data

3D Model contains metadata at an object level. Any custom and default data is exported from the authoring tool. You can access this data via a web browser or using APIs.

Element Data

3D Repo API

3D Repo’s API allows access and interaction with the wealth of data stored on 3D Repo’s servers. In fact, the API is utilised by 3D Repo’s online interface, meaning that all of the functionalities seen there can also be accessed via the API. You can query any of your models for their metadata and filter through various revisions, should there be any. In this particular case, we will perform GET Request for the 'Get all metadata' and 'Get all metadata by revision'. Response to our API call will be a serialised JSON.

API Call Metadata All Get Metadata of a Specific revision

Power BI Query

In order to establish a live link between 3D Repo's database and Power BI, you will need to create a new Query. In this case, you will get data from Web and for the URL link, you can use a href link (URL) as written in the API Docs. In order to highlight the difference (Delta) between revisions, we will need to perform the following GET Requests:

  • api1.www.3drepo.io/api/teamspace/model/revision/master/head/meta/all.json?key=APIKey
  • api1.www.3drepo.io/api/teamspace/model/revision/rev/meta/all.json?key=APIKey

To perform a client-side diffing, you can append 'Get all metadata by revision' query (revision 01) to the 'Get all metadata' table (the latest model revision/revision 02. When merging the results of both API queries into a single table (All Metadata), ensure that you add a custom field 'Revision' with the relevant value attributed to each row. This way you will be able to distinguish revisions.

Web API Request Query Table

Calculated Value (client-side diffing)

Create a new quick measure in your Power BI Fields to highlight the change between revisions. In this example, we will calculate the Volume Difference of each element by querying 'Element:Volume' parameter value.

Volume Difference = 
VAR __BASELINE_VALUE =
	CALCULATE(
		SUM('All Metadata'[Element:Volume]),
		'All Metadata'[Revision] IN { "Rev01" }
	)
VAR __MEASURE_VALUE = SUM('All Metadata'[Element:Volume])
RETURN
	IF(NOT ISBLANK(__MEASURE_VALUE), __MEASURE_VALUE - __BASELINE_VALUE)


3D Diff (in Speckle)

Revision_A:

MeshPartial

Revision_B:

MeshAll

Diffing between Revision_A and Revision_B:

MeshDiff

The diffing response (between Revision_A and Revision_B) as specified in the REST API has been implemented here. Calling http://{server.com}/api/streams/{streamId}/delta/{streamId} produces the following response:

{
	"success": true,
	"revision_datetime": "2/4/2020, 4:15:41 PM",
	"autor": {
		"name": "Paul",
		"surname": "Poinet",
		"_id": "5dfa34188b094663fcd94381",
		"email": "[email protected]",
		"company": "UCL"
	},
	"delta": {
		"created": [530 items],
		"deleted": [],
		"common": [3401 items]
	},
	"revision_A": {
		"id": "Zt0Jg36lp",
		"updatedAt": "2/4/2020, 3:08:00 PM",
		"sender": "Grasshopper"
	},
	"revision_B": {
		"id": "F9UVibS-C",
		"updatedAt": "2/4/2020, 3:08:00 PM",
		"sender": "Grasshopper"
	}
}

SpeckleViz (Project Architecture)

SpeckleViz has been implemented to help users getting a better understanding of the data flow of a specific Speckle Project across users, streams and documents. As SpeckleViz directly reflects how your Streams are organized and relate to each other within a same Speckle Project, it will make more sense if your Streams have been created, named and organized in a meaningful way.

Where to find the graph

  • You can simply find the graph at the bottom of your Speckle Project's page.
  • Depending on how many Streams are contained within the Project and how many users have read/write access, the graph might take a few seconds to generate itself. So please be patient if you see a blank canvas at first.

where

--

Overview

The SpeckeViz interface is divided into 2 main parts: the control interface (composed of the main toolbar, the time range slider and the tag query selector) and the graph canvas itself. Each of these is further explained throughout the next sections.

specklevizinterface2

--

Toolbar

To interact with the graph, your main weapon of choice is the toolbar, located above the interface. Each command is further explained throughout the next sections.

toolbarclientgraph

--

How to read the graph

  • Circle nodes represent Senders 🚀 and Receivers 📡 (a.k.a local clients)
  • Square nodes represent Streams 🛰️📦 (a.k.a a box of data on the cloud)
  • 🚀--->🛰️📦 This pattern represents data that has been shared to a Stream by a user through a Sender
  • 🛰️📦--->📡 This pattern represents data that has been retrieved by a user from a Stream through a Receiver
  • 🚀--->🛰️📦--->📡 This pattern is the sum of the two patterns above...
  • Generally, both nodes and edges are coloured according to their createdAt timestamps (dark blue for the newest created, light grey for the oldest). The edge's thickness corresponds to the number of objects being transferred through the respective stream.

oldNew

--

  • The pink clusters group the local clients per software file, or document. Below, a Grasshopper session contains two Speckle Receivers and one Speckle Sender. For now, each type of document is represented by a logo, such as:
    • Grasshopper: 🦗
    • Rhino3D: 🦏
    • Dynamo: 🔧
    • GSA: 💎

clusterDoc

--

  • The blue clusters group the local clients per user. The user's initials and company are exposed. Below, someone from UCL instantiated 5 Speckle Receivers and 3 Speckle Senders.

clusterUser

--

How to interact with the graph

  • You can drag nodes around! Just hold left click on a node and drag it around.

moveNode

--

  • You can zoom in and zoom out by using the scroll wheel, and pan by holding left click on the background canvas and moving your mouse around it. If your graph disappears, you can still click on the GPS button situated on the toolbar, which will re-center it!

scrollpanzooom2

--

  • As mentioned above, the clients are grouped either by Document GUID or User GUID. You can switch between the two by using the toggle switch situated on left side of the toolbar.

switchGroups

--

  • Use the slider on the right side of the toolbar to expand/collapse the clusters. This might be useful to disentangle the wires and zoom in on a particular cluster.

expandcollapse

--

  • Right clicking on a stream will trigger a context menu from which you can:
    • view the stream within the viewer app
    • view the stream within the admin app
    • view the stream's raw data
    • view the stream's related client's raw data

streamMenu

--

  • Use the slider below the toolbar to select and highlight a specific timeframe of your project!

timerange inspectTimeFrame

When dragging the slider, a badge below it will update and inform you on the number of streams comprised within the specified time range. Clicking on the button itself will open the viewer interface within which the selected streams will be aggregated.

inspectTimeFrameSnap

--

  • If you modify your project in any way (by adding/removing streams to it, for example), you might want to hit the refresh button situated on the toolbar as the graph won't listen for changes and therefore won't update automatically.

refresh

--

  • It is possible to anchor/release the graph by clicking on these icons situated on the toolbar.

fixrelease

--

  • You can query streams by tag(s) (which you previously set through the management interface) with the search bar situated below the time range slider.

queryselector

--

  • The graph's layout and edges display can be customized through the toolbar buttons EDGES DISPLAY and GRAPH LAYOUT. Regarding the edges display, you can choose between Line, Arc, Diagonal Horizontal, Diagonal Vertical and Diagonal Smart. You can also set the overall graph's layout as Free, Horizontal or Vertical

layouts

--

  • Finally, you can save your graph as .PNG and share it with your team!

saveaspng

--

Example

This video showcases how the graph can be deployed onto a design data set, from design to fabrication stages.