Skip to content

SNAC Architecture Overview

Jump to bottom
jglass-st edited this page Jul 24, 2024 · 1 revision

SNAC Overview

Useful Links

SNAC Architecture

Architecture Diagram

Snac is built in a server-client architecture. The server code is located in /src/snac/server, and the frontend client code is largely located in /src/snac/client. User HTTP requests come in to the frontend client, the client queries the backend, receives a response, and returns a page or response to the browser.

Frontend Client

WebUI.php - acts as a thin router layer. Receives requests from src/virtualhosts/www/index.php and calls the appropriate function in WebUIExecutor.php. WebUIExecutor.php - Handles the logic of the front-end client. Runs the function called by WebUI, does some validation checks on input, and sends the request to the backend Server.php. When it receives a response, it sets the data to send to the view template, and selects the correct template to display.

Templates (aka Views) - The views are held in src/snac/client/webui/templates. They are built with PHP Twig Templating (similar to ERB). The interactive edit forms are built with JS and jQuery, and we use jQuery Datatables for our searchable and filterable tables. Styling is mostly Bootstrap 3 CSS and some custom css. A few files of note:

  • view_page.html - Main constellation profile view page.
  • edit_page.html - Constellation editing interface.
  • /resources/view.html - Resource view page.
  • edit_scripts.js - Required javascript for the Constellation editing interface. It transforms the fields on the editing interface from inactive to active fields, handles adding new fields, undoing, deleting and submitting.
  • browse_scripts.js - Handles javascript for the browse page.

Backend

  • Server.php - Checks user permissions and routes the request from the WebUIExecutor to the appropriate function in ServerExecutor.php
  • ServerExecutor.php - Handles much of the logic of the application.
  • Calls DBUtil, Neo4jUtil, ElasticSearchUtil
  • DBUtil (aka cStore) - a homemade ORM that calls SQL.php to perform SQL queries and then builds Constellation and Resource PHP objects.
  • SQL.php - Holds all SQL queries.
  • Neo4jUtil - Connects to Neo4j database for relationship data.
  • ElasticSearchUtil - Connects to ElasticSearch for most oro our fast searching functionality.

Databases

PostgreSQL Database

We use postgres as our main database and authoritative data store. Changes are first made in postgres and then synced to Neo4j and ElasticSearch when changes are published.

Versioning

The version_history table holds the constellation ID and version number. The version field is a global sequence shared across all constellation-related tables.

When a field of a versioned table is edited, rather than changing the field in postgres, an entirely new row is inserted with the new values and an incremented version number. The current state of the constellation is pulled by querying for the max published version for a given constellation id. This allows us to see the exact state of the constellation in each changed version number as well as who made that change. The downsides are greatly increased query complexity, slower development and slower joins.

Versioning is only tracked in postgres. ElasticSearch, Neo4j are not versioned, and keep only the current published state.

ElasticSearch

ElasticSearch is used for fast indexing and searching, comparable to Solr.

Neo4j

Neo4j is a graph database that represents data as nodes and edges pointing to one another. We use it when we need more in-depth relationship querying, as well as for our visualizations. We use the Cypher language to query the Neo4j database.

For example Nancy Bragdon and John Brown are connected to each other, and their relationship is visible in their connection graph. They are both listed as creatorsOf the Resource "Letters to Nancy W. Bragdon", which is held by the Smith College Neilson Library.

SNAC Terminology

Constellation - A record of a Person, Family, or Corporate Body, and its relationships and metadata. Basically means an entity or agent record. Sometimes called "Identity Constellation" and often abbreviated as "ic" (e.g. ic_id, current_ic_id) e.g. Alan Turing's profile , Alan Turing as json

Resource - A description of an archival record. They're displayed on the Resources tab of any profile, or you can search for them on the Resources search page.

e.g. J.H. Wilkinson Interview, Alan Turing Papers.

Ark ID - alternative ID used for making permalinks to SNAC

Endpoints

REST API: http://snac-dev.iath.virginia.edu/api

SNAC Dev API docs API Test area Example scripts interacting with REST API The REST API can be queried via PUT requests with json data containing the command and information. For commands that require authentication, you'll need to generate an API key.

Example Query

curl -X PUT  http://snac-dev.iath.virginia.edu/api/ -d '{"command": "read", "constellationid": "536430"}'

OpenRefine Endpoint Used for reconciling name strings with SNAC identities.

Clone this wiki locally