diff --git a/docs/assets/api-keys-menu.png b/docs/assets/api-keys-menu.png new file mode 100644 index 0000000..f74e733 Binary files /dev/null and b/docs/assets/api-keys-menu.png differ diff --git a/docs/assets/connect-to-atom-menu.png b/docs/assets/connect-to-atom-menu.png new file mode 100644 index 0000000..6ec0f78 Binary files /dev/null and b/docs/assets/connect-to-atom-menu.png differ diff --git a/docs/assets/user-account-menu.png b/docs/assets/user-account-menu.png new file mode 100644 index 0000000..2220b99 Binary files /dev/null and b/docs/assets/user-account-menu.png differ diff --git a/docs/documentation.markdown b/docs/documentation.markdown index 93f1db4..e250539 100644 --- a/docs/documentation.markdown +++ b/docs/documentation.markdown @@ -267,12 +267,10 @@ of your workgroup. You can upload files or folders to this space for your own personal content, which would not normally be part of a group archive. -Files uploaded into Personal are virus checked once and cannot be moved -out into any other area of Curate TM. - -If you have files in Personal that you later want to add to the archive, -you will need to re-upload into Quarantine, or if you don't have a local -copy, download from Personal and then re-upload into Quarantine. +Files uploaded into Personal are virus checked once and should not be moved +out into any other area of Curate TM. +If you have files in Personal that you do want to add to the archive, you should move them +into Quarantine so they can be submitted for a full quarantine period. Files in Personal are not characterised, but you do have basic file information available and you can add descriptive metadata. Note that @@ -287,16 +285,14 @@ uploaded into the Quarantine workspace [(*see upload and ingest for more informa members of a workgroup. Uploaded files are virus checked and quarantined for 30 days. It is recommended that files should not be moved from the Quarantine workspace until the quarantine period is up, however -Curate TM is not prescriptive and you can do this. +Curate TM is not prescriptive and you can choose to move files out early if you wish. If the virus scanner detects a virus or malware then the affected file will be moved to a folder called Infected within the Quarantine workspace. After the 30-day quarantine period and a second successful virus scan, -files are moved automatically into the Appraisal space into a folder -called Released (but note that this folder will not appear until it is -needed). +files are moved automatically into the Appraisal space and labelled "Released". #### Appraisal Space @@ -305,22 +301,21 @@ file characterisation tool Siegfried, which generates a file type unique identifier from the PRONOM registry of filetypes maintained by the National Archive (see ). -If you search in PRONOM by a file's unique identifier you will be able -to see detailed information on the file format. Within the Appraisal -workspace you can move, copy and delete files, create folders and add +Once Siegfried has completed its analysis, the identified PRONOM ID is added to the file information area. +Clicking on the PRONOM ID will open a new tab in your browser linked to the PRONOM registry page with more detailed information on the file format. + +Within the Appraisal workspace you can move, copy and delete files, create folders and add descriptive metadata. Using detailed file information you can make appraisal decisions. From the appraisal space, you can also preserve and -package your objects in any configuration you wish. [*See preservation*](#preservation) and [*packaging*](#packaging) for more information. +package your objects in any configuration you wish. [*See appraisal*](#appraisal-selection-and-arrangement) [*preservation*](#preservation) and [*packaging*](#packaging) for more information. #### Archive Space Files and folders can be moved to the Archive workspace for long-term retention. The organisation of the workspace is up to you, but the search function can help any system you have by allowing simple searches -on filenames or advanced searches on descriptive metadata. Files, -folders or packages moved to the Archived File workspace are read only -(you cannot delete or edit them) and cannot be moved, but you can create -a copy into another workspace. +on filenames or advanced searches on descriptive metadata. AIPs generated +by the preservation process are also uploaded to the Archive workspace by default. #### Common Files Space @@ -334,9 +329,11 @@ The object information area provides you with all the critical details about a file or object inside Curate. The object information area will appear on the right-hand side of the screen when you select a single file or folder. +
+ #### Object information area panels ##### File actions panel @@ -377,7 +374,7 @@ information area. Click on "more" to view the full extracted EXIF data. Clicking on any file or folder in Curate -will summon the metadata panel in the object information area. The +will open the metadata panel in the object information area. The Curate metadata panel comes with simple 15 field Dublin Core and 26 field ISAD(G) schemas built in. Any additional standardised or customized schemas can be supported upon request. There is also a field @@ -414,8 +411,7 @@ The final panel in the object information area, when a file or folder is selecte is the file activity panel. The file activity panel records and displays a log of all the actions and modifications taken that relate to the selected object. Please note that the activity stream does not record PREMIS -preservation events. Those are recorded internally. See [*PREMIS in -Curate*](#) for more information. +preservation events. Those are recorded internally. See [*PREMIS in Curate*](#premis-in-curate) for more information. ## Upload and Ingest @@ -449,11 +445,6 @@ This will greatly improve your upload performance. We would strongly recommend using the advanced web-uploader for the majority of your ingests, where the size and volume is appropriate. -*If you need to ingest content using another protocol, either to adhere -to institutional policy or for workloads that are larger than the -webuploader is able to handle, please read the* *Curate Mission Control -and SFTP sections* - Try uploading files and folders (you can upload any complex folder structure) using the advanced web-uploader. You can do this by selecting 'New' at the top of the screen, and then Upload. @@ -567,29 +558,18 @@ To resolve this, you will need to generate a JSON manifest of checksums for your ingest content before you perform your SFTP upload, and then perform manual ingest integrity verification once it is complete. [*See Manual Ingest Integrity Verification for more information.*](#manual-ingest-integrity-verification) -
-mission control features powerful uploading capabilities, as -performant as SFTP, as well as checksum generation. It's always strongly -recommended to use mission control over SFTP for massive ingests for a -faster, more robust, less fragmented and more pleasant user experience. -
- Once you have generated your checksum manifest, you may proceed to upload your content via SFTP. Once your content is uploaded, and we have completed the necessary reindex, you can upload your checksum manifest. [*See Manual Integrity Verification for more information*](#manual-ingest-integrity-verification) -### Curate Mission Control - -For massive ingest workloads that exceed the capabilities. - ### Ingest Integrity Verification Curate offers powerful methods for ingesting content into the system which come with automated facilities for verifying that your files were not compromised as they were uploaded to the system. -Uploads using either the Advanced Web-uploader or mission control will +Uploads using the Advanced Web-uploader will have their integrity verified completely automatically. Curate will notify you with status tags attached to the respective object if any items in your ingest workload are found to have been compromised during @@ -598,20 +578,59 @@ the upload process. Uploads using SFTP require manual checksum verification, which you can read more about below. -### Manual Ingest Integrity Verification +#### Manual Ingest Integrity Verification If you spot an inconsistency in the checksums or verification results generated by Curate during your ingest upload, if you'd like to manually verify that the automatic validation was completed successfully, or if -you have chosen to upload your content using SFTP, you might need to +you have chosen to upload your content using a method that does not support automatic validation, you might need to perform manual ingest integrity verification. To perform manual ingest integrity verification, follow these steps: -#### Generating a checksum manifest +##### Generating a checksum manifest + +There are free several tools available to do this, but your chosen tool must be able to generate MD5 checksums, and it must be +able to provide a manifest file in this format: + +**Comments and Metadata:** Lines starting with a semicolon (;) should include metadata about the checksum generation, such as the software used, URL for more information, and the timestamp of generation. + +**Checksum Entries:** Each line contains an MD5 checksum followed by the filename, separated by an asterisk (*) + +
+
+ Example Function + JavaScript +
+ 
+            
+                [md5_checksum] *[filename]
+            
+
+
+ +**Footer:** Optionally, a footer can summarize the process, e.g., total number of files hashed. + +**Example of a Properly Formatted Checksum Manifest** + +
+
+ Example Function + JavaScript +
+ 
+            
+                ; Checksums generated by ExampleSoftware 1.0.0
+                ; http://www.example.com
+                ; 4/16/2023 9:25:42 PM
+                \n
+                f5c5777e2e66258b2b4e1a5820fb3eff *CNA_HIST_ADAQuestionsAndAnswersDocument.mp3
+                ae7f5ebd507470dc7944cb64909ec665 *CNA_HIST_AdvisoryMeeting_2007_12_12.mp3
+                ; 2 files hashed.
+            
+
+
-There are free several tools available to do this, but we would strongly -recommend using our companion tool "mission control". [*See Curate mission control for more information.*](#curate-mission-control)
For best practice, we recommend uploading your checksum manifest @@ -1247,6 +1266,70 @@ window, which should be labelled with a truncated version of your user ID and select Support. This will take you to a contact form which goes directly to the Penwern support team. +## Integrations + +### Microsoft Sharepoint + +Curate's Sharepoint integration provides a robust and user-friendly solution for organisations to safeguard their valuable digital content stored in Microsoft 365 environments. This integration seamlessly connects Sharepoint with Curate, enabling efficient management and long-term protection of records directly from within Sharepoint. + +This documentation will guide you through setting up the integration between Curate and Sharepoint, and also using the feature. + +NB: Curate Sharepoint is an additional feature available to Curate Enterprise customers. If you have not yet added Sharepoint integration to your Curate Enterprise contract and you would like to explore your options, please get in touch with us. + +
+ + + Things you'll need: +
+
    +
  • Administrative access to your organisations Sharepoint admin centre, or help from someone who does.
    • +
  • Permission to create an API key for Curate in your Sharepoint admin centre that confers read access to Document Libraries in which you would like the Curate integration to be available.
    • +
  • Permission to upload and deploy Sharepoint extensions to your organisations Sharepoint environment.
    • +
  • An account on your organisations Curate Enterprise system with a user-admin tier role.
    • +
  • 15-20 minutes of time.
    • +
+ +
+ +#### Security Setup + +To connect your Sharepoint system to Curate, you will first need to give both Curate and Sharepoint sufficient permissions to interract with eachother securely. + +**Explanation** + +Both Curate and Sharepoint are highly secure platforms protected by comprehensive access protocols that require explicit and controlled access in order to allow users, or in this case eachother, to perform useful actions amongst themselves. + +In order to facilitate a seamless user-experience, the Curate Sharepoint integration leverages the Microsoft Graph API, which enables your Curate system to programatically access data stored in Microsoft services like Sharepoint. When you preserve a file with Curate from Sharepoint, you actually just tell Curate which files you would like to preserve, rather than supplying it the files directly. Curate then uses it's access to your content via the Graph API to directly stream the specified files. + +Not only does this make the Preserve action *instant*, no matter the size of your file selection, but it also means data movement is kept to a minimum, performance is improved, and the stability and robustness of your local connection is irrelevent. Uploads of any size are managed completely automatically with no requirement for your device to remain connected to a network, or for you to monitor its progress. + +As a result, to access and retrieve your specified data, Curate requires specific permissions to use your Sharepoint data. Similarly, Sharepoint requires specific permissions from Curate in order for Curate to allow your requests to be authenticated and actioned securely. + +All traffic between Curate and a properly configuree Sharepoint environment is implicitly encrypted in transit by the HTTPS protocol, and your data never leaves a secure stream set directly between your Curate and Sharepoint systems, which are both highly protected platforms with thorough authentication systems. + +Ultimately, this solution is much more secure, robust and frictionless than the alternative of downloading content to your, potentially risky, local client and sending the data to Curate thereafter. + +#### Generating a Curate API Key + +First, lets generate a Curate API key which will be supplied to Sharepoint in order for your Curate system to authenticate requests coming in from your Sharepoint environment. + +
You should only generate an API key for any system if you understand what you are doing and the potential risks. If you are unsure and would like some advice, please get in touch with us. You should follow common best-practice guidance for storing, securing and retaining your API keys.
+ +To generate a Curate API key: + +- log in to a Curate account with at least a user-admin tier role for your organisation. +- Open the user account menu by clicking your account profile icon in the top left hand corner of the Curate interface (it's the same menu you log-out from). + +
+ +
+ +- Select the "API Keys " button to open the API keys menu. + +
+ +
+ # Roadmap and Suggestions Curate is an incredibly dynamic and flexible platform on which it is easy for us diff --git a/docs/style.css b/docs/style.css index 0dd504f..fe96d12 100644 --- a/docs/style.css +++ b/docs/style.css @@ -106,3 +106,38 @@ .md-header__topic:first-child { font-weight: 300; } +.code-block { + background-color: #2d2d2d; + border-radius: 8px; + box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); + overflow: hidden; + width: 90%; + max-width: 600px; +} +.code-header { + background-color: #1e1e1e; + padding: 10px 20px; + display: flex; + justify-content: space-between; + align-items: center; +} +.code-title { + color: #ffffff; + font-size: 14px; + font-weight: bold; +} +.code-language { + color: #999999; + font-size: 12px; +} +.code-block pre { + margin: 0; + padding: 20px; + overflow-x: auto; +} +.code-block code { + font-family: 'Courier New', Courier, monospace; + color: #ffffff; + font-size: 14px; + line-height: 1.5; +} \ No newline at end of file