Islandora is an open-source framework that provides the necessary tools to use a Drupal website as a fully-functional Digital Assets Management System.
Islandora:
Can handle messages at scale - Islandora created Alpaca, an integration middleware based on Apache Camel, to handle messaging and queueing at an enterprise scale. To the user, this means large batch uploads can be processed gracefully.
Offers digital preservation features - Using a robust storage layer for preservation (Fedora), and generating technical metadata with FITS, are digital preservation tools provided by our automated installation methods. Community members have created additional features for doing digital preservation using Islandora, which are not yet part of our automated setup.
Try Islandora without installing anything at sandbox.islandora.ca.
Login credentials for the sandbox can be found here.
Anyone can log in to this sandbox as an administrator and explore the interface! However, this site is refreshed every night so your changes will not be permanent. This site uses the Islandora Starter Site. This is not the only way that Islandora can be made to work! This sandbox includes some sample content and configuration (such as views and blocks) to increase its usefulness as a sandbox. Learn more about the Islandora Starter Site.
"},{"location":"#ansible-playbook","title":"Ansible Playbook","text":"Islandora can be installed via an an Ansible Playbook, a machine-operable set of instructions which provisions the full Islandora stack. It can be used to create a locally installed Islandora (requiring your computer to have Vagrant and Virtualbox) or can be used to provision a remote Linux server. The provisioning process involves many steps where software is downloaded and installed, so it can take a while. There is an option to get a basic (\"standard\") site, or to install a suite of demo configurations known as the Demo Install Profile. See Installation - Ansible Playbook for more details.
"},{"location":"#docker","title":"Docker","text":"Islandora sites can also be created using Docker. This can be done using the ISLE-DC project, which launches the Islandora Docker images created by Isle Buildkit. Like in the Ansible Playbook, there is an option to use a pre-built demo site, or build it completely from scratch. (There is also an option to build the demo site on command, which takes a bit more time, or to use ISLE-DC to build up the environment to support a Drupal site that you have already exported.) See Installation - Docker ISLE for more details
"},{"location":"#join-the-community","title":"Join the Community","text":"The Islandora community is an active group of users, managers, librarians, documenters, and developers from galleries, libraries, archives, museums, and other institutions worldwide. We welcome discussion and contribution through various mailing lists, channels, interest groups, and calls. The Islandora community operates under the Islandora Code Of Conduct. See our Contributing Guidelines for more information.
Documentation for previous versions
Documentation for Islandora 6 and 7 is on the Lyrasis documentation wiki.
"},{"location":"placeholder/","title":"Placeholder page","text":"This page serves as a placeholder for functionality that still needs documentation.
Interested in contributing to the Islandora documentation? Join the Community!
The Islandora community is an active group of users, managers, librarians, documenters, and developers from GLAM (and beyond!) institutions worldwide. We welcome discussion and contribution through various mailing lists, channels, interest groups, and calls. The Islandora community operates under the Islandora Code Of Conduct. See our Contributing Guidelines for more information, or drop by a meeting of the Documentation Interest Group for a helping hand.
Documentation for previous versions
Documentation for Islandora 6 and 7 is on the Lyrasis documentation wiki.
"},{"location":"alpaca/alpaca-technical-stack/","title":"Alpaca Technical Stack","text":"As of version 2.0.0, Alpaca contains several tools bundled into a single runnable jar file. The different tools can be enabled/disabled depending on the configuration you define.
"},{"location":"alpaca/alpaca-technical-stack/#gradle","title":"Gradle","text":"Gradle is used by Alpaca as a build and package management tool. It is similar to Maven.
"},{"location":"alpaca/alpaca-technical-stack/#apache-camel","title":"Apache Camel","text":"Apache Camel is an integration framework that aids in implementing integration patterns.
"},{"location":"alpaca/alpaca-technical-stack/#apache-activemq","title":"Apache ActiveMQ","text":"Apache ActiveMQ is a JMS compliant Messaging Queue. Messaging client can make use of JMS to send messages.
"},{"location":"alpaca/alpaca-technical-stack/#installing-activemq","title":"Installing ActiveMQ","text":"Installing ActiveMQ is relatively easy. Download the latest stable release here. Go to the activemq_install_dir/bin. Start the ActiveMQ by using the activemq script or batch file and start command.
$ cd activemq_install_dir/bin\n$ ./activemq start\nWhen ActiveMQ gets started, go to http://localhost:8161/admin/. You can login using admin:admin.
Note that ActiveMQ in Islandora playbook does not have a UI.
"},{"location":"alpaca/alpaca-technical-stack/#references","title":"References","text":"Collections are groups of related content that can be viewed or managed as a unit. Islandora-specific use cases include:
Islandora provides:
field_member_of). This mechanism is also used by Paged Content and Compound Objects. Islandora on its own does not prescribe any particular Content Type, so this field can be configured for any node bundle intended to represent Islandora resources.field_model) which can take various values including \"Collection\".Islandora Starter Site is an optional set of presets for Islandora, intended to provide a more user-friendly out-of-the-box experience and starting point for more specific customization.
Islandora Starter Site provides:
field_member_of field, so that users may add nodes of this type to a collection (or paged content, or compound resource),For more details, see the tutorial on How to create and add to a collection
"},{"location":"concepts/collection/#bulk-management-of-members-of-a-collection","title":"Bulk management of members of a collection","text":"Bulk management of items can be done from a Views Bulk Operations-compatible View, including the main Drupal Content View (at admin/content) and the Manage Members View (at node/[node]/members). This is possible due to the Drupal contrib modules Views Bulk Operations (VBO) and Views Bulk Edit. VBO allows you to perform Drupal Actions on objects. These Actions include making Islandora Derivatives such as \"Audio - Generate a Service File from an Original File\" and also to perform Drupal core Actions such as publishing/unpublishing content. The Views Bulk Edit module extends VBO and allows you to edit field values (such as the Member Of field, which would change which collection a group of nodes are in).
For instance, if you want to move a number of nodes from one collection into another, then you can
For more information see the video tutorial on Batch Editing.
"},{"location":"concepts/collection/#deleting-a-collection","title":"Deleting A Collection","text":"Deleting Collections creates orphans.
When you delete a Collection object, it will be deleted from the \"Member of\" field of all its Children. This means that it will not be possible to track down the children after the Collection has been deleted. If you intend to move the children, see above. If you intend to delete the children, this must be done before-hand. This is a known issue.
Deleting content creates orphan media.
Deleting content (Collections or otherwise) will NOT delete any attached Media. This is a known issue.
"},{"location":"concepts/collection/#permissions-on-a-collection","title":"Permissions on a Collection","text":"No reliable access control methods currently allow you to assign people to administer and manage only specific collections.
"},{"location":"concepts/derivatives/","title":"Derivatives","text":"Derivatives are files, usually generated automatically from a source file, which may be useful to the repository. Examples of derivatives include:
There are two schemes you can use for derivatives:
Derivative configuration is stored using Drupal's Actions. This means that all derivative configuration, such as parameters dictating the derivative size and quality can be edited by a repository administrator in the Drupal GUI under Manage > System > Actions.
As Actions, they can be executed on nodes manually using Views Bulk Operations. They can also be configured to run automatically on media save thanks to Islandora's additions to the Drupal [Contexts] module.
Derivative actions will replace existing files
If a derivative action runs but the target derivative (as identified by its taxonomy term) already exists, the new file will replace the old file (leaving the rest of the Media intact).
"},{"location":"concepts/derivatives/#derivatives-can-run-automatically-with-contexts","title":"Derivatives can run automatically with Contexts","text":"With the Contexts module, you can configure derivatives to run under specific conditions. These are set in the \"Conditions\" section. Islandora provides a number of context conditions including:
You can set up as many of these as you like, with \"and\" or \"or\" logic between them.
In the \"Reactions\" section is where you can set up derivatives. For standard model derivatives, choose the \"Derivatives\" reaction. This lists all actions, including derivative actions. Note that multifile derivatives won't work here.
Multi-file media
The multi-file media derivatives can NOT be selected from within the \"Derivatives\" reactions. From the \"Reactions\" pop-up window, you must choose \"Derive file for Existing Media\". This panel lists only Multi-file media-type derivatives.
"},{"location":"concepts/derivatives/#derivatives-have-types","title":"Derivatives have Types","text":"When creating a new Derivative Action, there are a number of flavours of derivative \"types\" available. All derivative Actions fall into one of these types.
Derivative type name machine name Supplying module Expected Microservice (software) Generate a Technical metadata derivativegenerate_fits_derivative Islandora FITS (roblib) Crayfits (FITS) Generate a audio derivative generate_audio_derivative Islandora Audio Homarus (FFmpeg) Generate a video derivative generate_video_derivative Islandora Video Homarus (FFmpeg) Generate an image derivative generate_image_derivative Islandora Image Houdini (ImageMagick) Get OCR from image generate_ocr_derivative Islandora Text Extraction Hypercube (tesseract/pdftotext) Multi-file media
The derivatives types available for multi-file media are the ones marked as \"for Media Attachment\" e.g. \"Generate an Image Derivative for Media Attachment\".
"},{"location":"concepts/derivatives/#derivatives-are-created-by-microservices-crayfish","title":"Derivatives are created by microservices (\"Crayfish\")","text":"In Islandora, we generate derivatives using microservices, which are small web applications that do a single task. Each microservice takes the name of a file as well as some parameters. It runs an executable and returns a transformed file, which can be loaded back into the repository. The microservices in Islandora stack are:
Repository Microservice name executable Crayfish Homarus FFmpeg Crayfish Houdini ImageMagick Crayfish Hypercube tesseract/pdftotext Crayfits Crayfits FITS"},{"location":"concepts/derivatives/#derivatives-are-created-using-an-external-queue","title":"Derivatives are created using an external queue","text":"To send orders to the microservices, Islandora sends messages to an external queue, from which the microservices process the jobs as they are able. This is a robust system that can \"operate at scale\", i.e. can large handle batches of uploads without slowing down the repository.
The queue system used by Islandora is ActiveMQ, and the listeners are part of \"Alpaca\"
"},{"location":"concepts/derivatives/#derivative-swimlane-diagram","title":"Derivative Swimlane Diagram","text":"The following diagram shows the flow of derivative generation from start to finish. A user saves a Media in Drupal, which may trigger Drupal to emit a derivative event to a queue, which is read by Alpaca and sent to a microservice. The microservice gets the original file and makes a transformation, returning the derivative file to Alpaca, which sends it back to Drupal to become a Drupal Media.
"},{"location":"concepts/node-concepts/","title":"Node Concepts","text":"This page describes the core fields and features that Islandora uses to manage content as nodes in an Islandora repository.
"},{"location":"concepts/node-concepts/#nodes-hold-metadata","title":"Nodes hold metadata","text":"In Islandora, Drupal nodes are created to hold descriptive metadata about content in the repository. This metadata is held in the usual way for nodes, which is by populating Drupal fields. Fields are configured on a Drupal content types, which serve as metadata profiles.
"},{"location":"concepts/node-concepts/#nodes-can-have-memberschildren","title":"Nodes can have members/children","text":"To build the capacity for creating hierarchical structures like collections, books and their pages, and complex objects, Islandora introduces a special field, \"Member of\" (field_member_of) which must be present on all Islandora content types. It enables a \"Children\" tab to display on Islandora nodes which lists the children of that node, and allows a repository manager to perform bulk operations on them. It also enables a repository manager to create children of nodes directly, individually or in bulk.
Within a single content type (i.e. metadata profile), Islandora provides the ability to designate some objects as different \"types\" than others. Key behaviours, such as what derivatives are created or what viewer is used, can be configured (see Contexts) based on this value. The available values are taxonomy terms in the Islandora Models vocabulary, and they are attached to nodes via the special mandatory field, \"Model\" (field_model), which must be present on all Islandora content types. These values are installed through a Drupal Migration after the Islandora module is installed. All installation methods perform this migration, so out of the box, the following values should be available in the Islandora Models vocabulary:
With Islandora alone, choosing a value from this list will have zero effects. The contingent behaviour must be configured during repository implementation. Islandora Starter Site provides an example of what behaviours are possible for these types.
The External URI field
This vocabulary, like many others in Islandora, includes an External URI field. This is intended to be used when transforming Islandora content into RDF, but also serves to make it easier to share configuration. Islandora provides code so that context conditions and derivative configs can be created without referencing the taxonomy term by ID, rather, using the taxonomy term's External URI. Since terms IDs are likely to change across sites, this makes our configs more portable.
"},{"location":"concepts/node-concepts/#nodes-are-attached-to-media","title":"Nodes are attached to Media","text":"In an Islandora repository, the files in the repository are uploaded as Media, which are linked to the node providing the descriptive metadata. Media belonging to a specific node can be found in the Islandora-provided \"Media\" tab on that node. For more, see the [Media in Islandora] section.
"},{"location":"concepts/node-media/","title":"Media in Islandora","text":"In Islandora, Media are created to hold the various files that pertain to the descriptive metadata provided by a Node. Islandora provides some fields and features to make this possible.
"},{"location":"concepts/node-media/#media-are-wrappers-for-files","title":"Media are wrappers for files","text":"Drupal Media are said to be wrappers around files, that allow that file to have fields attached. These fields may contain technical metadata about that file, such as its mimetype or size. This is how Islandora uses Media (though our approach to [technical metadata] differs).
"},{"location":"concepts/node-media/#media-may-store-files-in-different-locations","title":"Media may store files in different locations.","text":"Configuration on Media types determines where uploaded files will be stored - for example, the Drupal public or private filesystem, or through a tool called Flysystem to another data store such as Fedora. Islandora does not dictate where you put your files. Islandora Starter Site sets all media types to store their files in Fedora. This can be overridden when creating media programmatically, such as for derivatives.
"},{"location":"concepts/node-media/#media-belong-to-nodes","title":"Media belong to nodes","text":"While in Drupal, media are said to be \"reusable files\" that can be referred to by any number of content items, in Islandora there is a special relationship, the \"Media Of\" (media_of) field, that links media to a single node that describes that file and how it fits into the repository. The field \"Media Of\" must be on all Islandora media types for media of those types to behave as Islandora media.
Then, the media \"of\" an Islandora node (if any) will appear in that node's \"Media\" tab. From this tab, there are also links to add new Media, individually or in batch. When media are created this way, their \"Media of\" field is automatically populated with the current node.
"},{"location":"concepts/node-media/#media-have-different-uses","title":"Media have different uses","text":"In a repository, a node may have several media that belong to it, which represent the originally uploaded file, a smaller service file, a thumbnail, and perhaps a file transformed into a specific format for preservation. All of these media can be distinguished using the Islandora-provided field, \"Media Use\" (field_media_use). This is a taxonomy reference field that points to the vocabulary, Islandora Media Use.
The Islandora module provides, through a migration, the following values:
Media Use term External URI Extracted Text http://pcdm.org/use#ExtractedText Intermediate File http://pcdm.org/use#IntermediateFile Original File http://pcdm.org/use#OriginalFile Preservation Master File http://pcdm.org/use#PreservationMasterFile Service File http://pcdm.org/use#ServiceFile Thumbnail Image http://pcdm.org/use#ThumbnailImage Transcript http://pcdm.org/use#TranscriptThis migration is performed by all installation methods, so these values should be available \"out of the box\". These values were provided by the PCDM data model (see RDF in Islandora), and not all values are associated with behaviours you might expect - for instance, Islandora is not configured out-of-the box to display a transcript if you add a Media (file) and tag it as \"Transcript\". See Audio and Video for setting up transcripts.
Standard vs Multi-file media model
This describes the standard file-media relationship in Islandora. There is an alternative method of arranging files and their derivatives which we call the \"Multi-file media\" method.
"},{"location":"concepts/rdf/","title":"RDF in an Islandora Repository","text":"Islandora was build on the idea that a repository can be represented in Drupal in a way that can be mapped to RDF and present the repository as Linked Data.
"},{"location":"concepts/rdf/#portland-common-data-model","title":"Portland Common Data Model","text":"Islandora uses the Portland Common Data Model (PCDM) to arrange elements of the repository. Nodes correspond to pcdm:Objects, and Media to pcdm:Files. The \"Member of\" field on nodes allows us to create pcdm:memberOf relationships (the opposite of pcdm:hasMember), and the \"Media of\" field represents pcdm:fileOf relationships (the opposite of pcdm:hasFile).
PCDM, plus additional metadata mappings, is the organizing model which allows us to create an RDF version of the repository in Fedora and/or in an RDF triplestore like Blazegraph.
"},{"location":"concepts/rdf/#mapping-to-rdf-rdf-module","title":"Mapping to RDF (RDF Module)","text":"In Islandora, the RDF module provides a way to map metadata from fields into RDF. This provides a mechanism of using configuration entities (YAML files, as there is no robust UI) to map fields on nodes, media, and taxonomy terms to RDF predicates.
Namepspaces in RDF must be registered before they can be used. The Islandora module registers a number of namespaces and more can be added using hook_rdf_namespaces(). See RDF Mappings for more details.
Nodes, Media, and taxonomy terms can have their RDF (per their mappings) exposed to the world as RDF formatted in the JSON-LD syntax thanks to the Islandora-built JSON-LD Drupal module. The JSON-LD module converts the RDF metadata, with some alterations, to a JSON-LD format that can be consumed by RDF consumers such as Fedora and Blazegraph.
"},{"location":"concepts/rdf/#syncing-to-fedora-and-blazegraph","title":"Syncing to Fedora and Blazegraph","text":"Islandora provides the pathways for objects and media in the repository to be synced to Fedora and Blazegraph.
Objects are sent to Fedora and Blazegraph through an \"Indexing\" Drupal Action, which, after being put on a queue, is read by an indexer which pushes the JSON-LD information to the appropriate target.
Files can be stored in Fedora directly, using the Flysystem module. Whether or not a file is in Fedora, information about that file can be synced (from Drupal Media) into Fedora.
Neither Fedora nor Blazegraph are read as part of the standard Islandora configuration.
"},{"location":"concepts/starter-site/","title":"Islandora Starter Site","text":"The Islandora Starter Site is an out-of-the-box deployment of Islandora. It is a complete exported Drupal site, that makes use of the Islandora modules and configures them in a way that is illustrative and useful.
The Islandora Starter Site contains no code, only references to other modules and lots of Drupal configuration. A very motivated person could re-create the Starter Site just by installing and configuring modules. This means there's nothing tying you to using the Starter Site. There's also nothing tying you to doing things in a particular way. Also, it means that you won't be getting any \"updates\" - there's no code to update.
To experience the full Islandora Starter Site, it requires access to external services such as Solr, Fedora, Alpaca, and Matomo. It is therefore suggested to deploy the Starter Site using one of our two deployment platforms: ISLE-DC (using the make starter or make starter_dev commands), or the Islandora Playbook (using the starter (default) or starter_dev option in the Vagrantfile).
If you are reading this document then you are interested in contributing to Islandora. You do not need to be a programmer to speak up! By joining the Islandora community, you agree to abide by the Islandora Code of Conduct.
"},{"location":"contributing/CONTRIBUTING/#join-the-community","title":"Join the community!","text":"Onboarding Form: Submit your information through the Onboarding Form, and we can invite you to all the listed channels below.
Slack: Use this invite link to join our Slack workspace. If you're not sure what channels you're looking for, get started by asking your questions in the #general channel!
Google Group: Subscribe to the mailing list by joining the Google Group.
Github: Although you can participate without a Github account, having one ensures you can be made a member of the Islandora General Members team to better enable your access to the Islandora project. Additionally, it will allow you to be tagged on relevant issues or pull requests that are of interest to you! To become a part of the Islandora General Members Github team, email community@islandora.ca or use the Onboarding form.
Attend our weekly Zoom meetings on Wednesdays at 1:00 PM EST! You can access these meetings through our Agendas or our Community Calendar. We discuss the past week's Github activity in Issues and Pull Requests, and any other topics that are raised. Anybody is welcome to join the call, and add items to the agenda.
"},{"location":"contributing/CONTRIBUTING/#github-issues","title":"Github Issues","text":"Anyone with a Github account can make issues. Please check the applicable places first to see if there's already a ticket.
Issues should be added in the individual module or component's repository where applicable. For issues that are broader in scope than one module, or about the official Islandora documentation, open an issue in the Documentation repository. This is also the repository for hundreds of \"legacy\" (but still valid) tickets from before we opened issue queues on individual repositories.
Issue templates are provided for the following:
Issues are reviewed during the Weekly Open Tech Call every Wednesday at 1:00 p.m. Eastern.
"},{"location":"contributing/CONTRIBUTING/#github-pull-requests","title":"Github Pull Requests","text":"If you have code to address an issue, or can offer documentation, please create a pull request (PR) in the appropriate repository. Before we can merge your pull request, you must have a signed CLA or CCLA (see below). Code PRs must conform to Drupal Coding Standards, and Documentation PRs must conform to our Documentation Style Guide. There are no guides specific to READMEs or other documentation (help text, etc) that lives in Drupal, but please try to write clearly and follow existing practices. See more in our Contributing Workflow.
"},{"location":"contributing/CONTRIBUTING/#new-modules","title":"New Modules","text":"If you have code that doesn't fit within the scope of Islandora's existing repositories, but that the Islandora community is in a better position to maintain than you are, please consider using the LSAP Process to contribute your code to the ownership of the Islandora Foundation.
"},{"location":"contributing/CONTRIBUTING/#contributor-license-agreements","title":"Contributor License Agreements","text":"Before you set out to contribute code you will need to have completed a Contributor License Agreement or be covered by a Corporate Contributor License Agreement. This license is for your protection as a contributor as well as the protection of the Foundation and its users; it does not change your rights to use your own contributions for any other purpose.
"},{"location":"contributing/committers/","title":"Islandora Committers","text":"Islandora is open source and released under open-source (usually MIT and GPLv2) licenses. The software, associated documentation, and community is developed collectively by a community of contributors and committers. All interested community members are encouraged to contribute to the project. Contributors who demonstrate sustained engagement with the project through quality participation in meetings, mailing lists, documentation and code updates can be self-nominated, or nominated by existing committers, to become a committer. Committers need not be limited to software developers! Contributors with skills in documentation and testing, or who are very active in the community for events and support, can also be committers.
Committers are supported in their specific work by having permissions to do things like merge code as well as manage Github, Slack, and the Website. The current list of committers is managed as a group in Github, and can be viewed in the Github repository by existing members. A separate list of committers is publicly available at the bottom of this page. If a committer does not make a contribution in a year (without known mitigating circumstances) they are usually moved out of the committers GitHub group and listed as an Emeritus committer. There are subgroups with specific enhanced access depending on their skills and levels of commitment to the codebase. These are more fully explained in this spreadsheet of keys and permissions. If you have questions about committers, you can send them to the Chair of the TAG committee, or to community@islandora.ca
"},{"location":"contributing/committers/#rights","title":"Rights","text":"Committers share the following rights:
The following is an alphabetized list of the current Islandora committers:
Name Organization Github username Bryan Brown Florida State University bryjbrown Joe Corall Lehigh University joecorall Jordan Dukart discoverygarden jordandukart Willow Gillingham Born-Digital wgilling Jonathan Hunt Catalyst.Net kayakr Mark Jordan Simon Fraser University mjordan Danny Lamb Born-Digital dannylamb Natkeeran Ledchumykanthan University of Toronto Scarborough natkeeran Rosie Le Faive University of Prince Edward Island rosiel Gavin Morris Born-Digital g7morris Annie Oelschlager Northern Illinois University aOelschlager Alexander O'Neill Born-Digital alxp Don Richards Johns Hopkins University DonRichards Bethany Seeger Johns Hopkins University bseeger Seth Shaw University of Nevada, Las Vegas seth-shaw-unlv Alan Stanley Agile Humanities ajstanley Yamil Suarez Berklee College of Music ysuarez Adam Vessey discoverygarden adam-vessey Jared Whiklo University of Manitoba whikloj"},{"location":"contributing/committers/#emeritus-committers","title":"Emeritus Committers","text":"The following is an alphabetized list of the prior Islandora committers:
Name Organization Daniel Aitken discoverygarden Melissa Anez LYRASIS Aaron Coburn Amherst College Jonathan Green LYRASIS Debbie Flitner Arizona State University Diego Pino METRO Nick Ruest York University Eli Zoller Arizona State University"},{"location":"contributing/committers/#how-are-you-evaluated-as-a-potential-committer","title":"How are you evaluated as a potential committer?","text":"When you are nominated, the other committers review these categories before recommending you as a committer. Note that appointment to committers/TAG is governed by the TAG Terms of reference, and appointment to TAG comes with additional rights and responsibilities. If you become a committer and a member of committers/code, you are eligible to serve in TAG.
"},{"location":"contributing/committers/#you-are-committed-to-the-islandora-community","title":"You are committed to the Islandora Community","text":"You\u2019ve been making strong and consistent community contributions, you can stick it out through tough issues and have demonstrated a willingness to lend a hand.
"},{"location":"contributing/committers/#you-have-skills-and-abilities-suited-to-the-role","title":"You have skills and abilities suited to the role","text":"You demonstrate that you understand the project and community and have skills suited to being a committer. Perhaps you have participated in multiple event planning sessions, you write a lot of documentation, or you chair an effective interest group. If you are a software developer, you\u2019ve written good code. If your patches are easy to apply, this could be you!
"},{"location":"contributing/committers/#you-are-a-great-colleague","title":"You are a great colleague","text":"You respond to and deliver criticism well, and do what you say you will do. You participate actively in decision-making processes, keeping the larger good of the Islandora codebase and community in your purview. You have a helpful attitude and respect everybody\u2019s ideas.
"},{"location":"contributing/committers/#you-mentor-the-less-experienced","title":"You mentor the less experienced","text":"You\u2019ve demonstrated a willingness to reach out to less experienced community members, and link them to resources they need (maybe you made some of these resources - that\u2019s great!)
"},{"location":"contributing/committers/#committers-acting-as-maintainers","title":"Committers acting as Maintainers","text":"If you are maintaining a specific Islandora repository, you commit to the following responsibilities for as long as you act as maintainer:
A great maintainer also communicates important changes within the component to the community.
As maintainer, you are empowered to:
If these responsibilities are not being performed, and/or you choose to step down as maintainer, the TAG will attempt to find another maintainer. If no maintainer can be found, and the TAG/Committers do not elect to jointly maintain the component, it may be deprecated.
"},{"location":"contributing/committers/#process-for-inducting-new-committers","title":"Process for inducting new Committers","text":"Contributors can be self-nominated or be nominated by others. These nominations can go to any committer willing to shepherd the process, or be sent to community@islandora.ca to be forwarded by IF staff. Whoever initiates the process is the sponsor and responsible for either completing the following steps, or delegating them to IF staff.
Upon receiving a nomination:
Checklist for a new committer (can be completed by sponsor or designate):
For additional information about committers, visit this Islandora community wiki page.
"},{"location":"contributing/contributing-workflow/","title":"Contributing Workflow","text":""},{"location":"contributing/contributing-workflow/#terms","title":"Terms","text":"Choosing how we work together is an important privilege the Islandora community offers to its contributors. Our workflow is built around engagement and consensus to encourage high quality code that is expected to meet defined standards. All contributors are expected to follow the defined workflow. Any contributor can propose changes to the workflow at any time by initiating a conversation on the mailing list. Any proposed changes must be confirmed by the committers by way of an Apache rules vote in order to take effect.
"},{"location":"contributing/contributing-workflow/#social-dynamics","title":"Social Dynamics","text":"We operate under the Islandora Community Code of Conduct. Some additional general observations to keep in mind when interacting with others in this workflow:
The Islandora community uses GitHub issues to track bug reports, requests for improvements, requests for new features, and use cases. Issues are welcome from anyone who works with Islandora.
The Islandora issue queue is maintained in the Islandora Documentation repository on GitHub. Issues posted to the queue are reviewed weekly on the Islandora Tech Call. Members of the Islandora community can then respond to posted issues by replying with comments, taking on the assignment to do the work described in the issue, or making pull requests relating to the issue.
Video version available
The material in this tutorial is presented in our video, Create a GitHub Issue.
"},{"location":"contributing/create-issues/#before-you-start","title":"Before you start","text":"Fill in the information for your issue:
Submit documentation formatted in Markdown format.
#)# and ##) to the content sections.Use the \"bold/emphasis\" style in Markdown by enclosing text in double asterisks or underscores, **bold text** or __bold text__, for UI elements that users will interact with. For example, a button label for a button that must be pressed should be made bold in Markdown.
*italic text* or _italic text_, for UI elements that have a label or title if you need to reference them in the documentation. For example, a title of a screen or page that will visit should be made italic in Markdown.>> and **bold text** to indicate clicking through nested menu items, and also include the direct path. Example: **Administration** >> **Structure** >> **Views** (/admin/structure/views)\n- instead of * for bulleted lists. Indent four (4) spaces for nested lists (Github renders nesting in markdown with 2 spaces, but mkdocs needs 4). Example: - I am a list item\n - And I am a sub-item.\nExample:
!!! note \"Helpful Tip\"\n I am a helpful tip!\nResult:
Helpful Tip
I am a helpful tip!
islandora type within the Admonition syntax to call attention to areas where Islandora configuration differs from standard Drupal configuration:Example:
!!! islandora \"Lobster trap\"\n This setting is specific to Islandora and is not standard to Drupal.\nResult:
Lobster trap
This setting is specific to Islandora and is not standard to Drupal.
"},{"location":"contributing/docs-style-guide/#donts","title":"Don'ts","text":"These documentation pages are written and maintained by volunteers from the Islandora community, and new contributions are always welcomed. The documentation that you see here is formatted in the easy-to-use Markdown markup language, which can add formatting such as links, headings, and bolded text. Then the documentation HTML site is built using mkdocs from the Markdown files hosted in the Islandora documentation GitHub repo. This documentation is rebuilt automatically upon changes being committed to GitHub, so you will see changes reflected here within a few minutes of making an edit.
The Islandora Documentation Interest Group holds regular meetings on the second Wednesday of each month between 2:00 p.m. - 3:00 p.m. EST to discuss how to improve and expand Islandora's documentation. If you are interested in becoming involved, all are welcome to join! You can view DIG meetings on the Islandora community calendar and add them to your own calendar by clicking the event and selecting \"copy to my calendar\". If you just want to make a contribution to the existing documentation, that process is outlined here.
"},{"location":"contributing/editing-docs/#before-you-start","title":"Before you start","text":"To edit documentation in Islandora, you must:
\u2192 If you are not a member of the Islandora GitHub organization, GitHub will create a new fork of the documentation repository in your GitHub account. From there you can make a pull request to have your changes sent back to the main Islandora documentation repo to be reviewed and accepted.
\u2192 If you are a member of the Islandora GitHub organization, you will be able to create a new temporary branch within the main Islandora documentation repo, and then create a pull request by selecting Create a new branch for this commit and start a pull request. Give your branch a brief name starting with your username, with no spaces, indicating what it is for. Then click Commit changes.
@ symbol. If you don't have anyone specific in mind, you may tag @Islandora/committers to alert all Islandora 8 Committers that there is a new pull request for their review.When you have completed the template, submit your changes for review by clicking Create pull request.
Once your pull request has been made, it will go into the pull request queue and anyone tagged in the request will receive a notification. Any member of the Islandora GitHub organization can review and merge your pull request, committing your changes to the documentation, with a few exceptions:
Once your changes are committed, these documentation pages will be rebuilt to reflect your changes within a few minutes.
For very minor changes
For members of the Islandora GitHub organization (i.e., those with write access to the documentation repo): If the change you made was very small, such as a formatting or spelling correction that does not change the semantics of the document, you may push your changes into the documentation immediately by selecting commit directly to the main branch and clicking Commit changes.
Warning
These steps are for more advanced users that are familiar with using the Git source control software, which is different from just using the GitHub website with a web browser.
#  Repository Name\n\n[](link)\n[](link)\n[](./CONTRIBUTING.md)\n[](./LICENSE)\n[](link)\n\n## Introduction\n\nA brief introduction and summary of the module.\n\n## Requirements\n\nThis module requires the following modules/libraries:\n\n* [Name](Link)\n* [Name](Link)\n* Any\n* Requirements\n\n## Installation\n\nInstallations instructions.\n\n## Configuration\n\nDescribe path to configuration.\n\nInclude a screenshot of configuration page. When using your choice of screenshot software, resize your browser\nfirst to avoid wide screenshots. Here are a few browser extension examples to take screenshots.\n\n - [Fireshots](https://chrome.google.com/webstore/detail/take-webpage-screenshots/mcbpblocgmgfnpjjppndjkmgjaogfceg)\n - [Nimbus](https://chrome.google.com/webstore/detail/nimbus-screenshot-screen/bpconcjcammlapcogcnnelfmaeghhagj)\n\nTo upload the image drag the image into the comment section of an existing Pull Request.\n\nThis will generate the image URL link for you\n \n\nVideo example on [How to attach an Image in README.md file with Github](https://youtu.be/wVHJtL-y7P0)\n\n## Other Sections As Needed\n\nSections specific to this repo, but not found in all repos, should go here.\n\n## Documentation\n\nFurther documentation for this module is available on the [Islandora documentation site](https://islandora.github.io/documentation/).\n\n## Troubleshooting/Issues\n\nHaving problems or solved a problem? Check out the Islandora google groups for a solution.\n\n- [Islandora Group](https://groups.google.com/forum/?hl=en&fromgroups#!forum/islandora)\n- [Islandora Dev Group](https://groups.google.com/forum/?hl=en&fromgroups#!forum/islandora-dev)\n\n## FAQ\n\nQ. Is this normal?\n\nA. Yes. This is normal. Why ...\n\n## Maintainers/Sponsors\n\nCurrent maintainers:\n\n- [Maintainer Name](https://github.com/maintainer_github)\n- [Another Maintainer](https://github.com/maintainer_github)\n\nThis project has been sponsored by:\n\n- Some really awesome sponsor\n\n## Development\n\nIf you would like to contribute, please get involved by attending our weekly [Tech Call](https://github.com/Islandora/documentation/wiki). We love to hear from you!\n\nIf you would like to contribute code to the project, you need to be covered by an Islandora Foundation [Contributor License Agreement](https://drive.google.com/file/d/1k6eCM5EV-w4I4ErkiGj4NJwvLnXkejyk/view?usp=sharing) or [Corporate Contributor License Agreement](https://drive.google.com/file/d/1-SQYuHWRxvltQYgkFWpYv7nGbvJp1u8h/view?usp=sharing). Please see the [Contributors](https://github.com/Islandora/islandora-community/wiki/Onboarding-Checklist) pages on Islandora.ca for more information.\n\nWe recommend using the [islandora-playbook](https://github.com/Islandora-Devops/islandora-playbook) to get started. If you want to pull down the submodules for development, don't forget to run git submodule update --init --recursive after cloning.\n\nAlso include any Travis gotcha's here.\n\n## License\n\n[Name](link). GPLv2 for Drupal modules. MIT for other modules.\nIslandora is an ecosystem of repositories and follows a Semantic Versioning Approach. This allows the community to remain aligned with Drupal's approach and support more a more modular approach and more frequent releases, as well as better upgrade paths for those using components of the system. In semantic versioning, a version has three elements 'MAJOR.MINOR.PATCH'. This looks something like 2.1.1, or you may see major versions labelled as 2.x.x. To guide repository maintainers, we recommend you increment the:
Learn more about Islandora's Versioning Policy
"},{"location":"contributing/releasing-islandora/#how-to-release-java-code","title":"How to Release Java Code","text":"You will need Java 8 on your system to release java code. The rest of the work is handled handled by Gradle, which is included in the Git repositories. If you cannot get Java 8, you can still release Syn using Docker and the openjdk:8-jdk image. For Alpaca, because of our use of keys, Java 8 is required.
To release Syn
-SNAPSHOT from projectVersion in build.gradle$ ./gradlew build shadowJar$ docker run --rm -v /path/to/Syn:/opt/Syn openjdk:8-jdk bash -lc 'cd /opt/Syn && ./gradlew build shadowJar'v to the release tag (i.e. use vX.X.X instead of just X.X.X)/path/to/Syn/build/libs. You want both islandora-syn-X.X.X.jar and islandora-syn-X.X.X-all.jar.projectVersion in build.gradle and add -SNAPSHOT to the end again.To make sure the release goes smoothly, you should ensure that:
~/.gradle/gradle.properties) exists and includes the following: ossrhUsername = jiraname\nossrhPassword = jirapass\nsigning.keyId = pubkeyid\nsigning.password = keypassphrase\nsigning.secretKeyRingFile = /your/home/.gnupg/secring.gpg\ngit is configured (locally or globally) to cache github credentials for https or use sshNote: As of GPG 2.1 secring.gpg has been removed so you need to export secret keys to create the secret key ring.
gpg --export-secret-keys -o secring.gpg\nThe following assumes you are using ssh (e.g. git@github.com for authentication).
It will also work for https if you properly cache your github credentials. The credentials must be cached and valid because Gradle will not prompt you for them.
"},{"location":"contributing/releasing-islandora/#release-artifacts-to-sonatype-and-github","title":"Release artifacts to Sonatype and Github","text":"You need to merge the latest code into the master branch and use Gradle to release. You can do this by running
git clone git@github.com:Islandora-CLAW/Alpaca.gitcd Alpacagit checkout mastergit pull origin dev./gradlew releaseThe script will start but then will ask you to confirm the version to release as.
??> This release version: [0.3.1] (WAITING FOR INPUT BELOW)\n<-------------> 0% EXECUTING [13s]\n> :release\n> :alpaca-release:confirmReleaseVersion\n0.4.0) or nothing to use the suggested version (0.3.1 in this example) and hit ENTER. The gradle release task will take care of dropping -SNAPSHOT from the version, uploading artifacts to Maven central for staging, tagging and pushing a release to github, and bumping master of the Alpaca repository up by a point release for the next development iteration.
The release process will also generate documentation at ./docs/<new-version>. This will need to be added to the git repository and pushed to the master branch:
git add docs/<new-version>git commit -m \"Add documentation for <new-version>\"git push origin\u26a0\ufe0f The following steps, once completed are final. They cannot be undone, revoked or altered. Only proceed if you've completed all the above steps and are absolutely certain the release is ready for publication.
Release the jsonld module by creating a new release for it in Github.
Release the openseadragon module by creating a new release for it in Github.
Release the islandora_mirador module by creating a new release for it in Github.
Release the migrate_islandora_csv module by creating a new release for it in Github.
Release chullo by creating a new release for it in Github.
Crayfish commons depends on the chullo library, and must have its dependencies updated before release.
islandora/chullo from dev-dev to the release you just made in the previous step.composer update -Wcomposer.json and composer.lock files to Github.crayfish-commons by creating a new release in Githubislandora/chullo back to dev-devcomposer update -W againcomposer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Crayfish depends on the crayfish-commons library, and must have its dependencies updated before release.
islandora/crayfish-commons in each composer.json for each microservice except for Houdini. Houdini needs the dev-symfony-flex branch of Crayfish Commons, so just leave that dependency alone for now.composer update -W on each microservice. I did this with a little bash-fu: for D in */; do (cd $D; composer update -W) donecomposer.json and composer.lock files to Github.islandora/crayfish-commons back to dev-dev except for Houdini.composer update -W on each microservice again. for D in */; do (cd $D; composer update -W) done makes this easy.composer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Release controlled_access_terms by slicing a new release for it in Github.
"},{"location":"contributing/releasing-islandora/#release-islandora","title":"Release Islandora","text":"The islandora module depends on the crayfish-commons library, and must have its dependencies updated before release.
islandora/crayfish-commons in composer.json.composer update -Wcomposer.json and composer.lock files to Github.islandora/crayfish-commons back to dev-devcomposer update -W again.composer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Note to module maintainers specifying composer requirements: please familiarize yourself with the Next Significant Release Modifiers so that module dependencies are as permissive as possible. Most often dependency versions should be tagged using the caret (^) so that bug-fixes and minor releases can be pulled in without updating their module\u2019s composer file.
"},{"location":"contributing/releasing-islandora/#undoing-a-release","title":"Undoing a Release","text":"Dependencies mean that if you are going to release all of the Islandora ecosystem, order is very important. At any point, releases can be deleted, updated, and redone in Github. You can reach out to the community if you have questions. Note that if you want to 'redo' a release, you can follow these steps:
git tag -d TAG_NAME; git push --delete origin TAG_NAMEYou cannot follow these steps when publishing to Sonatype with Alpaca, but this should rarely be an issue. Instead, increment the version number and tag a new release.
Learn more about Islandora's Versioning Policy
"},{"location":"contributing/sandbox/","title":"Updating the Sandbox","text":"Note
This page is about the online sandbox, https://sandbox.islandora.ca. If you are looking to set up your own site, see the Installation section.
The sandbox at https://sandbox.islandora.ca is built, configured, and deployed, by the https://github.com/Islandora-Devops/sandbox repository. This build process happens nightly from the Sandbox's latest release.
"},{"location":"contributing/sandbox/#maintaining-the-front-end-site","title":"Maintaining the front-end site","text":"The Sandbox makes use of the following components, which are version-locked in the drupal/Dockerfile file:
In order to update them, you need to update the desired XXX_COMMIT hash; compute and add a sha-256 checksum; and then cut a release of the Sandbox repository. It will be deployed to the production site overnight. See full instructions at the Sandbox README.
Note that some of these components will need to be updated together, for example a new column in the Demo Objects may require a new field in the Starter Site.
"},{"location":"contributing/sandbox/#maintaining-the-back-end-containers","title":"Maintaining the back-end containers","text":"The containers used in the Sandbox are specified by the ISLANDORA_TAG value in the .env file.
A functioning Islandora Stack is made up of dozens of components working in synchronization with each other to store information in your repository, manage that information, and disseminate it intelligently to users. Whether running an installation using the provided Ansible playbook or installing the stack manually, it may be helpful to have a brief overview of all the components we're going to need, in the order we're going to install them, as well as a brief introduction to each component's installation and configuration process.
This list includes four different kinds of components:
Combined together, Apache, PHP, and MySQL/PostgreSQL comprise a LAMP or LAPP server used to provide end-user-facing components - namely, the website.
Apache is the webserver that will serve up webpages to the public. It will also manage some internal functionality provided by Crayfish, and will expose Cantaloupe to the public. We\u2019ll be making changes to the VirtualHost entry, enabling some modules, and modifying the ports configuration. The VirtualHost entry will eventually be modified when we need to expose other services like Cantaloupe to the public.
PHP is the runtime interpreter for all the code Drupal and Crayfish need to be processed. By default, installing PHP 7.2 will give us a command-line interpreter, as well as an interpreter for Apache. We\u2019re going to install several PHP modules required and/or useful for the components that make use of PHP.
MySQL and PostgreSQL are database management systems that we will use to store information for many different components like Drupal and Fedora. By default, the Ansible playbook installs MySQL, though this can be switched to PostgreSQL. The manual installation guide recommends and walks through installing and using PostgreSQL.
"},{"location":"installation/component-overview/#the-front-facing-cdm-composer-drush-and-drupal","title":"The Front-Facing CDM - Composer, Drush, and Drupal","text":"Composer will be used to install both Drupal and Drush simultaneously using Islandora's fork of the drupal-project repository.
Composer is an installer and dependency manager for PHP projects. We're going to need it to install components for any PHP code we need to make use of, including Drupal and Crayfish.
Drush and Drupal are installed simultaneously using drupal-project. Drupal will serve up webpages and manage Islandora content, and Drush will help us get some things done from the command-line.
"},{"location":"installation/component-overview/#the-web-application-server-tomcat-and-cantaloupe","title":"The Web Application Server - Tomcat and Cantaloupe","text":"Several applets will be deployed via their .war files into Tomcat, including Fedora and Cantaloupe.
Tomcat serves up webpages and other kinds of content much like Apache, but is specifically designed to deploy Java applications as opposed to running PHP code.
Cantaloupe is an image tileserver that Islandora will connect to and use to serve up extremely large images in a way that doesn't have an adverse effect on the overall system.
"},{"location":"installation/component-overview/#the-back-end-file-management-repository-fedora-syn-and-blazegraph","title":"The Back-End File Management Repository - Fedora, Syn, and Blazegraph","text":"Fedora will be installed in its own section, rather than as part of the Tomcat installation, as the installation process is rather involved and requires some authorization pieces to be set up in order to connect them back to Drupal and other components.
Fedora is the default backend repository that Islandora content will be synchronized with and stored in. A great deal of configuration will be required to get it up and running, including ensuring a database is created and accessible.
Syn is the authorization piece that allows Fedora to connect to other components.
Blazegraph will store representative graph data about the repository that can be queried using SPARQL. Some configuration will also be required to link it back to Fedora, as well as to ensure it is being properly indexed.
"},{"location":"installation/component-overview/#the-search-engine-solr-and-search_api_solr","title":"The Search Engine - Solr and search_api_solr","text":"The installation of Solr itself is rather straightforward, but a configuration will have to be generated and applied from the Drupal side.
Solr will be installed as a standalone application. Nothing of particular importance needs to happen here; the configuration will be applied when search_api_solr is installed.
search_api_solr is the Drupal module that implements the Solr API for Drupal-side searches. After installing and configuring the module, the drush solr-gsc command will be used to generate Solr configs, and these configs will be moved to the Solr configuration location.
Crayfish is a series of microservices that perform different asynchronous tasks kicked off by Islandora. It contains a series of submodules that will be installed via Composer. Later, these configured components will be connected to Alpaca.
"},{"location":"installation/component-overview/#the-broker-connecting-everything-karaf-and-alpaca","title":"The Broker Connecting Everything - Karaf and Alpaca","text":"Karaf\u2019s job is similar to Tomcat, except where Tomcat is a web-accessible endpoint for Java applets, Karaf is simply meant to be a container for system-level applets to communicate via its OSGI. Alpaca is one such applet; it will broker messages between Fedora and Drupal, and between Drupal and various derivative generation applications.
Alpaca contains Karaf services to manage moving information between Islandora, Fedora, and Blazegraph as well as kicking off derivative services in Crayfish. These will be configured to broker between Drupal and Fedora using an ActiveMQ queue.
"},{"location":"installation/component-overview/#finalized-drupal-configurations","title":"Finalized Drupal Configurations","text":"Drupal configuration exists as a series of .yaml files that can either be created in a feature, or exported from Drupal using the content_sync module. It can also be manually entered in via the UI. We're going to place configuration in a few different ways; Some content will be synchronized onto the site, and some core configurations from the main Islandora module will need to be run in order to facilitate ingest.
There's a lot of useful commands available to you from within the isle-dc folder.
make -B docker-compose.yml will rebuild your docker-compose.yml file based on configuration in your .env file. make demo and make local will automatically do this for you, but if you change configuration at a later point in time, you'll need to run this command to see your changes reflected.
make config-export will export your site's configuration to your site's config sync directory (usually config/sync inside your Drupal root folder).
make config-import will import site's configuration from your site's config sync directory (usually config/sync inside your Drupal root folder).
make drupal-database-dump DEST=/your/path/dump.sql will dump your Drupal database and place the file at DEST.
make drupal-database-import SRC=/your/path/dump.sql will import your Drupal database from the file at SRC.
make drupal-public-files-dump DEST=/your/path/public_files.tgz will export your public filesystem and place it as a single zipped tarball at DEST.
make drupal-public-files-import SRC=/your/path/public_files.tgz will import your public filesystem from a single zipped tarball at SRC.
make fcrepo-export DEST=/your/path/fcrepo-export.tgz will export your Fedora repository and place it as a single zipped tarball at DEST
make fcrepo-import SRC=/your/path/fcrepo-export.tgz will import your Fedora repository from a single zipped tarball at SRC
make reindex-fcrepo-metadata will reindex RDF metadata from Drupal into Fedora. Requires the Views Bulk Operations Module.
make reindex-solr will rebuild rebuild Solr search index for your repository.
make reindex-triplestore will reindex RDF metadata from Drupal into Blazegraph. Requires the Views Bulk Operations Module.
ISLE Docker Compose has a single configuration file, .env. Within it, here is all the values you can set and their effects. This list is subjectively sorted in order of importance and usefulness.
Setting the ENVIRONMENT changes how your docker-compose.yml file gets constructed. When you switch from a demo to development to production environment, this is the variable to change.
Available values for this setting are
demo - For demo environments where you do not need access to the codebaselocal - For development environments where you need edit the codebasecustom - For production environments where your codebase gets baked into a custom containerBy default, this is set to demo.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
What domain your Islandora site will be available at. If you are deploying anywhere other than your personal machine, you'll probably want to change this.
By default, this is set to islandora.traefik.me.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are indexing RDF metadata in Fedora and/or a triplestore, please be aware of the implications of changing this once you have installed. By changing your domain, all of the subjects of your triples will be incorrect and you will have to reindex. There are make commands to streamline this process, but if you have lots of data it can potentially be costly in terms of time and effort.
Repository to use for pulling isle-buildkit images.
By default, this is set to islandora. Change to local to use images you have built locally with isle-buildkit, or use your custom docker registry if you have set up one.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
The version of the isle-buildkit images to use. Non isle-buildkit images have their versions specified explicitly in their respective docker-compose files.
By default, this is set to the latest stable release of isle-buildkit.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not you want to use secrets. For demo and development environments, secrets are not required. They are essential if you are running a production environment, though.
Secrets are contained in the secrets folder within your isle-dc installation. Each file represents an individual secret, and its contents are the value you're trying to protect.
Available values for this setting are:
truefalseIf you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Which install profile to use when making an initial installation. Valid values for this setting are the machine names of any Drupal profile.
By default, this is set to standard.
Set this to true if you want to install an existing Drupal site whose configuration was exported with drush config:export
Available values for this setting are - true - false
By default, this is set to false.
If you set this to true, be sure to set DRUPAL_INSTALL_PROFILE to minimal.
Whether or not to include Watchtower as a service. When developing isle-buildkit, this is extremely useful and will auto-deploy new containers as you make changes. You should not use watchtower on production environments, though.
Available values for this setting are - true - false
By default, this is set to true.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not to include etcd as a service.
Available values for this setting are - true - false
By default, this is set to false. If you don't know what etcd is, then leave this be.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not to include the coder IDE as a service. If you're developing on Islandora, this can be pretty useful when developing. You should not deploy this service on production environments.
Available values for this setting are - true - false
By default, this is set to false.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Which database engine to use for Drupal.
Available values are - mariadb - postgres
By default, this value is set to mariadb.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are changing from mariadb to postgres or vice versa, you MUST migrate your data yourself. ISLE will not convert your database from one to the other, and it's generally not advised to change this once you've installed.
"},{"location":"installation/docker-available-configuration/#fcrepo_database_service","title":"FCREPO_DATABASE_SERVICE","text":"Which database engine to use for Fedora.
Available values are - mariadb - postgres
By default, this value is set to mariadb.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are changing from mariadb to postgres or vice versa, you MUST migrate your data yourself. ISLE will not convert your database from one to the other, and it's generally not advised to change this once you've installed.
"},{"location":"installation/docker-available-configuration/#compose_http_timeout","title":"COMPOSE_HTTP_TIMEOUT","text":"Sometimes when bringing up containers, you can encounter timeouts with an error like this:
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.\nIf you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: XXX).\nBy default, this value is set to 480, but if you have slow network conditions and encounter this error, consider bumping it higher.
"},{"location":"installation/docker-available-configuration/#compose_project_name","title":"COMPOSE_PROJECT_NAME","text":"Used for naming services in traefik as well as defining network alias and urls. For example the drupal service will be found at ${COMPOSE_PROJECT_NAME}_drupal_1. This is useful to change if you are running multiple ISLE instances on one machine.
By default, this value is set to isle-dc.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Allows building your custom image when setting ENVIRONMENT=custom
By default, this is set to 1.
You most likely will never have a need to change this.
"},{"location":"installation/docker-available-configuration/#docker_buildkit","title":"DOCKER_BUILDKIT","text":"Instructs Docker to use buildkit when building your custom image.
By default, this is set to 1.
You most likely will never have a need to change this. Trust us, you want buildkit.
"},{"location":"installation/docker-available-configuration/#project_drupal_dockerfile","title":"PROJECT_DRUPAL_DOCKERFILE","text":"The name of the Dockerfile to use after setting ENVIRONMENT=custom.
By default, this is set to Dockerfile.
You most likely will never have a need to change this.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Set this to true to disable JWT authentication for Fedora. This is necessary when performing a Fedora import using their import/export tools.
By default, this is set to false.
Whether or not to use Fedora 6. If you set this to false, you will be given Fedora 5. In general, unless you already have an existing site on Fedora 5, you'll want Fedora 6.
By default, this is set to true.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
After you've finished installing Islandora using ISLE, here's some useful information to keep close at hand about running your site.
"},{"location":"installation/docker-basic-usage/#important-files","title":"Important Files","text":"The make commands that you used to install Islandora will leave you with two very important files.
.env A configuration file that is yours to customize. This file controls how the docker-compose.yml file gets generated to meet your use case.It also allows you to set variables that make their way into the final docker-compose.yml file, such as your site's domain. docker-compose.yml A ready to run docker-compose.yml file based on your .env file. This file is considered disposable. When you change your .env file, you will generate a new one."},{"location":"installation/docker-basic-usage/#available-services","title":"Available Services","text":"Here's a list of all the available services. Note that there are some services over http and not https. Those aren't meant to be exposed to the public, but internally people from your organization will want to access them. In practice, you can restrict access to these services using firewall rules to just those who you trust.
If you want to stop Islandora, you can bring down all the containers with
docker-compose down\nIf you want to start Islandora back up after stopping it, use
docker-compose up -d\nIf you want to stop Islandora and delete all of its content, use
docker-compose down -v\nIf you make changes to configuration in the .env file, you may need to regenerate your docker-compose.yml file so that those changes take effect.
make -B docker-compose.yml\nOnce you have a new docker-compose.yml file, you'll need to restart your containers that have had configuration change. You can do this easily with
docker-compose up -d\nEven if the site is up and running, that command will only restart the containers it needs to.
"},{"location":"installation/docker-basic-usage/#listing-services","title":"Listing services","text":"You can see a list of all the containers that you have running and their statuses by running
docker ps -a\nYou can tail logs using
docker-compose logs service_name
For example, to tail nginx logs for Drupal, use docker-compose logs drupal.
If you don't know what you're looking for exactly, you can turn on the fire hose and look through all logs by dropping the service name and simply using
docker-compose logs
If you are deploying Islandora on a server that is publicly accessible, there are extra precautions you should take for performance and security reasons.
For local development we bind mount our codebase (Drupal) folder so that we can do development work locally, outside of our Docker container. For a production site we don\u2019t want to do this. Instead, we make changes to our development site, then build a Drupal image with our changes that we can use on our production site instead of the default Islandora Drupal image.
"},{"location":"installation/docker-custom/#creating-your-image","title":"Creating your Image","text":"In order to generate that custom Drupal image we need to set up a development environment. You will do this on your development computer, rather than your production server.
Once your development site is set up you will need somewhere to store your custom Drupal image. You should create a private repository in your container registry of choice (Dockerhub, GitHub, GitLab, etc.)
Once you have a place to store it, you can create your custom Drupal image by editing your .env to set CUSTOM_IMAGE_NAMESPACE to your dockerhub username or the URL and username for your container repository, and CUSTOM_IMAGE_NAME to the name of the repository you just created.
Once this is done you can run make build to create the image from a Dockerfile. If you don't have a custom Dockerfile, it will create one from sample.Dockerfile. This will create a new image from the Islandora Drupal image, with your codebase folder copied into the image during the build process.
You should now have a custom Drupal image on your local machine. You will need to docker login to your container registry to push your image, so make sure to do that if you haven't yet (you should only need to do this once). You can then push this image to your container registry by running make push-image.
Now that you have a development site set up with your own codebase folder, you should create a new git repository for your site. This way you can easily spin up a new site based on your modules and configuration, instead of the Islandora Starter Site. This will also allow you to sync changes between your production, staging, and development sites.
You will likely want to include the Isle-dc directory as well as your codebase folder in your git repository. This will allow you to make modifications to your Makefile, Dockerfile, docker-compose.yml, etc. and keep those modifications alongside your Drupal files in the codebase folder. To store these files in your own private repository you should change the git remote repository URL from the Isle-DC URL to the URL for your private repository.
Note that Isle-DC has the codebase folder in its .gitignore, so you will want to remove codebase* from your .gitignore. You don't need the web or vendor directories to be stored in your repo, since they will be added when you do a composer install during the site setup, so you can add those into your .gitignore as codebase/vendor/* and codebase/web/*.
At this point you should have a custom git repository and a custom Drupal image, and both should be accessible by your production server. On your production server you will need to clone your custom copy of Isle-dc, copy the sample.env file, and name it .env.
In that .env you should set the following variables:
ENVIRONMENT=custom\nCOMPOSE_PROJECT_NAME=(same as dev site)\nCUSTOM_IMAGE_NAMESPACE=(same as dev site)\nCUSTOM_IMAGE_NAME=(same as dev site)\nDOMAIN=your-domain.com\nOnce this is set up, run make production to install your drupal site. You can generate new secrets for your production passwords, or copy them from your dev server if you would like them to be the same.
At this point, your site is ready, but you won't be able to access it at your URL until you update the SSL certificates (see TLS section below).
Codebase Folder
Because your codebase folder is in your git repository it will be cloned to your development server, but unlike in your development environment, it is not bound to the Drupal container. This means that any change you make to those files will not be represented in your Drupal site.
You can also use this folder to build your Drupal image on the production server instead of on your development server if you like.
"},{"location":"installation/docker-custom/#secrets","title":"Secrets","text":"Sensitive information, such as passwords, should never be built into a container. It also shouldn't ever be bind-mounted in like we do with our codebase folder. If you use secrets, it's like bind-mounting in a file, except that file is provided from the host machine to the container using an encrypted channel.
Secrets are on by default. They can be toggled in your .env file, but you should never turn them off for production sites.
USE_SECRETS=true\nThe secrets themselves are stored in the secrets/live folder of isle-dc. If you navigate to that directory, you'll see several small files, where each file represents a different password. When you run make production you can choose whether to generate new random secrets or create them yourself.
All public facing sites need to use HTTPS, and it's definitely a stumbling block for the uninitiated. Fortunately, isle-dc is set up to use HTTPS by default. Even when running make demo, your site runs over HTTPS at https://islandora.traefik.me.
The default certificates are stored in the certs folder of isle-dc, and you can simply overwrite them with certificates from your certificate authority. As long as the certificates match the DOMAIN variable in your .env file, that is. Changing the files in the certs folder requires restarting the Traefik container, which you can do by running docker-compose restart traefik.
To use Let's Encrypt to acquire your SSL Certificate, set the following in your .env file and run make -B docker-compose.yml && make up.
USE_ACME=true\nACME_EMAIL=your-email@example.org\nBe sure to replace your-email@example.org with the email address you've associated with Let's Encrypt.
The way this is setup, is it performs an HTTP Challenge to verify you are in control of the domain. So your system will need to be accessible at http://DOMAIN/.
??? warning \"Let's Encrypt Rate Limit\" If you aren't careful, you can hit Let's Encrypt's rate limit, and you'll be locked out for up to a week! If you want to use their staging server instead while testing things out, add the following to your .env file
```\nACME_SERVER=https://acme-staging-v02.api.letsencrypt.org/directory\n```\nYou'll still get security exceptions when it's working, but you should be able to check the certificate from the browser and confirm you are getting it from the staging server.
\nIf you are still getting security exceptions, check what certificate is being used through your browser. Setting TRAEFIK_LOG_LEVEL=DEBUG in your .env file will help out greatly when debugging Traefik. You can tail the logs with docker-compose logs -tf traefik.
The *.traefik.me certificate that covers islandora.traefik.me will need to be redownloaded occasionally, due to the certificate expiring or possibly being revoked. You can download the updated certificates by performing the following commands:
rm certs/cert.pem\nrm certs/privkey.pem\nmake download-default-certs\ndocker-compose restart traefik\ntraefik.me Certificate Note
Please note that sometimes the upstream provider of the traefik.me certificate takes a couple of days to update the certificate after it expires or is accidentally revoked.
"},{"location":"installation/docker-custom/#requesting-certificates-through-acme-and-external-account-binding","title":"Requesting Certificates through ACME and External Account Binding","text":"To request certificates through another Certificate Authority (CA) that supports External Accounting Binding through ACME such as InCommon or ZeroSSL you will need to add the following to your .env file:
USE_ACME=true\nACME_EMAIL=your-email@example.org\nACME_SERVER=\nACME_EAB_KID=\nACME_EAB_HMAC=\nWhere ACME_SERVER is the CA server to use, ACME_EAB_KID is the key identifier from the External CA, and ACME_EAB_HMAC is the HMAC key from the External CA.
Once you have added these commands you will need to run the following commands:
make -B docker-compose.yml\nmake up\nISLE, short for ISLandora Enterprise, is a community initiative to ease the installation and maintenance of Islandora by using Docker. It was originally started by the Islandora Collaboration Group for Islandora Legacy. When transitioning to Islandora, the project came under the purview of the Islandora Foundation. All documentation on this site assumes you're trying to install Islandora. See here if you are looking for ISLE for Islandora Legacy.
"},{"location":"installation/docker-introduction/#why-use-isle","title":"Why use ISLE?","text":"ISLE's architecture using Docker separates out the \"state\" of your site (i.e. all the content, files, and configurations that you've entered) from the underlying software that runs it (e.g. webserver, database, microservices, etc). This allows for easier upgrades, faster development, and more flexible deployment. It is hands down the easiest way to install, run, and maintain and Islandora instance.
"},{"location":"installation/docker-introduction/#where-is-isle","title":"Where is ISLE?","text":"ISLE is a suite of Docker containers that run the various components of Islandora: drupal, fedora, solr, alpaca, crayfish, matomo, etc. The individual containers are created (and automatically pushed to Docker Hub) by ISLE BuildKit.
In order to deploy the containers, however, you need to use a container orchestration tool. The ISLE project provides tools for running and maintaining the containers using docker-compose with ISLE Docker Compose.
"},{"location":"installation/docker-local/","title":"Installing a Development or Demo Server","text":"When developing locally, your Drupal site resides in your isle-dc/codebase folder and is bind-mounted into your Drupal container. This lets you update code using the IDE of your choice on your host machine, and the changes are automatically reflected on the Drupal container.
If you don't already have a Drupal site, you'll be given a basic setup using Drupal 9 and the Islandora Starter Site.
If you do already have a Drupal site, use git to clone it into place as the codebase folder.
cd /path/to/isle-dc\ngit clone https://github.com/your_org/your_repo codebase\nNow you'll need to tell isle-dc to look for it by setting the ENVIRONMENT variable in your .env file. If you don't have one, copy over sample.env and name it .env. Then set
ENVIRONMENT=starter\nYou should also change the COMPOSE_PROJECT_NAME variable. This determines the name of the Docker containers and volumes that are created when you run make starter. If you leave this as the default you will need to be careful not to overwrite the containers with another install of isle-dc later.
COMPOSE_PROJECT_NAME=isle-dc\nIf your site includes exported configuration from drush config:export, then you'll also need to set
INSTALL_EXISTING_CONFIG=true\nDRUPAL_INSTALL_PROFILE=minimal\nOnce you are ready, run
make starter\nto install the Drupal site in your codebase folder and spin up all the other containers with it.
Enjoy your Islandora instance! Check out the basic usage documentation to see all the endpoints that are available and how to do things like start and stop Islandora. Your passwords, including the Drupal admin password, can be found in the secrets/live directory after you run make starter.
To populate your site with some demo content, you can run make demo_content. This will import some sample objects into your Islandora site.
Drupal has a dedicated security team, and regularly produces updates to address security issues that are discovered. You should always keep your Drupal site up to date so that you are protected against known vulnerabilities. Updating Drupal can be intimidating, but we have step-by-step instructions to help you do it safely.
"},{"location":"installation/docker-maintain-drupal/#note-on-production-sites","title":"Note on Production Sites","text":"Please note that maintaining Drupal should be done on your development site. For production sites, you should build an image from the codebase folder on your development machine, and run that image in production. For more information on this, please see the production install instructions.
Once you have a codebase folder, how do you maintain it and keep it up to date? The quick answer is \"the same way you maintain any Drupal site with Composer and Drush\"... with one small caveat. You most likely do not have PHP or Composer on your machine, and even if you do, you want to make sure you're using the exact same version that your Docker container is using. So to ensure all the versions of things line up, we use Docker to execute Drush and Composer from the Drupal container. The general template for running a command in your Drupal container looks like this:
docker compose exec -T drupal with-contenv bash -lc 'YOUR COMMAND'\nYou can also just shell into the Drupal container and run commands as well, just be aware that if you shut down your container for any reason, you'll lose your bash history. If you want to shell in to run commands, drop the -T and -lc 'YOUR COMMAND' bits.
docker compose exec drupal with-contenv bash\nUse Composer to update your site's modules and their dependencies. The working directory of the Drupal container is the Drupal root (a.k.a. codebase), so you don't need to cd into any other directory before running the command. The following command will update all modules and their dependencies that are not pinned to specific versions.
docker compose exec -T drupal with-contenv bash -lc \"su nginx -s /bin/bash -c 'composer update -W'\"\nWhen running Composer commands you may come across the following error
[ErrorException]\nfile_put_contents(/var/www/drupal/web/sites/default/settings.php): failed to open stream: Operation not permitted\nAfter getting the newest code, you'll want to use Drush to update the Drupal database and run any other update hooks that have been introduced. However, YOU SHOULD BACK UP YOUR DATABASE BEFORE GOING ANY FURTHER. You never know when something will go wrong and you don't want to be stuck with an unusable database and no plan B.
make drupal-database-dump DEST=/path/to/dump.sql\nNow you can safely update the Drupal database with Drush via
docker compose exec -T drupal with-contenv bash -lc 'drush updb'\nIf for any reason, something goes wrong, you can Restore the Drupal database at any time by running
make drupal-database-import SRC=/path/to/dump.sql\nYou will regularly be updating your Drupal site as security patches and module updates are released. Less often, you will also need to update the rest of your Islandora installation. ISLE makes this easy. In fact, it was specifically designed to streamline this process.
Since Islandora is not a single piece of software, but instead many pieces of software working together in concert, maintaining all of it is a daunting task. There's nginx, tomcat, karaf, etc... Then there's everything needed for the authentication layer and JWT keys. Plus there's all the microservices. You can see that all this adds up to a significant maintenance burden.
Now imagine if all that time and effort spent on security updates and getting the newest versions could be boiled down to a handful of simple commands. That's exactly what ISLE does!
"},{"location":"installation/docker-maintain-isle/#updating-isle","title":"Updating ISLE","text":"Updating
Before we update ISLE we should pull the most recent version of the isle-dc repository. This ensures that our docker-compose.yml file has any new additions that the new version of ISLE will need.
You may also want to check your .env file against the sample.env found in the repository to see if there are new variables you wish to make use of.
Updating ISLE is easy. When a new release is made available, you update the TAG variable in your .env file to the latest version. Suppose you are on ISLE 1.0.0, and ISLE 1.1.0 has been released. Then we would set
TAG=1.1.0\nBefore we update our docker-compose.yml file we want to stop the running containers with
make down\nWe'll then generate a new docker-compose.yml file that includes the new tag with
make -B docker-compose.yml\nAfter that, we pull the new containers with
make pull\nAnd finally we deploy the updated containers by running
make up\nYou can check that everything is running at the version you've specified with
docker ps -a\nThe version that's running can be confirmed by looking at the IMAGE column in the output.
Major version updates may require a bit more work after the new containers are up and running, if the new images are running higher major versions of software. For example, if there has been an update to PHP or Solr.
"},{"location":"installation/docker-maintain-isle/#solr","title":"Solr","text":"When upgrading to a new major version of Solr, you need to regenerate a new set of config files. You will need to do this on both your development site, and your production site.
First, remove the old solr data. This will remove everything that has been indexed, so we will have to reindex our site in a later step.
docker compose exec -T solr with-contenv bash -lc 'rm -r server/solr/ISLANDORA'\nNext, recreate our solr core with a new set of config files.
make solr-cores\nFinally, we reindex our site.
make reindex-solr\nDrupal updates will be handled by your composer.json file, but you may require a certain version of ISLE in order to have the correct version of PHP. Once you have the correct PHP version installed, you can update Drupal to a new major version in the same way you normally update modules. It may be helpful to reference the Starter Site's composer.json file at https://github.com/Islandora-Devops/islandora-starter-site/blob/main/composer.json
Once you have done this on your development site, the process for your production site is the same as any other Drupal updates.
"},{"location":"installation/docker-maintain-isle/#mariadb","title":"Mariadb","text":"After updating Mariadb, you may need to run mariadb-upgrade inside your mariadb container, to update your system tables. This should be safe to run any time, but it is a good idea to back up your database first, just in case.
You can run this from your isle directory with
docker compose exec mariadb mariadb-upgrade\nUpgrading ISLE from 1.x to the next major version requires the TAG be set to 2.0.5 or higher. Once you create your containers you will need to follow the above instructions for Mariadb, Solr, and Drupal.
ISLE 2.0 bumps PHP up to version 8.1, which allows you to upgrade to Drupal 10.
"},{"location":"installation/docker-prereq/","title":"Prerequisites","text":""},{"location":"installation/docker-prereq/#software-involved-in-isle","title":"Software involved in ISLE","text":"ISLE uses the following software in order to help you set up and run your site. If you are new to any of the following software, that's OK, but learning more about each piece will make it much easier to modify ISLE to suit your particular needs, and maintain your Islandora site.
The following pieces of software are not unique to Islandora, and there are many great resources online to learn more about them. Below is a brief introduction to each piece and how they fit into your Islandora site. There are also many great videos on the Islandora Foundation's YouTube page if you would like to learn more about how they are used within the context of Islandora.
"},{"location":"installation/docker-prereq/#git","title":"Git","text":"Git is a version control system that is used to store the code for Islandora. When you first download ISLE you will be downloading it using Git. Git is also useful for managing your site's Drupal codebase, once you have a site up and running.
More about using Git for Drupal sites can be found in the Drupal documentation.
"},{"location":"installation/docker-prereq/#docker","title":"Docker","text":"Docker allows us to run our Islandora sites in virtualized environments, called containers. Each container runs one part of the Islandora software, and it takes several containers to run a complete Islandora site.
The three main components of Docker are Images, Containers, and Volumes.
"},{"location":"installation/docker-prereq/#images","title":"Images","text":"Images are what our containers are built from. Images are read-only, and allow us to quickly create containers based on them.
The Islandora images are provided by Islandora Buildkit.
"},{"location":"installation/docker-prereq/#containers","title":"Containers","text":"Containers are where the Islandora software actually runs. ISLE gives us tools to easily download the Islandora images and start our containers from them.
"},{"location":"installation/docker-prereq/#volumes","title":"Volumes","text":"Volumes are where our persistent storage lives. Because containers are often rebuilt from images, any data stored in them is easily lost. Volumes allow us to store specific parts of our containers that we don't want to lose. For example, our Drupal database will live in a volume so that it is not lost when a container is shut down.
"},{"location":"installation/docker-prereq/#docker-compose","title":"Docker Compose","text":"Docker Compose is a tool to simplify the process of running multiple Docker containers for a single project. It uses a file called docker-compose.yml to store the settings for your project's containers. ISLE gives us tools to create this docker-compose.yml file, so we can use it to manage the containers for our site.
"},{"location":"installation/docker-prereq/#gnu-make","title":"GNU Make","text":"Make allows us to define commands that simplify installing and maintaining our Islandora site. For a complete list of available commands see the Makefile included with ISLE.
"},{"location":"installation/docker-prereq/#composer","title":"Composer","text":"Composer is a dependency manager for PHP, and is the recommended way to install and update Drupal modules. Composer is installed in the Islandora Drupal container and should be run within the container to manage your Drupal site. More information on using Composer with Drupal is available in the Drupal documentation.
"},{"location":"installation/docker-prereq/#drupal","title":"Drupal","text":"Drupal is a Content Management System used to create websites. During the setup of your Islandora site, a Drupal site is created for you in your Drupal container, by the commands in the Makefile.
"},{"location":"installation/docker-prereq/#drush","title":"Drush","text":"Drush is a command line tool for managing your Drupal site. It comes installed in your Drupal container and allows you to perform Drupal actions from the command line inside your container.
"},{"location":"installation/docker-prereq/#requirements-for-using-isle-with-docker-compose","title":"Requirements for using ISLE with Docker Compose","text":"sudo apt update and sudo apt install make to installWhat are we missing?
Are you, or your computer, new to spinning up development sandboxes? Do you have wisdom about installing make on Windows? We suspect these requirements may be incomplete and your experience would be appreciated at Issue #1640.
"},{"location":"installation/docker-prereq/#installing-docker","title":"Installing Docker","text":"To see if you have Docker installed, type docker --version in a terminal.
If you need to install Docker, we recommend using the application Docker Desktop. It provides a GUI for managing Docker container in Windows and MacOS, along with the Docker engine and suite of command-line tools. Linux users don't get a desktop client, but can install the Engine and command-line tools the instructions here.
Memory, Processors, and Swap Requirements
To run ISLE on Docker Desktop, you must increase the resources allocated to the software. See Docker docs on setting resources on Windows (see note on how to allocate/restrict memory when using WSL 2) or setting resources on Mac.
CPUs (processors): The CPUs allowed to Docker Desktop are still shared with the host machine, so increasing this to the maximum value should allow both the Docker containers and your host machine to run simultaneously.
Memory (RAM): This memory is completely dedicated to Docker while Docker Desktop is running, so do not allocate more than you can spare and still run your host machine. Windows users may not require as much memory for Docker as Mac users. Current suggestions for memory allocated to Docker Desktop are below, but please edit this document if you have new information.
Swap: Swap space is space borrowed from your hard disk drive to serve as makeshift RAM as needed. If you cannot provide as much RAM as you would like, increase this as is reasonable given your free disk space.
"},{"location":"installation/docker-prereq/#installing-isle-docker-compose","title":"Installing ISLE Docker Compose","text":"Use Git to install the ISLE Docker Compose tool:
git clone https://github.com/islandora-devops/isle-dc
Tagged versions are available here.
"},{"location":"installation/docker-troubleshooting/","title":"Troubleshooting","text":""},{"location":"installation/docker-troubleshooting/#make-command-not-found","title":"Make command not found","text":"If you get the error bash: make: command not found then you need to install GNU make. Run sudo apt update and sudo apt install make to install.
If you get an error such as: ERROR: Version in \"./docker-compose.activemq.yml\" is unsupported., then you need to upgrade Docker. Enter the command make clean before re-attempting to make demo.
make demo","text":"If make fails for any reason, enter make clean before attempting to make again. If not, you may see an error such as: ERROR: Top level object in './docker-compose.yml' needs to be an object not '<class 'NoneType'>'.
If you notice some Docker containers drop (exited(0)), and (in Docker Desktop) the isle-dc app icon is yellow instead of green, try increasing the resources allocated to Docker (see note above).
"},{"location":"installation/docker-troubleshooting/#connection-timed-out-mac","title":"Connection timed out (Mac).","text":"If you are using Docker Desktop for Mac, and get timeout errors when spinning up the containers (during docker-compose up -d or during make local) such as this:
ERROR: for isle-dc_mariadb_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=480)\nyou can try quitting Docker completely (make sure there is no whale icon in your top toolbar - may need to select \"Quit\" from the whale icon itself) and then restart Docker.
"},{"location":"installation/docker-troubleshooting/#504-bad-gateway","title":"504 Bad Gateway","text":"If you get a white screen with a 504 Bad Gateway error, this means your containers haven't finished initializing themselves yet. If you've waiting an appropriate amount of time (2-5 minutes), then there is most likely an error in a container's startup script. Use docker ps -a to see which services have Exited status, and then tail their logs with docker-compose logs service_name.
To test out Islandora, there is a Starter-Site based Portainer demo available (Download the .zip file). This demo is loaded with sample content to demonstrate features of Islandora and provide a starting point for exploration. Follow the instructions in the README.md.html file provided in the download.
It is recommended that you have a minimum of 4GB of hard drive space available on your machine before getting started. It will take some time to build the image, do not be surprised if it takes a long time. Thanks, and happy building!
"},{"location":"installation/install-a-demo/#installation-video-tutorial","title":"Installation Video Tutorial","text":"The README.md.html is the most up to date version of the installation documentation, but you can also view this video: How to Install Docker Desktop and Set up Starter Site (video tutorial).
"},{"location":"installation/install-a-demo/#troubleshooting","title":"Troubleshooting","text":"These troubleshooting issues and solutions for Docker Desktop have arised from and been used with Windows 10.
"},{"location":"installation/install-a-demo/#docker-failed-to-start-error-message","title":"\"Docker Failed to Start\" Error Message","text":"For Windows 10, you may receive the \"Docker Failed to Start\" Error message. To resolve this:
If no content is showing up in your local installation when on islandora.io in a browser, you may need to wait longer for the deployment process to complete. 1. Go to \u2018containers\u2019, click the \u2018logs\u2019 for \u2018sandbox-drupal-1\u2019 to see the deployment process. When completed you will see:
#####################\n# Install Completed #\n#####################\nIf Docker Desktop and Virtual Machine (Vmmem in Task Manager) are taking up too much CPU and RAM, you can create a .wslconfig file to limit how much of your computers resources are allocated to these processes: 1. Open Notepad++ and type the following:
[wsl2]\nmemory=4GB\nprocessors=2\nThe Islandora Playbook (Islandora-Devops/islandora-playbook) is a tool for installing the Islandora stack on a single virtual machine. It can be used both as a Vagrant project to create a local development environment, or as an Ansible playbook to provision a local or remote server. It can set up a Drupal based either on Islandora Starter Site, or on the Install Profile Demo.
"},{"location":"installation/playbook/#basic-usage-vagrant","title":"Basic Usage (Vagrant):","text":"Install requirements (below), then:
$ git clone -b dev https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\n$ ISLANDORA_BUILD_BASE=true vagrant up\n$ vagrant package --output islandora_base\n$ vagrant destroy\n$ vagrant up\nWhen used this way, you can trash your working site and get a fresh Islandora relatively quickly, with vagrant destroy (you will be asked to confirm, as this will delete all your changes and your content), and then vagrant up.
Full instructions below.
"},{"location":"installation/playbook/#requirements-vagrant","title":"Requirements (Vagrant)","text":"To create a local VM, download and install the following.
OpenSSL is already pre-installed on MacOS. Git can be installed using XCode's command line tools (see below). Python and Pip can either be installed via the downloaded installer direct from the site or via Homebrew (not shown below). Ansible is best installed using Homebrew (see below).
# Use xcode-select to install command line components, including git\n$ xcode-select --install\n# Install homebrew\n$ /usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n# Use homebrew to install ansible\n$ brew install ansible\nYou will use vagrant up twice, first to create an Islandora base box, and then again to provision that base box into a full Islandora. This method uses Vagrant, VirtualBox, and Ansible.
Warn
Make sure that no required ports are currently in use.
"},{"location":"installation/playbook/#clone-the-playbook","title":"Clone the playbook","text":"$ git clone -b dev https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\nThe Playbook will create a \"base box\" that includes core software (PHP, Apache, etc). Since these do not need to be updated too often, this can be packaged as a Vagrant base box that will be used to quickly provision the Islandora part.
Notes: - If building a CentOS box, you also need to install the vbguest additions with vagrant plugin install vagrant-vbguest. - If this is not your first time spinning up the Islandora Playbook from this directory, you may want to clear cached ansible roles in roles/external\" rm -rf roles/external
First, create the islandora base box by overriding the Vagrant variable ISLANDORA_BUILD_BASE (which defaults to false in the Vagrantfile):
$ ISLANDORA_BUILD_BASE=true vagrant up\nOnce it is complete (without errors), store it as a Vagrant base box. It will appear as islandora_base in the current directory.
$ vagrant package --output islandora_base\nThen, get rid of the existing VM since we've saved what we need:
$ vagrant destroy\n default: Are you sure you want to destroy the 'default' VM? [y/N] y\n==> default: Forcing shutdown of VM...\n==> default: Destroying VM and associated drives...\nThen, to install Islandora (including Drupal and its configuration, Crayfish, Alpaca, etc) on top of the new base box:
vagrant up\nAccess your site at http://localhost:8000.
You can quickly create a new \"out of the box\" Islandora VM by just re-doing the last step (provisioning Islandora) while using your existing base box. In doing this, you will destroy your existing VM, and all of its configuration and data. This will not upgrade PHP, apache, or other back-end services (see below, \"Refreshing your base box.\")
vagrant destroy\nvagrant up\nIf you want to use the Playbook to rebuild the base box (e.g. if this playbook now includes updated versions of PHP or Apache), then you will need to take some extra steps to ensure that Vagrant sees your new base box.
vagrant destroy # Delete your existing VM\nvagrant box remove islandora_base # clear Vagrant's cached version of islandora_base\nISLANDORA_BUILD_BASE=true vagrant up # and proceed to package the box, as above.\nThis is because once you use the base box once, it is stored in the vagrant box list under your home directory. While subsequent builds of ISLANDORA_BUILD_BASE=true vagrant up will build a new box and you can package it, but that file does not update the cached box and consequently you still get the old base box when you try build a VM. See vagrant box documentation for more useful commands for managing base boxes.
The ansible-playbook command using playbook.yml can provision a remote environment. First, you will need to create a new folder in inventory/ with the details of your remote deployment. This includes:
Before beginning, you may want to:
sudo) as this user. See Ansible Docs on Understanding privilege escalation: becomeWe're going to build up this new remote environment configuration from the default provided Vagrant configuration. To start, take the inventory for the vagrant development environment and make a copy of it. Be sure to give it an appropriate name. Here we're using production.
$ git clone https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\n$ cp -r inventory/vagrant inventory/production\nThen you can update the following entries in the following files using your own information. If an entry does not exist in a file, just add it. Ansible will then use the value you provide instead of relying on its defaults.
We're using changeme to represent passwords and assume the server will be available at example.org, but you'll want to provide your own values.
# Drupal\ndrupal_db_password: changeme\ndrupal_account_pass: changeme\n\n# MySQL/Postgres\nislandora_db_root_password: changeme\n\n# Tomcat\nislandora_tomcat_password: changeme\n\n# Syn\nislandora_syn_token: changeme\n\n# Cantaloupe\ncantaloupe_admin_password: changeme\n\n# Fedora\nfcrepo_db_password: changeme\nThis is where we specify that the webserver is listening on the default port 80, instead of the development machine port 8000.
apache_listen_port: 80\nYou will have to add the matomo line.
matomo_site_url: http://example.org\nIf you have a domain name, change the default to your domain name.
- servername: \"myactualddomain.com\"\nIf you have a domain name, change the default to your domain name. And set the Drupal site name to whatever you need it to be.
drupal_domain: \"myactualdomain.com\"\ndrupal_site_name: \"Example Sitename\"\ndrupal_trusted_hosts: drupal_trusted_hosts:\n - ^localhost$\n - \"{{ hostvars[groups['webserver'][0]].ansible_host }}\"\n - '^myactualdomain\\.com$'\nNote the backslash which escapes the period (which would otherwise match any character). Because of this escape character, the string needs to be surrounded by single quotes.
"},{"location":"installation/playbook/#hosts","title":"hosts","text":"You'll need to put particulars for logging into your server in the inventory/production/hosts file . This example is set up to login as the ansible user (to avoid trying to run Composer as root) and uses an SSH key. You'll need to get the details for logging into your remote server from your hosting provider (AWS, Digital Ocean, etc...) or your systems administrator if you're running the server in-house. See this page for more details about what you can put into a host file
default ansible_host=example.org ansible_port=22 ansible_user=ansible ansible_ssh_private_key_file='/home/username/.ssh/id_rsa'\nFirst, you'll want to install the ansible roles that are needed for the version of Islandora you are trying to install. This can be done with
$ ansible-galaxy install -r requirements.yml\nThen, if you are on Ubuntu 22.04, run the following commands:
$ ansible-playbook -i inventory/production playbook.yml -e \"islandora_distro=ubuntu/jammy64\" -e \"islandora_profile=starter\" -e \"islandora_build_base_box=true\"\n$ ansible-playbook -i inventory/production playbook.yml -e \"islandora_distro=ubuntu/jammy64\" -e \"islandora_profile=starter\" -e \"islandora_build_base_box=false\"\nAnsible caches the code used to provision the environment, so if you've already installed once you may not be getting the latest version of things even if you've git pull'd the latest playbook. The code is stored in roles/external, so if you want to clear it out you can remove these before attempting to provision an environment
$ rm -rf roles/external\nWhen provisioning using a local environment, you should be aware of any ports that are already in use by your computer that are also going to be used by Vagrant, as these may clash and cause problems during and after provisioning. These include:
If there are port clashes for any of these, you will need to either find and replace them in the configuration .yml files under inventory/vagrant/group_vars, or provide new values for the different playbooks that support changing the ports (for example, postgresql_databases supports adding a port property which is currently simply unused). You will also need to replace the port forwarding values in Vagrantfile.
Additionally, Ansible attempts to use port 2200 for SSH. If this port is already in use, your local environment cannot be provisioned. To change this, set a new value for ansible_port in inventory/vagrant/hosts.
If you run into any issues installing the environment, do not hesitate to email the mailing list to ask for help. If you think you've stumbled across a bug in the installer, please create an issue in the Islandora issue queue and give it an ansible tag.
The ISLE Site Template is a system for installing Islandora on Docker. As with ISLE-DC, it uses Docker Compose to orchestrate the installation of all the different services (Docker containers) that make up Islandora. Unlike ISLE-DC, in ISLE Site Template you use Docker Compose commands directly, helping you to get familiar with the kinds of commands that will be a key part of running and maintaining Islandora.
"},{"location":"installation/site-template/#usage","title":"Usage","text":"Do not clone the Isle Site Template!
curl and installed either manually or automatically.Instead, follow the instructions in the ISLE Site Template's README.md and README.template.md files.
dev and prod environments, with different services available on each.During installation, you will install a copy of the Islandora Starter Site.
Customizing your site can be persisted to your own repo.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
After all of the above pieces are in place, installed, configured, started, and otherwise prepared, the last thing we need to do is to finally configure the front-end Drupal instance to wire all the installed components together.
"},{"location":"installation/manual/configuring-drupal/#drupal-pre-configuration","title":"Drupal Pre-Configuration","text":""},{"location":"installation/manual/configuring-drupal/#settingsphp","title":"settings.php","text":"Notice
By default, settings.php is read-only for all users. It should be made writable while this pre-configuration is being done, then set back to 444 afterwards.
Some additional settings will need to be established in your default settings.php before Drupal-side configuration can occur.
The below configuration will establish localhost as a trusted host pattern, but on production sites this will need to be expanded to include the actual host patterns used by the site.
/opt/drupal/web/sites/default/settings.php
Before (at around line 789):
'driver' => 'pgsql',\n);\nAfter:
'driver' => 'pgsql',\n);\n\n$settings['trusted_host_patterns'] = [\n 'localhost',\n];\n\n$settings['flysystem'] = [\n 'fedora' => [\n 'driver' => 'fedora',\n 'config' => [\n 'root' => 'http://localhost:8080/fcrepo/rest/',\n ],\n ],\n];\nOnce this is done, refresh the cache to take hold of the new settings.
cd /opt/drupal\ndrush -y cr\nSkip this by using the Islandora Starter Site
The Islandora Starter Site, which was presented as an option in the \"Installing Composer, Drush, and Drupal\" step, installs Islandora and other modules and configures them, allowing you to skip this section. You may want to use this manual method in the case where you want to pick and choose which Islandora modules you use.
"},{"location":"installation/manual/configuring-drupal/#downloading-islandora","title":"Downloading Islandora","text":"The Islandora Drupal module contains the core code to create a repository ecosystem in a Drupal environment. It also includes several submodules; of importance to us is islandora_core_feature, which contains the key configurations that allow you to use Islandora features.
Take note of some of the other comments in the below bash script for an idea of what the other components are expected, and which may be considered optional.
cd /opt/drupal\n# Since islandora_defaults is near the bottom of the dependency chain, requiring\n# it will get most of the modules and libraries we need to deploy a standard\n# Islandora site.\nsudo -u www-data composer require \"drupal/flysystem:^2.0@alpha\"\nsudo -u www-data composer require \"islandora/islandora:^2.4\"\nsudo -u www-data composer require \"islandora/controlled_access_terms:^2\"\nsudo -u www-data composer require \"islandora/openseadragon:^2\"\n# These can be considered important or required depending on your site's\n# requirements; some of them represent dependencies of Islandora submodules.\nsudo -u www-data composer require \"drupal/pdf:1.1\"\nsudo -u www-data composer require \"drupal/rest_oai_pmh:^2.0@beta\"\nsudo -u www-data composer require \"drupal/search_api_solr:^4.2\"\nsudo -u www-data composer require \"drupal/facets:^2\"\nsudo -u www-data composer require \"drupal/content_browser:^1.0@alpha\" ## TODO do we need this?\nsudo -u www-data composer require \"drupal/field_permissions:^1\"\nsudo -u www-data composer require \"drupal/transliterate_filenames:^2.0\"\n# These tend to be good to enable for a development environment, or just for a\n# higher quality of life when managing Islandora. That being said, devel should\n# NEVER be enabled on a production environment, as it intentionally gives the\n# user tools that compromise the security of a site.\nsudo -u www-data composer require drupal/restui:^1.21\nsudo -u www-data composer require drupal/console:~1.0\nsudo -u www-data composer require drupal/devel:^2.0\nsudo -u www-data composer require drupal/admin_toolbar:^2.0\nComponents we've now downloaded using composer require can be enabled simultaneously via drush, which will ensure they are installed in the correct dependent order. Enabling islandora_defaults will also ensure all content types and configurations are set up in Islandora. The installation process for all of these modules will likely take some time.
Notice
This list of modules assumes that all of the above components were downloaded using composer require; if this is not the case, you may need to pare down this list manually. It also includes devel, which again, should not be enabled on production sites.
cd /opt/drupal\ndrush -y en rdf responsive_image devel syslog serialization basic_auth rest restui search_api_solr facets content_browser pdf admin_toolbar controlled_access_terms_defaults islandora_breadcrumbs islandora_iiif islandora_oaipmh\n# After all of this, rebuild the cache.\ndrush -y cr\nTo allow our installation to talk to other services via Syn, we need to establish a Drupal-side JWT configuration using the keys we generated at that time.
Log onto your site as an administrator at /user, then navigate to /admin/config/system/keys/add. Some of the settings here are unimportant, but pay close attention to the Key type, which should match the key we created earlier (an RSA key), and the File location, which should be the ultimate location of the key we created for Syn on the filesystem, /opt/keys/syn_private.key.
Click Save to create the key.
Once this key is created, navigate to /admin/config/system/jwt to select the key you just created from the list. Note that before the key will show up in the Private Key list, you need to select that key's type in the Algorithm section, namely RSASSA-PKCS1-v1_5 using SHA-256 (RS256).
Click Save configuration to establish this as the JWT key configuration.
"},{"location":"installation/manual/configuring-drupal/#configuring-islandora","title":"Configuring Islandora","text":"Navigate to the Islandora core configuration page at /admin/config/islandora/core to set up the core configuration to connect to Gemini. Of note here, the Gemini URL will need to be established to facilitate the connection to Fedora, and the appropriate Bundles with Gemini URI pseudo field types will need to be checked off.
Notice
Any other Drupal content types you wish to synchronize with Fedora should also be checked off here.
"},{"location":"installation/manual/configuring-drupal/#configuring-islandora-iiif","title":"Configuring Islandora IIIF","text":"Navigate to /admin/config/islandora/iiif to ensure that Islandora IIIF is pointing to our Cantaloupe server.
Next, configure OpenSeadragon by navigating to /admin/config/media/openseadragon and ensuring everything is set up properly.
Navigate to /admin/config/media/file-system to set the Default download method to the one we created in our settings.php.
fedoraAdmin Role","text":"In order for data to be pushed back to Fedora, the site administrative user needs the fedoraAdmin role.
cd /opt/drupal\nsudo -u www-data drush -y urol \"fedoraadmin\" islandora\nFinally, to get everything up and running, run the Islandora Core Features and Islandora Defaults migrations.
cd /opt/drupal\nsudo -u www-data drush -y -l localhost --userid=1 mim --group=islandora\nSome views provided by Islandora are not enabled by default.
cd /opt/drupal\ndrush -y views:enable display_media\nNeeds Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Composer provides PHP code that we can use to install it. After downloading and running the installer, we\u2019re going to move the generated executable to a place in $PATH, removing its extension:
curl \"https://getcomposer.org/installer\" > composer-install.php\nchmod +x composer-install.php\nphp composer-install.php\nsudo mv composer.phar /usr/local/bin/composer\nAt this point, you have the option of using the Islandora Starter Site, with its pre-selected modules and configurations which function \"out of the box,\" or build a clean stock Drupal via the Drupal Recommended Project and install Islandora modules as you desire.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#option-1-create-a-project-using-the-islandora-starter-site","title":"Option 1: Create a project using the Islandora Starter Site","text":"Navigate to the folder where you want to put your Islandora project (in our case /var/www), and create the Islandora Starter Site:
cd /var/www\ncomposer create-project islandora/islandora-starter-site\nThis will install all PHP dependencies, including Drush, and scaffold the site.
Drush is not accessible via $PATH, but is available using the command composer exec -- drush
Navigate to the folder where you want to put your Drupal project (in our case /var/www), and create the Drupal Recommended Project:
cd /var/www\ncomposer create-project drupal/recommended-project my-project\nBefore we can proceed with the actual site installation, we\u2019re going to need to make our new Drupal installation the default web-accessible location Apache serves up. This will include an appropriate ports.conf file, and replacing the default enabled site.
Notice
Out of the box, these files will contain support for SSL, which we will not be setting up in this guide (and therefore removing with these overwritten configurations), but which are absolutely indispensable to a production site. This guide does not recommend any particular SSL certificate authority or installation method, but you may find DigitalOcean's tutorial helpful.
/etc/apache2/ports.conf | root:root/644
Listen 80\nRemove everything but the \"Listen 80\" line. You can leave the comments in if you want.
/etc/apache2/sites-enabled/000-default.conf | root:root/777
<VirtualHost *:80>\nServerName SERVER_NAME\n DocumentRoot \"/opt/drupal/web\"\n <Directory \"/opt/drupal/web\">\nOptions Indexes FollowSymLinks MultiViews\n AllowOverride all\n Require all granted\n </Directory>\n# Ensure some logging is in place.\n ErrorLog \"/var/log/apache2/localhost_error.log\"\n CustomLog \"/var/log/apache2/localhost_access.log\" combined\n</VirtualHost>\nSERVER_NAME: localhost - For a development environment hosted on your own machine or a VM, localhost should suffice. Realistically, this should be the domain or IP address the server will be accessed at. Restart the Apache 2 service to apply these changes:
sudo systemctl restart apache2\nPostgreSQL roles are directly tied to users. We\u2019re going to ensure a user is in place, create a role for them in PostgreSQL, and create a database for them that we can use to install Drupal.
# Run psql as the postgres user, the only user currently with any PostgreSQL\n# access.\nsudo -u postgres psql\n# Then, run these commands within psql itself:\ncreate database DRUPAL_DB encoding 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8' TEMPLATE template0;\ncreate user DRUPAL_DB_USER with encrypted password 'DRUPAL_DB_PASSWORD';\ngrant all privileges on database DRUPAL_DB to DRUPAL_DB_USER;\n# Then, quit psql.\n\\q\nDRUPAL_DB: drupal9 - This will be used as the core database that Drupal is installed into - DRUPAL_DB_USER: drupal - Specifically, this is the user that will connect to the PostgreSQL database being created, not the user that will be logging into Drupal - DRUPAL_DB_PASSWORD: drupal - This should be a secure password; it\u2019s recommended to use a password generator to create this such as the one provided by random.org"},{"location":"installation/manual/installing-composer-drush-and-drupal/#install-drupal-using-drush","title":"Install Drupal using Drush","text":"The Drupal installation process can be done through the GUI in a series of form steps, or can be done quickly using Drush's site-install command. It can be invoked with the full list of parameters (such as --db-url and --site-name), but if parameters are missing, they will be asked of you interactively.
Follow the instructions in the README of the Islandora Starter Site. The steps are not reproduced here to remove redundancy. When this installation is done, you'll have a starter site ready-to-go. Once you set up the external services in the next sections, you'll need to configure Drupal to know where they are.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#option-2-site-install-the-basic-drupal-recommended-project","title":"Option 2: Site install the basic Drupal Recommended Project","text":"cd /var/www/drupal\ndrush -y site-install standard --db-url=\"pgsql://DRUPAL_DB_USER:DRUPAL_DB_PASSWORD@127.0.0.1:5432/DRUPAL_DB\" --site-name=\"SITE_NAME\" --account-name=DRUPAL_LOGIN --account-pass=DRUPAL_PASS\nSITE_NAME: Islandora 2.0DRUPAL_LOGIN: islandoraDRUPAL_PASS: islandoraCongratulations, you have a Drupal site! It currently isn\u2019t really configured to do anything, but we\u2019ll get those portions set up in the coming sections.
"},{"location":"installation/manual/installing-crayfish/","title":"Installing Crayfish","text":"Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-crayfish/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Some packages need to be installed before we can proceed with installing Crayfish; these packages are used by the microservices within Crayfish. These include:
apt-get, and a list of available packs can be procured with sudo apt-cache search tesseract-ocrsudo add-apt-repository -y ppa:lyrasis/imagemagick-jp2\nsudo apt-get update\nsudo apt-get -y install imagemagick tesseract-ocr ffmpeg poppler-utils\nNOTICE: If you get the sudo: apt-add-repository: command not found, run sudo apt-get install software-properties-common in order to make the command available.
We\u2019re going to clone Crayfish to /opt, and individually run composer install against each of the microservice subdirectories.
cd /opt\nsudo git clone https://github.com/Islandora/Crayfish.git crayfish\nsudo chown -R www-data:www-data crayfish\nsudo -u www-data composer install -d crayfish/Homarus\nsudo -u www-data composer install -d crayfish/Houdini\nsudo -u www-data composer install -d crayfish/Hypercube\nsudo -u www-data composer install -d crayfish/Milliner\nsudo -u www-data composer install -d crayfish/Recast\nNot much needs to happen here; Crayfish opts for a simple logging approach, with one .log file for each component. We\u2019ll create a folder where each logfile can live.
sudo mkdir /var/log/islandora\nsudo chown www-data:www-data /var/log/islandora\nEach Crayfish component requires one or more .yaml file(s) to ensure everything is wired up correctly.
NOTICE
The following configuration files represent somewhat sensible defaults; you should take consideration of the logging levels in use, as this can vary in desirability from installation to installation. Also note that in all cases, http URLs are being used, as this guide does not deal with setting up https support. In a production installation, this should not be the case. These files also assume a connection to a PostgreSQL database; use a pdo_mysql driver and the appropriate 3306 port if using MySQL.
/opt/crayfish/Homarus/cfg/config.yaml | www-data:www-data/644
---\nhomarus:\nexecutable: ffmpeg\nmime_types:\nvalid:\n- video/mp4\n- video/x-msvideo\n- video/ogg\n- audio/x-wav\n- audio/mpeg\n- audio/aac\n- image/jpeg\n- image/png\ndefault: video/mp4\nmime_to_format:\nvalid:\n- video/mp4_mp4\n- video/x-msvideo_avi\n- video/ogg_ogg\n- audio/x-wav_wav\n- audio/mpeg_mp3\n- audio/aac_m4a\n- image/jpeg_image2pipe\n- image/png_image2pipe\ndefault: mp4\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/homarus.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\nCurrently the Houdini microservice uses a different system (Symfony) than the other microservices, this requires different configuration.
/opt/crayfish/Houdini/config/services.yaml | www-data:www-data/644
# This file is the entry point to configure your own services.\n# Files in the packages/ subdirectory configure your dependencies.\n# Put parameters here that don't need to change on each machine where the app is deployed\n# https://symfony.com/doc/current/best_practices/configuration.html#application-related-configuration\nparameters:\napp.executable: /usr/local/bin/convert\napp.formats.valid:\n- image/jpeg\n- image/png\n- image/tiff\n- image/jp2\napp.formats.default: image/jpeg\nservices:\n# default configuration for services in *this* file\n_defaults:\nautowire: true # Automatically injects dependencies in your services.\nautoconfigure: true # Automatically registers your services as commands, event subscribers, etc.\n# makes classes in src/ available to be used as services\n# this creates a service per class whose id is the fully-qualified class name\nApp\\Islandora\\Houdini\\:\nresource: '../src/*'\nexclude: '../src/{DependencyInjection,Entity,Migrations,Tests,Kernel.php}'\n# controllers are imported separately to make sure services can be injected\n# as action arguments even if you don't extend any base controller class\nApp\\Islandora\\Houdini\\Controller\\HoudiniController:\npublic: false\nbind:\n$formats: '%app.formats.valid%'\n$default_format: '%app.formats.default%'\n$executable: '%app.executable%'\ntags: ['controller.service_arguments']\n# add more service definitions when explicit configuration is needed\n# please note that last definitions always *replace* previous ones\n/opt/crayfish/Houdini/config/packages/crayfish_commons.yml | www-data:www-data/644
crayfish_commons:\nfedora_base_uri: 'http://localhost:8080/fcrepo/rest'\nsyn_config: '/opt/fcrepo/config/syn-settings.xml'\n/opt/crayfish/Houdini/config/packages/monolog.yml | www-data:www-data/644
monolog:\nhandlers:\nhoudini:\ntype: rotating_file\npath: /var/log/islandora/Houdini.log\nlevel: DEBUG\nmax_files: 1\nThe below files are two versions of the same file to enable or disable JWT token authentication.
/opt/crayfish/Houdini/config/packages/security.yml | www-data:www-data/644
Enabled JWT token authentication:
security:\n# https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers\nproviders:\njwt_user_provider:\nid: Islandora\\Crayfish\\Commons\\Syn\\JwtUserProvider\nfirewalls:\ndev:\npattern: ^/(_(profiler|wdt)|css|images|js)/\nsecurity: false\nmain:\nanonymous: false\n# Need stateless or it reloads the User based on a token.\nstateless: true\nprovider: jwt_user_provider\nguard:\nauthenticators:\n- Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator\n# activate different ways to authenticate\n# https://symfony.com/doc/current/security.html#firewalls-authentication\n# https://symfony.com/doc/current/security/impersonating_user.html\n# switch_user: true\n# Easy way to control access for large sections of your site\n# Note: Only the *first* access control that matches will be used\naccess_control:\n# - { path: ^/admin, roles: ROLE_ADMIN }\n# - { path: ^/profile, roles: ROLE_USER }\nDisabled JWT token authentication:
security:\n# https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers\nproviders:\njwt_user_provider:\nid: Islandora\\Crayfish\\Commons\\Syn\\JwtUserProvider\nfirewalls:\ndev:\npattern: ^/(_(profiler|wdt)|css|images|js)/\nsecurity: false\nmain:\nanonymous: true\n# Need stateless or it reloads the User based on a token.\nstateless: true\n/opt/crayfish/Hypercube/cfg/config.yaml | www-data:www-data/644
---\nhypercube:\ntesseract_executable: tesseract\npdftotext_executable: pdftotext\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/hypercube.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\n/opt/crayfish/Milliner/cfg/config.yaml | www-data:www-data/644
---\nfedora_base_url: http://localhost:8080/fcrepo/rest\ndrupal_base_url: http://localhost\nmodified_date_predicate: http://schema.org/dateModified\nstrip_format_jsonld: true\ndebug: false\ndb.options:\ndriver: pdo_pgsql\nhost: 127.0.0.1\nport: 5432\ndbname: CRAYFISH_DB\nuser: CRAYFISH_DB_USER\npassword: CRAYFISH_DB_PASSWORD\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/milliner.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\n/opt/crayfish/Recast/cfg/config.yaml | www-data:www-data/644
---\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\ndrupal_base_url: http://localhost\ndebug: false\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/recast.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\nnamespaces:\n-\nacl: \"http://www.w3.org/ns/auth/acl#\"\nfedora: \"http://fedora.info/definitions/v4/repository#\"\nldp: \"http://www.w3.org/ns/ldp#\"\nmemento: \"http://mementoweb.org/ns#\"\npcdm: \"http://pcdm.org/models#\"\npcdmuse: \"http://pcdm.org/use#\"\nwebac: \"http://fedora.info/definitions/v4/webac#\"\nvcard: \"http://www.w3.org/2006/vcard/ns#\"\nFinally, we need appropriate Apache configurations for Crayfish; these will allow other services to connect to Crayfish components via their HTTP endpoints.
Each endpoint we need to be able to connect to will get its own .conf file, which we will then enable.
NOTICE
These configurations would potentially have collisions with Drupal routes, if any are created in Drupal with the same name. If this is a concern, it would likely be better to reserve a subdomain or another port specifically for Crayfish. For the purposes of this installation guide, these endpoints will suffice.
/etc/apache2/conf-available/Homarus.conf | root:root/644
Alias \"/homarus\" \"/opt/crayfish/Homarus/src\"\n<Directory \"/opt/crayfish/Homarus/src\">\n FallbackResource /homarus/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Houdini.conf | root:root/644
Alias \"/houdini\" \"/opt/crayfish/Houdini/public\"\n<Directory \"/opt/crayfish/Houdini/public\">\n FallbackResource /houdini/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Hypercube.conf | root:root/644
Alias \"/hypercube\" \"/opt/crayfish/Hypercube/src\"\n<Directory \"/opt/crayfish/Hypercube/src\">\n FallbackResource /hypercube/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Milliner.conf | root:root/644
Alias \"/milliner\" \"/opt/crayfish/Milliner/src\"\n<Directory \"/opt/crayfish/Milliner/src\">\n FallbackResource /milliner/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Recast.conf | root:root/644
Alias \"/recast\" \"/opt/crayfish/Recast/src\"\n<Directory \"/opt/crayfish/Recast/src\">\n FallbackResource /recast/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\nEnabling each of these configurations involves creating a symlink to them in the conf-enabled directory; the standardized method of doing this in Apache is with a2enconf.
sudo a2enconf Homarus Houdini Hypercube Milliner Recast\nFinally, to get these new endpoints up and running, we need to restart the Apache service.
sudo systemctl restart apache2\nWe're going to stop the Tomcat service while working on setting up Fedora to prevent any autodeploy misconfigurations.
sudo systemctl stop tomcat\nFedora\u2019s configuration and data won\u2019t live with Tomcat itself; rather, we\u2019re going to prepare a space for them to make them easier to manage.
sudo mkdir -p /opt/fcrepo/data/objects\nsudo mkdir /opt/fcrepo/config\nsudo chown -R tomcat:tomcat /opt/fcrepo\nThe method for creating the database here will closely mimic the method we used to create our database for Drupal.
sudo -u postgres psql\ncreate database FEDORA_DB encoding 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8' TEMPLATE template0;\ncreate user FEDORA_DB_USER with encrypted password 'FEDORA_DB_PASSWORD';\ngrant all privileges on database FEDORA_DB to FEDORA_DB_USER;\n\\q\nFEDORA_DB: fcrepoFEDORA_DB_USER: fedoraFEDORA_DB_PASSWORD: fedorafedora is not recommended.The Fedora configuration is going to come in a few different chunks that need to be in place before Fedora will be functional. We\u2019re going to place several files outright, with mildly modified parameters according to our configuration.
The basics of these configuration files have been pulled largely from the templates in Islandora-Devops/islandora-playbook internal Fedora role; you may consider referencing the playbook\u2019s templates directory for more details.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#namespace-prefixes","title":"Namespace prefixes","text":"i8_namespaces.yml is a list of namespaces used by Islandora that may not necessarily be present in Fedora; we add them here to ensure we can use them in queries.
/opt/fcrepo/config/i8_namespaces.yml | tomcat:tomcat/644
# Islandora 8/Fedora namespaces\n#\n# This file contains ALL the prefix mappings, if a URI\n# does not appear in this file it will be displayed as \n# the full URI in Fedora. \nacl: http://www.w3.org/ns/auth/acl#\nbf: http://id.loc.gov/ontologies/bibframe/\ncc: http://creativecommons.org/ns#\ndc: http://purl.org/dc/elements/1.1/\ndcterms: http://purl.org/dc/terms/\ndwc: http://rs.tdwg.org/dwc/terms/\nebucore: http://www.ebu.ch/metadata/ontologies/ebucore/ebucore#\nexif: http://www.w3.org/2003/12/exif/ns#\nfedoraconfig: http://fedora.info/definitions/v4/config#\nfedoramodel: info:fedora/fedora-system:def/model#\nfoaf: http://xmlns.com/foaf/0.1/\ngeo: http://www.w3.org/2003/01/geo/wgs84_pos#\ngn: http://www.geonames.org/ontology#\niana: http://www.iana.org/assignments/relation/\nislandorarelsext: http://islandora.ca/ontology/relsext#\nislandorarelsint: http://islandora.ca/ontology/relsint#\nldp: http://www.w3.org/ns/ldp#\nmemento: http://mementoweb.org/ns#\nnfo: http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#\nore: http://www.openarchives.org/ore/terms/\nowl: http://www.w3.org/2002/07/owl#\npremis: http://www.loc.gov/premis/rdf/v1#\nprov: http://www.w3.org/ns/prov#\nrdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#\nrdfs: http://www.w3.org/2000/01/rdf-schema#\nrel: http://id.loc.gov/vocabulary/relators/\nschema: http://schema.org/\nskos: http://www.w3.org/2004/02/skos/core#\ntest: info:fedora/test/\nvcard: http://www.w3.org/2006/vcard/ns#\nwebac: http://fedora.info/definitions/v4/webac#\nxml: http://www.w3.org/XML/1998/namespace\nxmlns: http://www.w3.org/2000/xmlns/\nxs: http://www.w3.org/2001/XMLSchema\nxsi: http://www.w3.org/2001/XMLSchema-instance\nWe have Fedora provide metadata for some resources that are contained in Drupal. Fedora needs to know to allow access to these External Content hosts.
We create a file /opt/fcrepo/config/allowed_external_hosts.txt | tomcat:tomcat/644
http://localhost:8000/\nNote: the trailing backslash is important here. For more information on Fedora's External Content and configuring it, see the Fedora Wiki pages
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#fedora-configuration-properties-file","title":"Fedora configuration properties file","text":"Fedora 6 now allows you to put all your configuration properties into a single file. We use 0640 permissions as you will want to put your database credentials in here.
/opt/fcrepo/config/fcrepo.properties | tomcat:tomcat/640
fcrepo.home=FCREPO_HOME\n# External content using path defined above.\nfcrepo.external.content.allowed=/opt/fcrepo/config/allowed_external_hosts.txt\n# Namespace registry using path defined above.\nfcrepo.namespace.registry=/opt/fcrepo/config/i8_namespaces.yml\nfcrepo.auth.principal.header.enabled=true\n# The principal header is the syn-setting.xml \"config\" element's \"header\" attribute\nfcrepo.auth.principal.header.name=X-Islandora\n# false to use manual versioning, true to create a version on each change\nfcrepo.autoversioning.enabled=true\nfcrepo.db.url=FCREPO_DB_URL\nfcrepo.db.user=FCREPO_DB_USERNAME\nfcrepo.db.password=FCREPO_DB_PASSWORD\nfcrepo.ocfl.root=FCREPO_OCFL_ROOT\nfcrepo.ocfl.temp=FCREPO_TEMP_ROOT\nfcrepo.ocfl.staging=FCREPO_STAGING_ROOT\n# Can be sha512 or sha256\nfcrepo.persistence.defaultDigestAlgorithm=sha512\n# Jms moved from 61616 to allow external ActiveMQ to use that port\nfcrepo.dynamic.jms.port=61626\n# Same as above\nfcrepo.dynamic.stomp.port=61623\nfcrepo.velocity.runtime.log=FCREPO_VELOCITY_LOG\nfcrepo.jms.baseUrl=FCREPO_JMS_BASE\nFCREPO_HOME - The home directory for all Fedora generated output and state. Unless otherwise specified, all logs, metadata, binaries, and internally generated indexes, etc. It would default to the Tomcat starting directory. A good default would be /opt/fcrepoFCREPO_DB_URL - This parameter allows you to set the database connection url. In general the format is as follows:
jdbc:<database_type>://<database_host>:<database_port>/<database_name>
Fedora currently supports H2, PostgresQL 12.3, MariaDB 10.5.3, and MySQL 8.0
So using the default ports for the supported databases here are the values we typically use:
jdbc:postgresql://localhost:5432/fcrepojdbc:mariadb://localhost:3306/fcrepojdbc:mysql://localhost:3306/fcrepoFCREPO_DB_USERNAME - The database username
FCREPO_DB_PASSWORD - The database passwordFCREPO_OCFL_ROOT - Sets the root directory of the OCFL. Defaults to FCREPO_HOME/data/ocfl-root if not set.FCREPO_TEMP_ROOT - Sets the temp directory used by OCFL. Defaults to FCREPO_HOME/data/temp if not set.FCREPO_STAGING_ROOT - Sets the staging directory used by OCFL. Defaults to FCREPO_HOME/data/staging if not set.FCREPO_VELOCITY_LOG - The Fedora HTML template code uses Apache Velocity, which generates a runtime log called velocity.log. Defaults to FCREPO_HOME/logs/velocity. A good choice might be /opt/tomcat/logs/velocity.logFCREPO_JMS_BASE - This specifies the baseUrl to use when generating JMS messages. You can specify the hostname with or without port and with or without path. If your system is behind a NAT firewall you may need this to avoid your message consumers trying to access the system on an invalid port. If this system property is not set, the host, port and context from the user's request will be used in the emitted JMS messages. If your Alpaca is on the same machine as your Fedora and you use the islandora-indexing-fcrepo, you could use http://localhost:8080/fcrepo/rest. Check the Lyrasis Wiki to find all of Fedora's properties
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#adding-the-fedora-variables-to-java_opts","title":"Adding the Fedora Variables toJAVA_OPTS","text":"We need our Tomcat JAVA_OPTS to include references to our repository configuration.
/opt/tomcat/bin/setenv.sh
Before:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -server -Xmx1500m -Xms1000m\"
After:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -server -Xmx1500m -Xms1000m\"
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#ensuring-tomcat-users-are-in-place","title":"Ensuring Tomcat Users Are In Place","text":"While not strictly necessary, we can use the tomcat-users.xml file to give us direct access to the Fedora endpoint. Fedora defines, out of the box, a fedoraAdmin and fedoraUser role that can be reflected in the users list for access. The following file will also include the base tomcat user. As always, these default passwords should likely not stay as the defaults.
/opt/tomcat/conf/tomcat-users.xml | tomcat:tomcat/600
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<tomcat-users xmlns=\"http://tomcat.apache.org/xml\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxsi:schemaLocation=\"http://tomcat.apache.org/xml tomcat-users.xsd\"\nversion=\"1.0\">\n<role rolename=\"tomcat\"/>\n<role rolename=\"fedoraAdmin\"/>\n<role rolename=\"fedoraUser\"/>\n<user username=\"tomcat\" password=\"TOMCAT_PASSWORD\" roles=\"tomcat\"/>\n<user username=\"fedoraAdmin\" password=\"FEDORA_ADMIN_PASSWORD\" roles=\"fedoraAdmin\"/>\n<user username=\"fedoraUser\" password=\"FEDORA_USER_PASSWORD\" roles=\"fedoraUser\"/>\n</tomcat-users>\nTOMCAT_PASSWORD: tomcatFEDORA_ADMIN_PASSWORD: islandoraFEDORA_USER_PASSWORD: islandoraFedora .war files are packaged up as releases on the official GitHub repository. You should download the most recent stable release.
sudo wget -O fcrepo.war FCREPO_WAR_URL\nsudo mv fcrepo.war /opt/tomcat/webapps\nsudo chown tomcat:tomcat /opt/tomcat/webapps/fcrepo.war\nFCREPO_WAR_URL: This can be found at the fcrepo downloads page; the file you're looking for is:As before, start the Tomcat service to get Fedora up and running.
sudo systemctl start tomcat\nNote: sometimes it takes a while for Fedora and Tomcat to start up, usually it shouldn't take longer than 5 minutes.
Once it starts up, Fedora REST API should be available at http://localhost:8080/fcrepo/rest. The username is fedoraAdmin and we defined the password before as FEDORA_ADMIN_PASSWORD (default: \"islandora\").
A compiled JAR of Syn can be found on the Syn releases page. We\u2019re going to add this to the list libraries accessible to Tomcat.
sudo wget -P /opt/tomcat/lib SYN_JAR_URL\n# Ensure the library has the correct permissions.\nsudo chown -R tomcat:tomcat /opt/tomcat/lib\nsudo chmod -R 640 /opt/tomcat/lib\nSYN_JAR_URL: The latest stable release of the Syn JAR from the releases page. Specifically, the JAR compiled as -all.jar is required.For Islandora and Fedora to talk to each other, an SSL key needs to be generated for use with Syn. We\u2019re going to make a spot where such keys can live, and generate one.
sudo mkdir /opt/keys\nsudo openssl genrsa -out \"/opt/keys/syn_private.key\" 2048\nsudo openssl rsa -pubout -in \"/opt/keys/syn_private.key\" -out \"/opt/keys/syn_public.key\"\nsudo chown www-data:www-data /opt/keys/syn*\nSyn sites and tokens belong in a settings file that we\u2019re going to reference in Tomcat.
/opt/fcrepo/config/syn-settings.xml | tomcat:tomcat/600
<config version='1' header='X-Islandora'>\n<site algorithm='RS256' encoding='PEM' anonymous='true' default='true' path='/opt/keys/syn_public.key'/>\n<token user='islandora' roles='fedoraAdmin'>ISLANDORA_SYN_TOKEN</token>\n</config>\nISLANDORA_SYN_TOKEN: islandoraReferencing the valve we\u2019ve created in our syn-settings.xml involves creating a <Valve> entry in Tomcat\u2019s context.xml:
There are two options here:
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#1-enable-the-syn-valve-for-all-of-tomcat","title":"1. Enable the Syn Valve for all of Tomcat.","text":"/opt/tomcat/conf/context.xml
Before:
29 | -->
30 | </Context>
After:
29 | -->
30 | <Valve className=\"ca.islandora.syn.valve.SynValve\" pathname=\"/opt/fcrepo/config/syn-settings.xml\"/>
31 | </Context>
Create a new file at
/opt/tomcat/conf/Catalina/localhost/fcrepo.xml
<Context>\n<Valve className=\"ca.islandora.syn.valve.SynValve\" pathname=\"/opt/fcrepo/config/syn-settings.xml\"/>\n</Context>\nYour Fedora web application needs to be deployed in Tomcat with the name fcrepo.war. Otherwise, change the name of the above XML file to match the deployed web application's name.
Finally, restart tomcat to apply the new configurations.
sudo systemctl restart tomcat\nNote: sometimes it takes a while for Fedora and Tomcat to start up, usually it shouldn't take longer than 5 minutes.
Note: after installing the Syn valve, you'll no longer be able to manually create/edit or delete objects via Fedora Web UI. All communication with Fedora will now be handled from the Islandora module in Drupal.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#redhat-logging","title":"Redhat logging","text":"Redhat systems have stopped generating an all inclusive catalina.out, the catalina.<date>.log does not include web application's log statements. To get Fedora log statements flowing, you can create your own LogBack configuration file and point to it.
/opt/fcrepo/config/fcrepo-logback.xml | tomcat:tomcat/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE configuration>\n<configuration>\n<appender name=\"STDOUT\" class=\"ch.qos.logback.core.ConsoleAppender\">\n<encoder>\n<pattern>%p %d{HH:mm:ss.SSS} [%thread] \\(%c{0}\\) %m%n</pattern>\n</encoder>\n</appender>\n<appender name=\"FILE\" class=\"ch.qos.logback.core.rolling.RollingFileAppender\">\n<file>${catalina.base}/logs/fcrepo.log</file>\n<append>true</append>\n<rollingPolicy class=\"ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy\">\n<fileNamePattern>${catalina.base}/logs/fcrepo.%d{yyyy-MM-dd}.log.%i</fileNamePattern>\n<maxFileSize>10MB</maxFileSize>\n<maxHistory>30</maxHistory>\n<totalSizeCap>2GB</totalSizeCap>\n</rollingPolicy>\n<encoder>\n<pattern>%p %d{HH:mm:ss.SSS} [%thread] \\(%c{0}\\) %m%n</pattern>\n</encoder>\n</appender>\n<logger name=\"org.fcrepo.auth\" additivity=\"false\" level=\"${fcrepo.log.auth:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.config\" additivity=\"false\" level=\"${fcrepo.log.config:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.event\" additivity=\"false\" level=\"${fcrepo.log.event:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.http.api\" additivity=\"false\" level=\"${fcrepo.log.http.api:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.http.commons\" additivity=\"false\" level=\"${fcrepo.log.http.commons:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.jms\" additivity=\"false\" level=\"${fcrepo.log.jms:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.kernel\" additivity=\"false\" level=\"${fcrepo.log.kernel:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.persistence\" additivity=\"false\" level=\"${fcrepo.log.persistence:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.search\" additivity=\"false\" level=\"${fcrepo.log.search:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.storage\" additivity=\"false\" level=\"${fcrepo.log.storage:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo\" additivity=\"false\" level=\"${fcrepo.log:-INFO}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<root level=\"${fcrepo.log.root:-WARN}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</root>\n</configuration>\nThen alter your $JAVA_OPTS like above to include
-Dlogback.configurationFile=/opt/fcrepo/config/fcrepo-logback.xml\nThis will generate a log file at ${catalina.base}/logs/fcrepo.log and will rotate each day or if the logs reaches 10MB. It will maintain 30 days of old logs, or 2GB whichever comes first.
Blazegraph needs a space for configurations and data; we\u2019re going to create this space in /opt.
sudo mkdir -p /opt/blazegraph/data\nsudo mkdir /opt/blazegraph/conf\nsudo chown -R tomcat:tomcat /opt/blazegraph\nThe Blazegraph .war file can be found in a few different places, but to ensure we\u2019re able to easily wget it, we\u2019re going to use the maven.org repository link to grab it.
cd /opt\nsudo wget -O blazegraph.war BLAZEGRAPH_WARFILE_LINK\nsudo mv blazegraph.war /opt/tomcat/webapps\nsudo chown tomcat:tomcat /opt/tomcat/webapps/blazegraph.war\n.war file within that version folder.Once this is downloaded, give it a moment to expand before moving on to the next step.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#configuring-logging","title":"Configuring Logging","text":"We would like to have an appropriate logging configuration for Blazegraph, which can be useful for looking at incoming traffic and determining if anything has gone wrong with Blazegraph. Our logger isn\u2019t going to be much different than the default logger; it can be made more or less verbose by changing the default WARN levels. There are several other loggers that can be enabled, like a SPARQL query trace or summary query evaluation log; if these are desired they should be added in. Consult the Blazegraph documentation for more details.
/opt/blazegraph/conf/log4j.properties | tomcat:tomcat/644
log4j.rootCategory=WARN, dest1\n\n# Loggers.\nlog4j.logger.com.bigdata=WARN\nlog4j.logger.com.bigdata.btree=WARN\n\n# Normal data loader (single threaded).\n#log4j.logger.com.bigdata.rdf.store.DataLoader=INFO\n\n# dest1\nlog4j.appender.dest1=org.apache.log4j.ConsoleAppender\nlog4j.appender.dest1.layout=org.apache.log4j.PatternLayout\nlog4j.appender.dest1.layout.ConversionPattern=%-5p: %F:%L: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-5p: %r %l: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-5p: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-4r(%d) [%t] %-5p %c(%l:%M) %x - %m%n\n\n# Rule execution log. This is a formatted log file (comma delimited).\nlog4j.logger.com.bigdata.relation.rule.eval.RuleLog=INFO,ruleLog\nlog4j.additivity.com.bigdata.relation.rule.eval.RuleLog=false\nlog4j.appender.ruleLog=org.apache.log4j.FileAppender\nlog4j.appender.ruleLog.Threshold=ALL\nlog4j.appender.ruleLog.File=/var/log/blazegraph/rules.log\nlog4j.appender.ruleLog.Append=true\nlog4j.appender.ruleLog.BufferedIO=false\nlog4j.appender.ruleLog.layout=org.apache.log4j.PatternLayout\nlog4j.appender.ruleLog.layout.ConversionPattern=%m\nOur configuration will be built from a few different files that we will eventually reference in JAVA_OPTS and directly apply to Blazegraph; these include most of the functional pieces Blazegraph requires, as well as a generalized configuration for the islandora namespace it will use. As with most large configurations like this, these should likely be tuned to your preferences, and the following files only represent sensible defaults.
/opt/blazegraph/conf/RWStore.properties | tomcat:tomcat/644
com.bigdata.journal.AbstractJournal.file=/opt/blazegraph/data/blazegraph.jnl\ncom.bigdata.journal.AbstractJournal.bufferMode=DiskRW\ncom.bigdata.service.AbstractTransactionService.minReleaseAge=1\ncom.bigdata.journal.Journal.groupCommit=false\ncom.bigdata.btree.writeRetentionQueue.capacity=4000\ncom.bigdata.btree.BTree.branchingFactor=128\ncom.bigdata.journal.AbstractJournal.initialExtent=209715200\ncom.bigdata.journal.AbstractJournal.maximumExtent=209715200\ncom.bigdata.rdf.sail.truthMaintenance=false\ncom.bigdata.rdf.store.AbstractTripleStore.quads=true\ncom.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false\ncom.bigdata.rdf.store.AbstractTripleStore.textIndex=false\ncom.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms\ncom.bigdata.namespace.kb.lex.com.bigdata.btree.BTree.branchingFactor=400\ncom.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=1024\ncom.bigdata.journal.Journal.collectPlatformStatistics=false\n/opt/blazegraph/conf/blazegraph.properties | tomcat:tomcat/644
com.bigdata.rdf.store.AbstractTripleStore.textIndex=false\ncom.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms\ncom.bigdata.rdf.sail.isolatableIndices=false\ncom.bigdata.rdf.store.AbstractTripleStore.justify=true\ncom.bigdata.rdf.sail.truthMaintenance=true\ncom.bigdata.rdf.sail.namespace=islandora\ncom.bigdata.rdf.store.AbstractTripleStore.quads=false\ncom.bigdata.namespace.islandora.lex.com.bigdata.btree.BTree.branchingFactor=400\ncom.bigdata.journal.Journal.groupCommit=false\ncom.bigdata.namespace.islandora.spo.com.bigdata.btree.BTree.branchingFactor=1024\ncom.bigdata.rdf.store.AbstractTripleStore.geoSpatial=false\ncom.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false\n/opt/blazegraph/conf/inference.nt | tomcat:tomcat/644
<http://pcdm.org/models#memberOf> <http://www.w3.org/2002/07/owl#inverseOf> <http://pcdm.org/models#hasMember> .\n<http://pcdm.org/models#fileOf> <http://www.w3.org/2002/07/owl#inverseOf> <http://pcdm.org/models#hasFile> .\nRWStore.properties in JAVA_OPTS","text":"In order to enable our configuration when Tomcat starts, we need to reference the location of RWStore.properties in the JAVA_OPTS environment variable that Tomcat uses.
/opt/tomcat/bin/setenv.sh
Before:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -server -Xmx1500m -Xms1000m\"
After:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -Dcom.bigdata.rdf.sail.webapp.ConfigParams.propertyFile=/opt/blazegraph/conf/RWStore.properties -Dlog4j.configuration=file:/opt/blazegraph/conf/log4j.properties -server -Xmx1500m -Xms1000m\"
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#restarting-tomcat_1","title":"Restarting Tomcat","text":"Finally, restart Tomcat to pick up the changes we\u2019ve made.
sudo systemctl restart tomcat\nThe two other files we created, blazegraph.properties and inference.nt, contain information that Blazegraph requires in order to establish and correctly use the datasets Islandora will send to it. First, we need to create a dataset - contained in blazegraph.properties - and then we need to inform that dataset of the inference set we have contained in inference.nt.
curl -X POST -H \"Content-Type: text/plain\" --data-binary @/opt/blazegraph/conf/blazegraph.properties http://localhost:8080/blazegraph/namespace\ncurl -X POST -H \"Content-Type: text/plain\" --data-binary @/opt/blazegraph/conf/inference.nt http://localhost:8080/blazegraph/namespace/islandora/sparql\nNeeds Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#in-this-section-we-will-install","title":"In this section, we will install:","text":"In our case, the default installation method for ActiveMQ via apt-get will suffice.
sudo apt-get -y install activemq\nThis will give us:
/var/lib/activemq/conf/var/lib/activemq/data/usr/share/activemqactivemq service that will be run on bootactivemq, who will be in charge of the ActiveMQ serviceTake note of the version of ActiveMQ we're going to be installing. It needs to match a Karaf blueprint we'll create later. Check the version with
sudo apt-cache policy activemq\nWrite down the version listed under Installed:.
Karaf, as well as its processes and service, will be owned by a user in charge of ensuring this portion of the stack is segregated and that the service is running.
sudo addgroup karaf\nsudo adduser karaf --ingroup karaf --home /opt/karaf --shell /usr/bin\nAs always, you will be prompted for a password, which you should create at this time. All other options can be left blank.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#downloading-and-placing-karaf","title":"Downloading and Placing Karaf","text":"Since there\u2019s no apt-get installer for Karaf, we\u2019re going to manually download and install it directly from its binary installer.
cd /opt\nsudo wget -O karaf.tar.gz KARAF_TARBALL_LINK\nsudo tar -xzvf karaf.tar.gz\nsudo chown -R karaf:karaf KARAF_DIRECTORY\nsudo mv KARAF_DIRECTORY/* /opt/karaf\nKARAF_TARBALL_LINK: It\u2019s recommended to get the most recent version of Karaf 4.2.x. This will depend on the current version of Karaf, which can be found on the Karaf downloads page under \u201cKaraf Runtime\u201d. Like Solr, you can\u2019t directly wget these links, but clicking on the .tar.gz link for the binary distribution will bring you to a list of mirrors, as well as provide you with a recommended mirror you can use here. - KARAF_DIRECTORY: This will depend on the exact version being used, but will likely be /opt/apache-karaf-VERSION, where VERSION is the current Karaf version number."},{"location":"installation/manual/installing-karaf-and-alpaca/#configuring-karaf-logging","title":"Configuring Karaf Logging","text":"We\u2019re going to apply some basic logging to our Karaf installation that should suffice for an example. In a production installation, you may want to play around with some of these values for more personally useful logging.
sudo mkdir /var/log/karaf\nsudo chown karaf:karaf /var/log/karaf\n/opt/karaf/etc/org.pos4j.pax.logging.cfg | karaf:karaf/644
# Root logger\nlog4j.rootLogger=INFO, out, osgi:*\nlog4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer\n\n# File appender\nlog4j.appender.out=org.apache.log4j.RollingFileAppender\nlog4j.appender.out.layout=org.apache.log4j.PatternLayout\nlog4j.appender.out.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name } - %X{bundle.version} | %m%n\nlog4j.appender.out.file=/var/log/karaf/karaf.log\nlog4j.appender.out.append=true\nlog4j.appender.out.maxFileSize=1MB\nlog4j.appender.out.maxBackupIndex=10\n\n# Camel Logger\nlog4j.appender.camel=org.apache.log4j.RollingFileAppender\nlog4j.appender.camel.layout=org.apache.log4j.PatternLayout\nlog4j.appender.camel.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.na me} - %X{bundle.version} | %m%n\nlog4j.appender.camel.file=/var/log/karaf/camel.log\nlog4j.appender.camel.append=false\nlog4j.appender.camel.maxFileSize=1MB\nlog4j.appender.camel.maxBackupIndex=10\n\nlog4j.logger.org.apache.camel=INFO, camel\n\n# Islandora Logger\nlog4j.appender.islandora=org.apache.log4j.RollingFileAppender\nlog4j.appender.islandora.layout=org.apache.log4j.PatternLayout\nlog4j.appender.islandora.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundl e.name} - %X{bundle.version} | %m%n\nlog4j.appender.islandora.file=/var/log/karaf/islandora.log\nlog4j.appender.islandora.append=false\nlog4j.appender.islandora.maxFileSize=1MB\nlog4j.appender.islandora.maxBackupIndex=10\n\nlog4j.logger.ca.islandora.camel=INFO, islandora\nsetenv.sh Script for Karaf","text":"Similar to Tomcat, our Karaf service is going to rely on a setenv shell script to determine environment variables Karaf needs in place when running. For now, this will simply be the path to JAVA_HOME, but this also accepts many other parameters you can find in the default setenv script.
/opt/karaf/bin/setenv | karaf:karaf/755
#!/bin/sh\nexport JAVA_HOME=\"PATH_TO_JAVA_HOME\"\nPATH_TO_JAVA_HOME: This will be the same JAVA_HOME we used when installing Tomcat , and can be found using the same method (i.e., still /usr/lib/jvm/java-11-openjdk-amd64 if that's what it was before)."},{"location":"installation/manual/installing-karaf-and-alpaca/#initializing-karaf","title":"Initializing Karaf","text":"We\u2019re going to start Karaf, then run the installer to put our configurations in place and generate a Karaf service. Once these are installed, we\u2019re going to stop Karaf, as from there on out its start/stop management should be handled via that service.
First we need to enable the default Karaf user in /opt/karaf/etc/users.properties:
Before:
32 | # karaf = karaf,_g_:admingroup
33 | # _g_\\:admingroup = group,admin,manager,viewer,systembundles,ssh
After:
32 | karaf = karaf,_g_:admingroup
33 | _g_\\:admingroup = group,admin,manager,viewer,systembundles,ssh
Save the file and close it, then:
sudo -u karaf /opt/karaf/bin/start\n# You may want to wait a bit for Karaf to start.\n# If you're not sure whether or not it's running, you can always run:\n# ps aux | grep karaf\n# to see if the server is up and running.\n/opt/karaf/bin/client feature:install wrapper\n/opt/karaf/bin/client wrapper:install\n/opt/karaf/bin/stop\nInstalling the Karaf wrapper generates several service files that can be used on different types of systems. For Debian and Ubuntu installation we want to enable the karaf.service service so that Karaf is properly started on boot.
sudo systemctl enable /opt/karaf/bin/karaf.service\nsudo systemctl start karaf\nWe can check if the service started correctly with:
sudo systemctl status karaf\nPress Q to close the status.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#alpaca-10x","title":"Alpaca 1.0.x","text":""},{"location":"installation/manual/installing-karaf-and-alpaca/#adding-the-required-karaf-repositories","title":"Adding the Required Karaf Repositories","text":"Karaf features can be installed from several different types of sources, but the fastest and easiest way to do so is from existing repository URLs that we can just plug into Karaf to provide us feature lists prepared and ready for installation. Like most interactions with Karaf, we can add these repositories using its built-in client.
Notice
These repositories are updated consistently, and their updates include revised dependency lists. Commonly, when repositories are out of date or otherwise mismatched, feature installation can result in an Unable to resolve root: missing requirement error; for this reason, this guide recommends using recently-updated versions of these repositories. That being said, if such errors occur despite installing the latest versions of these features, the maintainer of the features repository should be informed.
For the Karaf features we\u2019re going to install, we need a few different repositories to be added to the list:
/opt/karaf/bin/client repo-add mvn:org.apache.activemq/activemq-karaf/ACTIVEMQ_KARAF_VERSION/xml/features\n/opt/karaf/bin/client repo-add mvn:org.apache.camel.karaf/apache-camel/APACHE_CAMEL_VERSION/xml/features\n/opt/karaf/bin/client repo-add mvn:ca.islandora.alpaca/islandora-karaf/ISLANDORA_KARAF_VERSION/xml/features\n# XXX: This shouldn't be strictly necessary, but appears to be a missing\n# upstream dependency for some fcrepo features.\n/opt/karaf/bin/client repo-add mvn:org.apache.jena/jena-osgi-features/JENA_OSGI_VERSION/xml/features\nACTIVEMQ_KARAF_VERSION: The version of ActiveMQ we wrote down at the beginning of this chapter when installing ActiveMQ via apt-get - APACHE_CAMEL_VERSION: The latest version of Apache Camel 2.x.x; you can find this listed at the apache-camel repository page (e.g., 2.25.4 at the time of writing) - ISLANDORA_KARAF_VERSION: The latest version of Islandora Karaf 1.x; you can find this listed at the islandora-karaf repository page (e.g., 1.0.5 at the time of writing) - JENA_OSGI_VERSION: The latest version of the Apache Jena 3.x OSGi features; you can find this listed at the jena-osgi-features repository page (e.g., 3.17.0 at the time of writing)"},{"location":"installation/manual/installing-karaf-and-alpaca/#configuring-karaf-features","title":"Configuring Karaf Features","text":"Our installed Karaf features require configuration files to know exactly where to route things coming and going from them.
/opt/karaf/etc/ca.islandora.alpaca.http.client.cfg | karaf:karaf/644
token.value=ISLANDORA_SYN_TOKEN\nISLANDORA_SYN_TOKEN: This should be the same token that was established during the installation of Syn in your syn-settings.xml file /opt/karaf/etc/org.fcrepo.camel.indexing.triplestore.cfg | karaf:karaf/644
input.stream=activemq:topic:fedora\ntriplestore.reindex.stream=activemq:queue:triplestore.reindex\ntriplestore.baseUrl=http://localhost:8080/blazegraph/namespace/islandora/sparql\n/opt/karaf/etc/ca.islandora.alpaca.indexing.triplestore.cfg | karaf:karaf/644
error.maxRedeliveries=10\nindex.stream=activemq:queue:islandora-indexing-triplestore-index\ndelete.stream=activemq:queue:islandora-indexing-triplestore-delete\ntriplestore.baseUrl=http://localhost:8080/blazegraph/namespace/islandora/sparql\n/opt/karaf/etc/ca.islandora.alpaca.indexing.fcrepo.cfg | karaf:karaf/644
error.maxRedeliveries=5\nnode.stream=activemq:queue:islandora-indexing-fcrepo-content\nnode.delete.stream=activemq:queue:islandora-indexing-fcrepo-delete\nmedia.stream=activemq:queue:islandora-indexing-fcrepo-media\nfile.stream=activemq:queue:islandora-indexing-fcrepo-file\nfile.delete.stream=activemq:queue:islandora-indexing-fcrepo-file-delete\nmilliner.baseUrl=http://localhost/milliner\nFor those services in Crayfish we have set up to provide derivatives to Islandora resources, we need connector blueprints to tell the derivative connector how to route incoming requests, run conversions, and return outgoing derivatives.
Our blueprints are going to look largely similar between services, with only a few properties changing between them. Largely, these mainly just need to match the ActiveMQ queues we established in the previous configuration, and route to the correct Crayfish service.
/opt/karaf/deploy/ca.islandora.alpaca.connector.ocr.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.ocr\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-ocr\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/hypercube\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorOCR\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.houdini.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.houdini\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-houdini\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/houdini/convert\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorHoudini\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.homarus.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.homarus\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-homarus\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/homarus/convert\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorHomarus\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.fits.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.fits\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-fits\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/crayfits\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorfits\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\nBefore we can configure the features we\u2019re going to use, they need to be installed. Some of these installations may take some time.
/opt/karaf/bin/client feature:install camel-blueprint\n/opt/karaf/bin/client feature:install activemq-blueprint\n/opt/karaf/bin/client feature:install fcrepo-service-activemq\n# This again should not be strictly necessary, since this isn't the triplestore\n# we're using, but is being included here to resolve the aforementioned\n# missing link in the dependency chain.\n/opt/karaf/bin/client feature:install jena\n/opt/karaf/bin/client feature:install fcrepo-camel\n/opt/karaf/bin/client feature:install fcrepo-indexing-triplestore\n/opt/karaf/bin/client feature:install islandora-http-client\n/opt/karaf/bin/client feature:install islandora-indexing-triplestore\n/opt/karaf/bin/client feature:install islandora-indexing-fcrepo\n/opt/karaf/bin/client feature:install islandora-connector-derivative\nAt this point, Karaf components should be up and running, but it's a good idea to double-check that this is the case. We can do this from within the Karaf client by taking a look at its component list.
# Until this point, we've been running Karaf commands from outside; we can hop\n# into the client, however, and run commands from directly within.\n/opt/karaf/bin/client\n# This takes us into the Karaf client so we can run commands.\nla | grep islandora\nla | grep fcrepo\n# It may be a good idea to use this to look up to the other components we\n# installed.\nlogout\nFor the above la | grep commands, components that are running should be listed as Active.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-solr/#in-this-section-we-will-install","title":"In this section, we will install:","text":"The Solr binaries can be found at the Solr downloads page; the most recent stable release of Solr 8 should be used.
# While generally we download tarballs as .tar.gz files without version\n# information, the Solr installer is a bit particular in that it expects a .tgz\n# file with the same name as the extracted folder it contains. It's odd, and we\n# can't really get around it.\ncd\nwget SOLR_DOWNLOAD_LINK\ntar -xzvf SOLR_TARBALL\nSOLR_DOWNLOAD_LINK: NOTICE: This will depend on a few different things, not least of all the current version of Solr. The link to the .tgz for the binary on the downloads page will take you to a list of mirrors that Solr can be downloaded from, and provide you with a preferred mirror at the top. This preferred mirror should be used as the SOLR_DOWNLOAD_LINK. - SOLR_TARBALL: The filename that was downloaded, e.g., solr-8.9.0.tgz"},{"location":"installation/manual/installing-solr/#running-the-solr-installer","title":"Running the Solr Installer","text":"Solr includes an installer that does most of the heavy lifting of ensuring we have a Solr user, a location where Solr lives, and configurations in place to ensure it\u2019s running on boot.
sudo UNTARRED_SOLR_FOLDER/bin/install_solr_service.sh SOLR_TARBALL\nUNTARRED_SOLR_FOLDER: This will likely simply be solr-VERSION, where VERSION is the version number that was downloaded. The port that Solr runs on can potentially be configured at this point, but we'll expect it to be running on 8983.
Wait until the command output reaches:
Started Solr server on port 8983 (pid=****). Happy searching!\nsystemd[1]: Started LSB: Controls Apache Solr as a Service.\nAfter which you can press q to quit the output (this won't kill Solr so it's safe).
You can check if Solr is running correctly by going to http://localhost:8983/solr
"},{"location":"installation/manual/installing-solr/#increasing-the-open-file-limit-optional","title":"Increasing the Open File Limit (Optional)","text":"Solr's installation guide recommends that you increase the open file limit so that operations aren't disrupted while Solr is trying to access things in its index. This limit can be increased while the system is running, but doing so won't persist after a reboot. You can hard-increase this limit using your system's sysctl file:
/etc/sysctl.conf
Add the following line to the end of the file:
fs.file-max = 65535\nThen apply your new configuration.
sudo sysctl -p\nInitially, our new Solr core will contain a configuration copied from the example included with the installation, so that we have something to work with when we configure this on the Drupal side. We\u2019ll later update this with generated configurations we create in Drupal.
cd /opt/solr\nsudo mkdir -p /var/solr/data/SOLR_CORE/conf\nsudo cp -r example/files/conf/* /var/solr/data/SOLR_CORE/conf\nsudo chown -R solr:solr /var/solr\nsudo -u solr bin/solr create -c SOLR_CORE -p 8983\nSOLR_CORE: islandora8 You should see an output similar to this:
WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.\n To turn off: bin/solr config -c islandora8 -p 8983 -action set-user-property -property update.autoCreateFields -value false\n\nCreated new core 'islandora8'\nsearch_api_solr","text":"Rather than use an out-of-the-box configuration that won\u2019t be suitable for our purposes, we\u2019re going to use the Drupal search_api_solr module to generate one for us. This will also require us to install the module so we can create these configurations using Drush.
cd /opt/drupal\nsudo -u www-data composer require drupal/search_api_solr:^4.2\ndrush -y en search_api_solr\nYou should see an output similar to this:
The following module(s) will be enabled: search_api_solr, language, search_api\n\n // Do you want to continue?: yes.\n\n [success] Successfully enabled: search_api_solr, language, search_api\nBefore we can create configurations to use with Solr, the core we created earlier needs to be referenced in Drupal.
Log in to the Drupal site at /user using the sitewide administrator username and password (if using defaults from previous chapters this should be islandora and islandora), then navigate to /admin/config/search/search-api/add-server.
Fill out the server addition form using the following options:
SERVER_NAME: islandora8machine_name generated next to the server name you type in; this will be used in the next step.As a recap for this configuration:
Click Save to create the server configuration.
NOTICE You can ignore the error about an incompatible Solr schema; we're going to set this up in the next step. In fact, if you refresh the page after restarting Solr in the next step, you should see the error disappear.
"},{"location":"installation/manual/installing-solr/#generating-and-applying-solr-configurations","title":"Generating and Applying Solr Configurations","text":"Now that our core is in place and our Drupal-side configurations exist, we\u2019re ready to generate Solr configuration files to connect this site to our search engine.
cd /opt/drupal\ndrush solr-gsc SERVER_MACHINE_NAME /opt/drupal/solrconfig.zip\nunzip -d ~/solrconfig solrconfig.zip\nsudo cp ~/solrconfig/* /var/solr/data/SOLR_CORE/conf\nsudo systemctl restart solr\nSERVER_MACHINE_NAME: This should be the machine_name that was automatically generated when creating the configuration in the above step."},{"location":"installation/manual/installing-solr/#adding-an-index","title":"Adding an Index","text":"In order for content to be indexed back into Solr, a search index needs to be added to our server. Navigate to /admin/config/search/search-api/add-index and check off the things you'd like to be indexed.
NOTICE You should come back here later and reconfigure this after completing the last step in this guide. The default indexing configuration is pretty permissive, and you may want to restrict, for example, indexed content to just Islandora-centric bundles. This guide doesn't set up the index's fields either, which are going to be almost wholly dependent on the needs of your installation. Once you complete that configuration later on, re-index Solr from the configuration page of the index we're creating here.
Click Save to add your index and kick off indexing of existing items.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/","title":"Installing Tomcat and Cantaloupe","text":"Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Tomcat runs in a Java runtime environment, so we'll need one to continue. In our case, OpenJDK 11 is open-source, free to use, and can fairly simply be installed using apt-get:
sudo apt-get -y install openjdk-11-jdk openjdk-11-jre\nThe installation of OpenJDK via apt-get establishes it as the de-facto Java runtime environment to be used on the system, so no further configuration is required.
The resultant location of the java JRE binary (and therefore, the correct value of JAVA_HOME when it\u2019s referenced) will vary based on the specifics of the machine it\u2019s being installed on; that being said, you can find its exact location using update-alternatives:
update-alternatives --list java\ntomcat User","text":"Apache Tomcat, and all its processes, will be owned and managed by a specific user for the purposes of keeping parts of the stack segregated and accountable.
sudo addgroup tomcat\nsudo adduser tomcat --ingroup tomcat --home /opt/tomcat --shell /usr/bin\nYou will be prompted to create a password for the tomcat user; all the other information as part of the adduser command can be ignored.
Tomcat 9 itself can be installed in several different ways; while it\u2019s possible to install via apt-get, this doesn\u2019t give us a great deal of control over exactly how we\u2019re going to run and manage it; as a critical part of the stack, it is beneficial for our purposes to have a good frame of reference for the inner workings of Tomcat.
We\u2019re going to download the latest version of Tomcat to /opt and set it up so that it runs automatically. Bear in mind that with the following commands, this is going to be entirely relative to the current version of Tomcat 9, which we\u2019ll try to mitigate as we go.
cd /opt\nsudo wget -O tomcat.tar.gz TOMCAT_TARBALL_LINK\nsudo tar -zxvf tomcat.tar.gz\nsudo mv /opt/TOMCAT_DIRECTORY/* /opt/tomcat\nsudo chown -R tomcat:tomcat /opt/tomcat\nTOMCAT_TARBALL_LINK: No default can be provided here; you should navigate to the Tomcat 9 downloads page and grab the link to the latest .tar.gz file under the \u201cCore\u201d section of \u201cBinary Distributions\u201d. It is highly recommended to grab the latest version of Tomcat 9, as it will come with associated security patches and fixes. - TOMCAT_DIRECTORY: This will also depend entirely on the exact version of tomcat downloaded - for example, apache-tomcat-9.0.50. Again, ls /opt can be used to find this."},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#creating-a-setenvsh-script","title":"Creating a setenv.sh Script","text":"When Tomcat runs, some configuration needs to be pre-established as a series of environment variables that will be used by the script that runs it.
/opt/tomcat/bin/setenv.sh | tomcat:tomcat/755
export CATALINA_HOME=\"/opt/tomcat\"\nexport JAVA_HOME=\"PATH_TO_JAVA_HOME\"\nexport JAVA_OPTS=\"-Djava.awt.headless=true -server -Xmx1500m -Xms1000m\"\nPATH_TO_JAVA_HOME: This will vary a bit depending on the environment, but will likely live in /usr/lib/jvm somewhere (e.g., /usr/lib/jvm/java-11-openjdk-amd64); again, in an Ubunutu environment you can check a part of this using update-alternatives --list java, which will give you the path to the JRE binary within the Java home. Note that update-alternatives --list java will give you the path to the binary, so for PATH_TO_JAVA_HOME delete the /bin/java at the end to get the Java home directory, so it should look something like this: export JAVA_HOME=\"/usr/lib/jvm/java-11-openjdk-amd64\"\nTomcat includes two shell scripts we\u2019re going to make use of - startup.sh and shutdown.sh - which are light wrappers on top of a third script, catalina.sh, which manages spinning up and shutting down the Tomcat server.
Debian and Ubuntu use systemctl to manage services; we\u2019re going to create a .service file that can run these shell scripts.
/etc/systemd/system/tomcat.service | root:root/755
[Unit]\nDescription=Tomcat\n\n[Service]\nType=forking\nExecStart=/opt/tomcat/bin/startup.sh\nExecStop=/opt/tomcat/bin/shutdown.sh\nSyslogIdentifier=tomcat\n\n[Install]\nWantedBy=multi-user.target\nWe\u2019re going to both enable and start Tomcat. Enabling Tomcat will ensure that it starts on boot, the timing of which is defined by the [Install] section\u2019s WantedBy statement, which specifies what it should start after. This is separate from starting it, which we need to do now in order to get Tomcat up and running without requiring a reboot.
sudo systemctl enable tomcat\nsudo systemctl start tomcat\nWe can check that Tomcat has started by running sudo systemctl status tomcat | grep Active; we should see that Tomcat is active (running), which is the correct result of startup.sh finishing its run successfully.
Since version 5, Cantaloupe is released as a standalone Java application and is no longer deployed in Tomcat via a .war file. Even so, we can still fine-tune how it runs and even install it as a service.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#downloading-cantaloupe","title":"Downloading Cantaloupe","text":"Releases of Cantaloupe live on the Cantaloupe release page; the latest version can be found here as a .zip file.
sudo wget -O /opt/cantaloupe.zip CANTALOUPE_RELEASE_URL\nsudo unzip /opt/cantaloupe.zip\nCANTALOUPE_RELEASE_URL: It\u2019s recommended we grab the latest version of Cantaloupe 5. This can be found on the above-linked release page, as the .zip version; for example, https://github.com/cantaloupe-project/cantaloupe/releases/download/v5.0.3/cantaloupe-5.0.3.zip - make sure not to download the source code zip file as that isn't compiled for running out-of-the-box."},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#creating-a-cantaloupe-configuration","title":"Creating a Cantaloupe Configuration","text":"Cantaloupe pulls its configuration from a file called cantaloupe.properties; there are also some other files that can contain instructions for Cantaloupe while it\u2019s running; specifically, we\u2019re going to copy over the delegates.rb file, which can also contain custom configuration. We won\u2019t make use of this file; we\u2019re just copying it over for demonstration purposes.
Creating these files from scratch is not recommended; rather, we\u2019re going to take the default cantaloupe configurations and plop them into their own folder so we can work with them.
sudo mkdir /opt/cantaloupe_config\nsudo cp CANTALOUPE_VER/cantaloupe.properties.sample /opt/cantaloupe_config/cantaloupe.properties\nsudo cp CANTALOUPE_VER/delegates.rb.sample /opt/cantaloupe_config/delegates.rb\nCANTALOUPE_VER: This will depend on the exact version of Cantaloupe downloaded; in the above example release, this would be cantaloupe-5.0.3 The out-of-the-box configuration will work fine for our purposes, but it\u2019s highly recommended that you take a look through the cantaloupe.properties and see what changes can be made; specifically, logging to actual logfiles isn\u2019t set up by default, so you may want to take a peek at the log.application.SyslogAppender or log.application.RollingFileAppender, as well as changing the logging level.
Since it is a standalone application, we can configure Cantaloupe as a systemd service like we did with Tomcat, so it can start on boot:
/etc/systemd/system/cantaloupe.service | root:root/755
[Unit]\nDescription=Cantaloupe\n\n[Service]\nExecStart=java -cp /opt/CANTALOUPE_VER/CANTALOUPE_VER.jar -Dcantaloupe.config=/opt/cantaloupe_config/cantaloupe.properties -Xmx1500m -Xms1000m edu.illinois.library.cantaloupe.StandaloneEntry\nSyslogIdentifier=cantaloupe\n\n[Install]\nWantedBy=multi-user.target\nCANTALOUPE_VER: This will depend on the exact version of Cantaloupe downloaded; in the above example release, this would be cantaloupe-5.0.3 We can now enable the service and run it:
sudo systemctl enable cantaloupe\nsudo systemctl start cantaloupe\nWe can check the service status with sudo systemctl status cantaloupe | grep Active and the splash screen of Cantaloupe should be available at http://localhost:8182
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
Notice
The manual installation guide is not intended to describe the Islandora installation but rather an Islandora installation. The server created using this guide is not hardened, will not be easily scalable, and the components may not be configured in a way you consider easy to work with. A production instance of Islandora should be installed and maintained by a professional with an understanding of Linux and server administration.
This guide will contain generalized steps on installation and configuration of the various components, but will contain specific example commands for executing these steps on an Ubuntu 18.04 Server.
"},{"location":"installation/manual/introduction/#some-prerequisite-knowledge","title":"Some Prerequisite Knowledge","text":"This guide assumes the user has some knowledge:
nano or vi/vim, or creating these files locally and uploading them to the server. It should also generally be assumed that most of these configuration files will have to be created or edited using sudo, and that permissions and ownership may need to be specified on these files.The steps in this guide are listed in chronological order of installation and configuration. Some sections will reference variables and rely on components installed in previous sections. It does not account for skipping over or otherwise changing the order of installation steps; bear this in mind if you decide to do things out of the provided order.
"},{"location":"installation/manual/introduction/#replacement-variables","title":"Replacement Variables","text":"It is expected that the person setting up the site may want to use different usernames, passwords, and other such variables than the ones presented by default in this guide. Additionally, some defaults can't be provided, such as up-to-date version information for externally-provided components. In such cases, the replacement variables will be placed in all capital letters, and a description of the variables, any possible defaults, and how to get up-to-date information, will be listed below.
"},{"location":"installation/manual/introduction/#bash-commands","title":"Bash Commands","text":"Notice
Command blocks are always assumed to start at the home folder of the user originally created during the server installation. They are never run as root; if root access is required, sudo will be specified, and if files are created belonging to root that should not belong to root, chmod and chown will be run against them immediately afterwards to ensure correct permissions. If commands need to be run from a different working directory, an absolute path will be specified to use with cd. If you're concerned about whether or not a code block can be run from your current working directory, run cd ~ before executing any commands in it.
Commands to be run on the command line will be placed in code blocks, with one command per line, and any replacement variables below, e.g.,
sudo run --this-command\npython3 run.py /this/other/command --with-param PARAMETER\nPARAMETER: some_sensible_default, perhaps with an explanation of why, or how to determine alternatives"},{"location":"installation/manual/introduction/#editing-files-in-place","title":"Editing Files In Place","text":"When an individual file needs to be modified in place (as opposed to replacing it outright), a Before and After quote will be provided that identifies one or more lines in the file, what the default installed version of that file looks like on that line, and what the line should look like after it has been modified, like so:
/path/to/file_being_modified
Before:
174 | Here is what line 174 in the file looked like before
175 | And here is what the following line looked like in the file before
After:
174 | Here is what line 174 should look like after modification
175 | And here is what the following line should look like after modification: VARIABLE
VARIABLE: some_value, perhaps with an explanation of whyIt should be noted that configuration files and the like are subject to change by the organizations that maintain their respective applications; this guide generally recommends installing the latest version of these applications, as these generally include security updates. It is expected that the implementer will be able to search through a file and find specific lines in the case where the maintaining organization has moved it in a subsequent patch. In most cases, configuration files will be provided outright to avoid these scenarios.
"},{"location":"installation/manual/introduction/#adding-or-replacing-files","title":"Adding or Replacing Files","text":"When a file needs to be added or replaced, it will be described in three sections:
chmod and chown appropriately to apply the owner, group, and umask/the/path/to/some/file.php | owner:group/umask
<?php\n/**\n * Here is the entire contents of a PHP file\n */\nfunction do_something($to_this) {\n$to_this += THE_NUMBER_TO_ADD_TO_THIS;\n}\n?>\nTHE_NUMBER_TO_ADD_TO_THIS: 12, perhaps with an explanation of why, or other numbers that may be appropriate"},{"location":"installation/manual/introduction/#troubleshooting","title":"Troubleshooting","text":"The most common issues you will likely run into when manually provisioning a server are:
ls -la, and ensure their ownership using chown USER for files, and chown -R USER for directoriesFor any other issues, don't hesitate to email the mailing list to ask for help. If you think that a part of the installation documentation is incorrect or could be improved, please create an issue in the documentation issues queue and give it a documentation tag. Bear in mind that this guide is built for Ubuntu 18.04 and attempts to give generalized instructions; you will likely naturally encounter situations where your own environment needs to differ from the guide.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/preparing-a-webserver/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Apache can typically be installed and configured outright by your operating system\u2019s package manager:
sudo apt-get -y install apache2 apache2-utils\nThis will install:
systemd service that will ensure Apache can be stopped and started, and will run when the machine is powered on/etc/apache2, including the basic configuration, ports configuration, enabled mods, and enabled sites/var/www/html, configured to be the provided server on port :80 in /etc/apache2/sites-enabled/000-default.conf; we\u2019ll make changes and additions to this file laterwww-data, which we will use to read/write web documents.We\u2019re going to enable a couple of Apache mods that Drupal highly recommends installing, and which are de-facto considered required by Islandora:
sudo a2enmod ssl\nsudo a2enmod rewrite\nsudo systemctl restart apache2\nwww-data Group","text":"Since the user we are currently logged in as is going to work quite a bit inside the Drupal directory, we want to give it group permissions to anything the www-data group has access to. When we run composer, www-data will also be caching data in our own home directory, so we want this group modification to go in both directions.
N.B. This code block uses backticks, not single quotes; this is an important distinction as backticks have special meaning in bash.
Note If doing this in the terminal, replace \"whoami\" with your username and remove the backticks
sudo usermod -a -G www-data `whoami`\nsudo usermod -a -G `whoami` www-data\n# Immediately log back in to apply the new group.\nsudo su `whoami`\nIf you're running Debian 11 you should be able to install PHP 7.4 from the apt packages directly:
sudo apt-get -y install php7.4 php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-imap php7.4-json php7.4-mbstring php7.4-opcache php7.4-xml php7.4-yaml php7.4-zip libapache2-mod-php7.4 php-pgsql php-redis php-xdebug unzip\nIf you're running Debian 10, the repository for the PHP 7.4 packages needs to be installed first:
sudo apt-get -y install lsb-release apt-transport-https ca-certificates\nsudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg\necho \"deb https://packages.sury.org/php/ $(lsb_release -sc) main\" | sudo tee /etc/apt/sources.list.d/php.list\nsudo apt-get update\nsudo apt-get -y install php7.4 php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-imap php7.4-json php7.4-mbstring php7.4-opcache php7.4-xml php7.4-yaml php7.4-zip libapache2-mod-php7.4 php-pgsql php-redis php-xdebug unzip\nThis will install a series of PHP configurations and mods in /etc/php/7.4, including:
mods-available folder (from which everything is typically enabled by default)apache2 foldercli folderunzip, which is important for PHP\u2019s zip module to function correctly despite it not being a direct dependency of the module. We will also need to unzip some things later, so this is convenient to have in place early in the installation process.PostgreSQL can generally be easily installed using your operating system\u2019s package manager. It is typically sensible to install the version the system recognizes as up-to-date. We\u2019re simply going to install the database software:
sudo apt-get -y install postgresql\nThis will install:
postgres; this will be the only user, by default, that has permission to run the psql binary and have access to Postgres configurations/usr/bin/psql, which anyone - even root - will get kicked out of the moment they run it, since only the postgres user has permission to run any Postgres commands/etc/postgresql/11/main which can be used to modify how PostgreSQL works.A modification needs to be made to the PostgreSQL configuration in order for Drupal to properly install and function. This change can be made to the main configuration file at /etc/postgresql/11/main/postgresql.conf:
Before:
558 | #bytea_output = \u2018hex\u2019 # hex, escape
After:
558 | bytea_output = \u2018escape\u2019
(Remove the \"# hex, escape\" comment and change the value from \"hex\" to \"escape\")
The postgresql service should be restarted to accept the new configuration:
sudo systemctl restart postgresql\nIslandora Starter Site uses the built-in Drupal \"Audio\" media type for video. Islandora Starter Site configures it to accept files of type mp3, wav, and aac.
Islandora uses Homarus (ffmpeg as a microservice) to create audio derivatives. Islandora Starter Site sets you up to create:
-codec:a libmp3lame -q:a 5, stored in the public filesystemThese parameters can be changed in the configuration for the Drupal Action that Islandora uses to generate an Audio derivative.
Islandora Starter Site sets up a context to automatically create this derivative when:
http://pcdm.org/use#OriginalFile)http://purl.org/coar/resource_type/c_18cc)The mimetype formats allowed by Homarus are configured in Homarus itself - see Installing Crayfish
"},{"location":"models/audio/#display","title":"Display","text":"Drupal provides an \"Audio\" field formatter for file fields that displays a simple playable audio widget. It works but does not support captions/subtitles. Islandora provides an \"Audio with captions\" formatter that allows for captions.
To use the captions feature out of the box, add the captions track as a WEBVTT file (.vtt) in the Audio media's \"Track\" field (see below regarding which Audio media to use). If you don't have the \"Track\" field (provided by Islandora Starter Site), create a field of type \"Media Track\" (a type provided by Islandora) on the same Media (or more broadly, same entity) as your audio file. Then use the Manage Display page to set your audio file field to render using the \"Audio with captions\" field formatter.
If you're using Islandora Starter Site, you can expect to see an audio player on your node. This is done with a special view (an EVA view) that displays service files, which is configured to show on Repository Item's default display mode (on the \"Manage Display\" page). The EVA view renders the service file media using the \"Source\" view mode. For Audio media, this is configured to show only the audio file using the \"Audio with Captions\" widget. Note that captions, to be displayed, must be on the media that is playing. Thus, when the Service file media is being played, captions on the Original File media are ignored.
"},{"location":"models/video/","title":"Video","text":""},{"location":"models/video/#media-and-file-types","title":"Media and file types","text":"Islandora Starter Site uses the built-in Drupal \"Video\" media type for videos. Islandora Starter Site configures it to accept files of type mp4.
The Install Profile also makes available the \"Remote video\" media type, but its behaviour has not been publicly documented.
"},{"location":"models/video/#derivatives","title":"Derivatives","text":"Islandora uses Homarus (ffmpeg as a microservice) to create video derivatives. Islandora Starter Site sets you up to create:
-ss 00:00:01.000 -frames 1 -vf scale=100:-2. These parameters can be changed in the configuration for the Drupal Actions that Islandora uses to create video derivatives.
Islandora Starter Site sets up a context to automatically create this derivative when:
http://pcdm.org/use#OriginalFile)http://purl.org/coar/resource_type/c_18cc)The mimetype formats allowed by Homarus are configured in Homarus itself - see Installing Crayfish
"},{"location":"models/video/#display","title":"Display","text":"Drupal provides a \"Video\" field formatter for file fields that displays a simple playable video widget. It works but does not support captions/subtitles. Islandora provides a \"Video with Captions\" formatter that allows for captions.
To use the Captions feature out of the box, add the captions track as a WEBVTT file (.vtt) in the Video media's \"Track\" field (see below regarding which Video media to use). If you don't have the \"Track\" field (provided by Islandora Starter Site), create a field of type \"Media Track\" (a type provided by Islandora) on the same Media (or more broadly, same entity) as your audio file. Then use the Manage Display page to set your audio file field to render using the \"Audio with captions\" field formatter.
If you're using Islandora Starter Site, you can expect to see a video player on your node. This is done with a special view (an EVA view) that displays service files, which is configured to show on the default view of Repository Items (see the \"Manage Display\" page). The EVA view renders the service file media using the \"Source\" view mode. For Video media, this is configured to show only the video file using the \"Video with Captions\" formatter. Thus, when the Service file media is being played, captions on the Original File media are ignored.
"},{"location":"release_notes/8.x-2.0/","title":"Islandora 2.0.0 Release Notes / Upgrade Instructions","text":""},{"location":"release_notes/8.x-2.0/#update-to-php-74","title":"Update to PHP 7.4","text":"Upstream dependencies have forced us to upgrade PHP to 7.4. How to do this varies by operating system, but for Ubuntu 18.04 the following will work:
sudo apt install software-properties-commonsudo add-apt-repository ppa:ondrej/phpsudo apt updatesudo apt install php7.4sudo apt install php7.4-curl php7.4-xml php7.4-gd php7.4-imap php7.4-mbstring php7.4-mysqlHere's the steps to update Alpaca code in general. None of this is specific to this release, it's just the process.
Move your blueprint config out of the deploy directory temporarily
mv /opt/karaf/deploy/ca.islandora.* /opt/karafNow we'll uninstall the old features and install the new ones. We're assuming you're currently on 1.0.3 and want to update to 1.0.5, but you may be on an older version. Keep that in mind when copy/pasting these commands.
/opt/karaf/bin/client \"feature:uninstall islandora-connector-derivative\"/opt/karaf/bin/client \"feature:uninstall islandora-indexing-fcrepo\"/opt/karaf/bin/client \"feature:uninstall islandora-indexing-triplestore\"/opt/karaf/bin/client \"feature:uninstall islandora-http-client\"/opt/karaf/bin/client \"feature:repo-remove mvn:ca.islandora.alpaca/islandora-karaf/1.0.3/xml/features\"/opt/karaf/bin/client \"feature:repo-add mvn:ca.islandora.alpaca/islandora-karaf/1.0.5/xml/features\"/opt/karaf/bin/client \"feature:install islandora-http-client\"/opt/karaf/bin/client \"feature:install islandora-indexing-triplestore\"/opt/karaf/bin/client \"feature:install islandora-indexing-fcrepo\"/opt/karaf/bin/client \"feature:install islandora-connector-derivative\"Move the blueprint files back to the deploy folder to deploy the derivative routes
mv /opt/karaf/ca.islandora.* /opt/karaf/deployYou can check that everything is working with
/opt/karaf/bin/client \"la | grep islandora\"If all the bundles have a state of Active, then you're all good!
Some users have experienced timeouts when generating derivatives. We've added the ability to set the timeout length to your routes. In order to do so, edit each ca.islandora.* file in the deploy directory and replace
<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\nwith
<bean id=\"requestConfigConfigurer\" class=\"ca.islandora.alpaca.connector.derivative.RequestConfigConfigurer\">\n <property name=\"connectionRequestTimeoutMs\" value=\"10000\"/>\n <property name=\"connectTimeoutMs\" value=\"10000\"/>\n <property name=\"socketTimeoutMs\" value=\"10000\"/>\n</bean>\n\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\">\n <property name=\"httpClientConfigurer\" ref=\"requestConfigConfigurer\"/>\n</bean>\n\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\">\n <property name=\"httpClientConfigurer\" ref=\"requestConfigConfigurer\"/>\n</bean>\nYou can set the timeout values to whatever you like, we're using 10000 in this example.
"},{"location":"release_notes/8.x-2.0/#update-drupal-core-and-modules","title":"Update Drupal core and modules","text":"To get the latest Drupal code, edit your composer.json file and change the islandora/* packages versions' to the latest (2.0.0 for most). Then run the following command to update them, your dependencies, and Drupal core:
sudo php -d memory_limit=-1 `which composer` update --with-all-dependencies\nAfterwards, you need to run drush updb or visit /update.php in your browser to trigger the database updates. This includes important changes, like stripping out configuration we no longer use as well as a fix to support large media (>2.2 GB). The database updates may take a while depending on how many media you have.
We removed the Gemini microservice and pushed its code into Crayfish-Commons so that it can be used by the other microservices and Drupal modules. Once you've updated your Drupal installation, Gemini will no longer be utilized. It will still, however, need to be removed. It is recommended to test this method in a staging/development environment before running in your production environment.
2.0.0 version of the islandora module. If you followed the steps above to update Drupal core and its modules, you'll have this.Import two new Islandora views
i. This can be done by coping the views into a new directory and using drush such as
mkdir /var/www/html/drupal/config/1time\ncp /var/www/html/drupal/web/modules/contrib/islandora/modules/islandora_core_feature/config/install/views.view.all_taxonomy_terms.yml /var/www/html/drupal/config/1time/\ncp /var/www/html/drupal/web/modules/contrib/islandora/modules/islandora_core_feature/config/install/views.view.non_fedora_files.yml /var/www/html/drupal/config/1time/\ndrush config:import --partial --source /var/www/html/drupal/config/1time\n/admin/config/development/configuration/single/import 4. Update Crayfish to 2.0.0 1. cd /var/www/html/Crayfish 2. git fetch origin --tags 3. git checkout 2.0.0 6. Run composer install for Milliner such as cd /var/www/html/Crayfish/Milliner && composer install 7. Run composer install for Recast such as cd /var/www/html/Crayfish/Recast && composer install 8. Update the Recast config 1. cd /var/www/html/Crayfish/Recast/cfg 2. open config.yaml 3. Remove gemini_base_url 4. Update base_url in fedora_resource section. It should contain a value something like http://localhost:8080/fcrepo/rest 8. Clear your cache in Drupal using Drush (cd /var/www/html/drupal && drush cr) or the UI. 9. Restart apache sudo systemctl restart apache2 9. Test to make sure you can view existing objects in Fedora by going to the Drupal UI and clicking one of the Fedora URI links 10. Create a new object and make sure it persists to Fedora and the Fedora URI link resolves correctly. Make sure that no new entries get added to Gemini. 11. Once you've confirmed everything is working as expected, you can then remove Gemini like 1. rm /var/www/html/Crayfish/Gemini 2. Remove the gemini database and the associated mysql user 3. Remove the vhost file for gemini: sudo rm /etc/apache2/conf-enabled/Gemini.conf 4. Restart apache: sudo systemctl restart apache2 You can now add captions to A/V content! If you are starting with a new installation, there's nothing to do, but if you have an existing install, here's how to configure Drupal to display captions/
2.0.0 version of islandora and islandora_defaults. If you followed the steps above to update Drupal core and its modules, you'll have this.islandora_defaults using the Features UI (recommended) or by copying them into a temporary directory and using drush to do a partial config import.drush crDrupal requires the use of a _format query parameter to get alternate representations of a node/media.
By default, Islandora deploys with the jsonld module and the Milliner microservice. These two components are configured to strip this _format query parameter off of the end of URIs.
This means that when your content is indexed in Fedora, the triplestore, etc... it's URI will be something like http://localhost:8000/node/1 and not http://localhost:8000/node/1?_format=jsonld.
If you are using a very early version of Islandora \"8\" (pre-release), then you may have URIs with _format=jsonld at the end of them.
If you update to newer code, you will need to ensure that your site is configured to add ?_format=jsonld back to the URLs if you want to maintain consistency.
If you don't do this, you can end up with two copies of your objects in your Fedora repository (one with and one without ?_format=jsonld). You will also have two sets of triples in your triplestore.
To turn the ?_format parameter back on:
admin/config/search/jsonld and confirm the \"Remove jsonld parameter from @ids\" checkbox is unchecked.strip_format_jsonld: false to your Milliner config. If you deployed using the default Islandora-playbook this file would be located at /var/www/html/Crayfish/Milliner/cfg/config.yaml.If you are using Islandora-playbook and are provisioning new environments for your older Islandora, you'll want to lock down the variables in your inventory that control this config.
crayfish_milliner_strip_format_jsonld: truewebserver_app_jsonld_remove_format: 1Alpaca is event-driven middleware based on Apache Camel for Islandora
Currently, Alpaca ships with four event-driven components
This service receives requests from Drupal when it wants to create derivatives and passes that request along to a microservice in Crayfish. When it receives the derivative file back from the microservice, it passes the file back to Drupal.
"},{"location":"technical-documentation/alpaca-tips/#islandora-http-client","title":"islandora-http-client","text":"This service overrides the default http client with Islandora specific configuration.
"},{"location":"technical-documentation/alpaca-tips/#islandora-indexing-fcrepo","title":"islandora-indexing-fcrepo","text":"This service receives requests from Drupal in response to write operations on entities. These requests are passed along to Milliner microservice in Crayfish to convert Drupal entities into Fedora resources and communicate with Fedora (via Chullo).
"},{"location":"technical-documentation/alpaca-tips/#islandora-indexing-triplestore","title":"islandora-indexing-triplestore","text":"This service receives requests from Drupal on indexing and deleting in order to persist/delete content in the triplestore.
"},{"location":"technical-documentation/alpaca-tips/#steps-for-developing-with-alpaca","title":"Steps for developing with Alpaca","text":"Alpaca now runs as a single executable jar which can enable none, some or all of the available services.
To develop your own module, start by cloning the Alpaca code base.
Then create a new directory (for example my-new-module) along side the islandora-indexing-fcrepo, islandora-indexing-triplestore directories
Add your new directory to the settings.gradle file, following the pattern of the others.
include ':islandora-support'\ninclude ':islandora-indexing-triplestore'\ninclude ':islandora-indexing-fcrepo'\ninclude ':islandora-connector-derivative'\ninclude ':islandora-http-client'\ninclude ':islandora-alpaca-app'\n+ include ':my-new-module'\nproject(':islandora-alpaca-app').setProjectDir(\"$rootDir/islandora-alpaca-app\" as File)\nproject(':islandora-support').setProjectDir(\"$rootDir/islandora-support\" as File)\nproject(':islandora-indexing-triplestore').setProjectDir(\"$rootDir/islandora-indexing-triplestore\" as File)\nproject(':islandora-indexing-fcrepo').setProjectDir(\"$rootDir/islandora-indexing-fcrepo\" as File)\nproject(':islandora-connector-derivative').setProjectDir(\"$rootDir/islandora-connector-derivative\" as File)\nproject(':islandora-http-client').setProjectDir(\"$rootDir/islandora-http-client\" as File)\n+ project(':my-new-module').setProjectDir(\"$rootDir/my-new-module\" as File)\nYou can explore the islandora-indexing-fcrepo module to see the pattern to develop your own module.
This module contains three classes.
You can ignore the CommonProcessor class, that is just some processing that is split out for reusability.
The first class is the FcrepoIndexer, this class extends the Apache Camel RouteBuilder and requires a configure method which defines the processing elements of your workflow. This is the Camel \"route\".
The second class is the FcrepoIndexerOptions, this class extends the Alpaca PropertyConfig base class which gets common configuration parameters into your module. It also contains any custom configuration parameters needed for your route.
Lastly it uses the @Conditional(FcrepoIndexerOptions.FcrepoIndexerEnabled.class) to define when this module is enabled.
FcrepoIndexerOptions.FcrepoIndexerEnabled.class refers to the static inner class.
This class is inside of FcrepoIndexerOptions and works like this:
[1] static class FcrepoIndexerEnabled extends ConditionOnPropertyTrue { \n[2] FcrepoIndexerEnabled() {\n[3] super(FcrepoIndexerOptions.FCREPO_INDEXER_ENABLED, false);\n[4] }\n[5] }\nLine 2 is the constructor for this static class
Line 3 passes to the parent constructor two things.
So in this case we check for the property fcrepo.indexer.enabled and if we don't find it, we pass false. So this module is assumed to be \"off\" unless the property fcrepo.indexer.enabled=true is located.
The last thing is to add your new module to the islandora-alpaca-app build.gradle file as a dependencies, like the existing modules. i.e.
dependencies {\n implementation \"info.picocli:picocli:${versions.picocli}\"\n implementation \"org.apache.camel:camel-spring-javaconfig:${versions.camel}\"\n implementation \"org.slf4j:slf4j-api:${versions.slf4j}\"\n implementation \"org.springframework:spring-context:${versions.spring}\"\n implementation project(':islandora-support')\n implementation project(':islandora-connector-derivative')\n implementation project(':islandora-indexing-fcrepo')\n implementation project(':islandora-indexing-triplestore')\n+ implementation project(':my-new-module')\n\n runtimeOnly \"ch.qos.logback:logback-classic:${versions.logback}\"\n\n}\nFinally from the top-level directory of Alpaca execute
./gradlew clean build shadowJar\nThis tells Gradle to clean the modules, then build the modules and finally create a single jar with all needed code (the shadow jar).
The final executable jar is:
<alpaca directory>/islandora-alpaca-app/build/libs/islandora-alpaca-<version>-all.jar\nBefore opening a pull request, you should check your code style. If you are using the Vagrant, you can run phpcs within the Drupal installation directory (on the Vagrant, that is /var/www/html/drupal) or from within the web directory (/var/www/html/drupal/web) as follows:
./vendor/bin/phpcs --standard=./vendor/drupal/coder/coder_sniffer/Drupal modules/contrib/my_module, where modules/contrib/my_module is the relative or full path to the PHP file you want to check.web directory: ../vendor/bin/phpcs --standard=../vendor/drupal/coder/coder_sniffer/Drupal yourfile, where yourfile is the relative or full path to the PHP file you want to check.In both cases:
web: --standard=../vendor/drupal/coder/coder_sniffer/DrupalIslandora runs phpcs in its Github continuous integration environment, and there, it specifies which files to ignore and which files to check. It is a good idea for developers to specify the same options when running phpcs locally, prior to opening a pull request. For example (running phpcs from the within Drupal's web directory), you should use the following --ignore and --extensions options:
../vendor/bin/phpcs --standard=../vendor/drupal/coder/coder_sniffer/Drupal --ignore=*.md --extensions=php,module,inc,install,test,profile,theme,css,info modules/contrib/my_module
Diagram prepared by Bethany Seeger based on work done by Gavin Morris
"},{"location":"technical-documentation/diagram/#components","title":"Components","text":""},{"location":"technical-documentation/diagram/#islandora","title":"Islandora","text":"The following components are microservices developed and maintained by the Islandora community. They are bundled under Islandora Crayfish:
The following components are deployed with Islandora, but are developed and maintained by other open source projects:
This documentation is built using MkDocs, a static site generator that is geared towards building project documentation. The documentation is created in the Markdown format, and it all resides in the docs directory in the repository. The organization of the documentation is controlled by the mkdocs.yml in the root of the repository.
Video version available
Some of the material in this tutorial is presented in our video, How to Build Documentation.
"},{"location":"technical-documentation/docs-build/#prerequisites","title":"Prerequisites","text":"You will need to have mkdocs software installed locally, as well as a required plugin and the MkDocs Material theme. Below we will show you how to install mkdocs using the Python language's pip tool. For more details on installing and using MkDocs visit the MkDocs installation guide.
Open a terminal window.
Install mkdocs:
Windows / Linux:
sudo -H pip install mkdocs
macOS:
pip3 install mkdocs
Install plugin to enable display of the last revision date:
Windows / Linux:
sudo -H pip install mkdocs-git-revision-date-localized-plugin
macOS:
pip3 install mkdocs-git-revision-date-localized-plugin
Install plugin to enable redirects:
Windows / Linux:
sudo -H pip install mkdocs-redirects
macOS:
pip3 install mkdocs-redirects
Install Material theme:
Windows / Linux:
sudo -H pip install mkdocs-material
macOS:
pip3 install mkdocs-material
Make sure you have all the submodules:
git submodule update --init --recursive
Documentation is build by running to the following command in the root of the repository:
mkdocs build --clean
This command will create a static site folder in the root of the repository.
You can preview any changes you have made to the documentation by running the following command:
mkdocs serve
And then visiting http://localhost:8111 in your browser.
To deploy documentation to GitHub Pages, issue the following command:
mkdocs gh-deploy --clean
To stop the mkdocs serve command just type the key combination \"Control-c\".
Islandora can make use of the majority of Drupal modules and themes. Common use cases have been documented in the Islandora Cookbook. There are several ways to download and install Drupal modules. Please refer to this guide on Drupal.org.
Composer is the recommended method to install and update drupal modules and themes in Islandora.
$ composer require \"<vendor>/<package>:<version>\"\n# Example\n$ composer require \"islandora/jsonld:^2\"\nIn the Islandora playbook, you can add a Drupal module's or theme's machine name to the drupal_composer_dependencies variable here. To enable the Drupal module or theme, add the module machine name to the drupal_enable_modules variable as well.
For modules that require additional steps, additional tasks may need to be added to the Ansible playbook. Re-provisioning your instance via Ansible will install the module.
"},{"location":"technical-documentation/install-enable-drupal-modules/#video-walkthroughs-modules-installing-modules-with-composer","title":"Video Walkthroughs: Modules & Installing modules with Composer","text":"Click the image below to open the introduction to Modules video tutorial on the Islandora Youtube channel.
Click the image below to open the Installing modules with Composer video tutorial on the Islandora Youtube channel.
See more videos from the Drupal 101 series here.
"},{"location":"technical-documentation/migrate-7x/","title":"Islandora 7","text":""},{"location":"technical-documentation/migrate-7x/#migrating-from-islandora-legacy-to-islandora-youtube-video","title":"Migrating from Islandora Legacy to Islandora (YouTube Video)","text":"Nov 21, 2019
!! note \"Migration from Islandora Legacy\" For tools to migrate into Islandora from an existing Islandora Legacy instance, please see migrate_7x_claw.
"},{"location":"technical-documentation/migrate-7x/#project-planning-for-migration","title":"Project Planning for Migration","text":"Presented by Mark Jordan and Janice Banser, this session will outline strategies for understanding your data and planning for a move to Islandora.
"},{"location":"technical-documentation/migrate-csv/","title":"CSV","text":""},{"location":"technical-documentation/migrate-csv/#table-of-contents","title":"Table of Contents","text":"This tutorial introduces you to the Drupal Migrate tools available to create Islandora content. Whether you will eventually use CSVs or other sources (such as XML or directly from a Legacy Islandora) this tutorial should be useful as it covers the basics and mechanics of migration.
This tutorial uses the configurations and code available in the migrate_islandora_csv module which, when enabled, will create three example migrations ready for you to use with the Migrate API. Each migration comes from one of the files in the config/install folder. We'll walk through them in detail below. The module is also uses Features which allows you to make changes to the configuration files and sync those changes into your Drupal site.
A note on using Features
This tutorial (and Islandora in general) makes heavy use of Features, which is an easy way to ship and install Drupal configuration. However, after enabling a Feature module, the code in that module's directory is no longer \"live\", as the configuration now resides in the Drupal database. If you change code in the YAML files, it will not take effect until you re-import the Feature. There is a walkthrough in the \"Configuration\" section of the Migrate 7.x to Islandora tutorial.
Sample CSV and images are also included in the module as a convenience so they are easily available on the Drupal server running the migration. (This is not the recommended method for making files available to Drupal in a real migration.)
The module also contains a Migrate process plugin that transforms strings into associative arrays. This is useful for populating multiple Contributor (field_linked_agent) fields. (See \"Typed Relation\" for more information on the Contributor field's type.) It will be available when this module is enabled, and the node migration uses it. It was written generically and will hopefully become part of Migrate Plus, but for now it is here.
When you are ready to create your actual migrations, the contents of this repository can function as a template for you to create the YAML files defining your own migrations.
"},{"location":"technical-documentation/migrate-csv/#introduction","title":"Introduction","text":"Why CSV? CSV files (whether separated by commas, tabs, or other delimiters) are easy to understand and work with, and there's good tooling available for using them with Drupal's Migrate API. The Drupal contrib module migrate_source_csv provides a source plugin that reads from a CSV file, and migrate_plus provides more tools and flexibility for creating migrations, including the ability to create customized migrations using YAML and package them up as Features.
In this tutorial, we'll be inspecting each migration file in detail before running it. You'll start out by migrating the images themselves first, and then you'll create various Drupal entities to describe the files from the metadata in the CSV. It's not as scary as it sounds (especially since this module contains the data we'll be using in a data directory), but you will need a few things before beginning:
vagrant ssh into the box, navigate into Drupal, and use git and drush, etc... If you can copy/paste into a terminal, you'll survive.The configuration referred to in this tutorial comes from the Islandora Starter Site, which defines the content model and metadata fields that we'll be migrating into. You're not required to use the Islandora Starter Site for your repository, but for the purposes of demonstration, it saves you a lot of user interface administrivia so you can focus just on the learning how to migrate. By the time you are done with this exercise, you'll be able to easily apply your knowledge to migrate using any custom metadata profile you can build using Drupal.
"},{"location":"technical-documentation/migrate-csv/#overview","title":"Overview","text":"The Migrate API is the main way to ingest batches of data into Drupal (and because Islandora is Drupal, into Islandora). The Migrate module only provides the framework, it's up to you to create the rules that take data from a source, through a process (i.e. a mapping) to a destination. A set of these rules is called a \"migration\". It has to be set up (as a Configuration Entity, either by importing a YAML file or by installing a Feature) and then it has to be run.
Once a migration has been run, it will have created (or updated) a bunch of Drupal entities of one type - whether that's taxonomy terms, nodes, files, etc. Since an Object in Islandora is made up of several different Drupal entities that refer to each other, it's going to take multiple migrations to create an Islandora object, and it's important to perform these migrations in a sensible order.
A basic Islandora object is at minimum: - a file, which holds the actual binary contents of an item - a node, which holds the descriptive metadata for an item and makes it discoverable - a media, which holds technical metadata and references the file and the node, linking the two together.
Therefore, each row in your CSV must contain enough information to create these.
Using multiple sources for object components
While this tutorial uses a single CSV to describe all the parts of an islandora object, a site can use multiple sources (CSV or otherwise) to migrate the various parts. The only requirements then is to have keys in place matching the parts together. For example, a 'files.csv' can use paths that serve as file identifiers and 'metadata.csv' can use an object identifier. As long as the source for the media migration has the key values for the file and metadata migrations (the file paths and object identifiers, respectively, in this example), we can create the necessary links between the object's components. This is very useful when you need to migrate multiple media/file pairs associated with a single node, such as pre-processed derivatives that need to be migrated.
Buried in your descriptive metadata are often references to other things which aren't repository items themselves, but records still need to be kept for them. Authors, publishers, universities, places, etc... can be modeled as Drupal Entities, so that they can be referenced by other Entities. So there's the potential to have a lot of different entity types described in a single row in a CSV.
In this tutorial, we will use the three migrations defined in this module to migrate into Drupal entities. The entities we are migrating content into have already been defined by the Islandora Starter Site:
We can do this because subjects, persons, and corporate bodies are (in this example) represented by simple 'name' strings. We create them on the fly, as if we were adding new tags to a tag vocabulary. If we wanted to model subjects, people, or corporate bodies as entities with multiple fields (first name, last name, date, subheadings, URI, etc.) then we would need up to six migrations.
Migrations follow the Extract-Transform-Load pattern. You extract the information from a source, process the data to transform it into the format you need, and load it into the destination system (i.e. Drupal). Migrations are stored in Drupal as configuration, which means they can be represented in YAML, transferred to and from different sites, and are compatible with Drupal's configuration synchronization tools. And the structure of each YAML file is arranged to follow the Extract-Transform-Load pattern.
To perform the migrations, we'll be using drush. We will be able to run each of the file, node, and media migrations separately or all at once in a group. We will also learn how to roll back a migration in case it didn't go as planned.
The sample migrations here will give us a chance to show off some techniques for working with multi-valued fields, entity reference fields, and complex field types like controlled_access_terms's typed_relation field. We'll also see how the migrate framework can help de-duplicate, and at the same time, linked data-ize :tm: your data by looking up previously migrated entities.
So hold on to your hats. First, let's get this module onto your Islandora instance.
"},{"location":"technical-documentation/migrate-csv/#installation","title":"Installation","text":"From your islandora-playbook directory, issue the following commands to enable this module: - vagrant ssh to open a shell in your Islandora instance. - cd /var/www/html/drupal/web/modules/contrib to get to your modules directory. - git clone https://github.com/Islandora/migrate_islandora_csv to clone down the repository from GitHub. - drush en -y migrate_islandora_csv to enable the module, installing the migrations as configuration.
Optionally, flush the cache (drush cr), so the migrations become visible in the GUI at Manage > Structure > Migrations > migrate_islandora_csv (http://localhost:8000/admin/structure/migrate/manage/migrate_islandora_csv/migrations)
Now lets go migrate some files.
Caution
As you saw, you can git clone into the modules directory, but if you're installing a custom module that's intended to stay installed for the long term (unlike a migration feature, which you should probably uninstall and delete when you're done with it) then you may want to check with your devops folks and use Composer instead. However, using Git directly allows you to be more flexible when iterating and testing.
To migrate files (i.e. just the raw binaries) from a CSV, you need: - a column in the CSV containing paths to the files you wish to ingest, and - the files need to be accessible from the server that's running Drupal so that the Migrate framework can find them.
This tutorial assumes you're working with the sample images provided in the module, which will be located at /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/images. When you're migrating for real, the files will have to be uploaded or otherwise made accessible to the server before this point.
Open up the CSV file at data/migration.csv and you'll see a file column containing paths to the sample images. You can use your favorite shell-based utility to open it at /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv, or browse to it on GitHub, or just look at it pasted below as this tutorial does not require you to edit the files.
Open up the \"files\" migration at config/install/migrate_plus.migration.file.yml. You'll see the following migration config:
id: file\nlabel: Import Image Files\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: '/var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv'\n delimiter: ','\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique key (\"id\") per row in the CSV. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # Islandora uses Flysystem and stream wrappers to work with files. What we're really saying here is\n # to put these files in Fedora in a 'csv_migration' folder. It doesn't matter if the directory\n # doesn't exist yet, it will get created for you automatically.\n destination_dir: 'fedora://csv_migration'\n\nprocess:\n\n ##\n # The following two fields are temporary, and just used to generate a destination for the file.\n ##\n\n # Hack the file name out of the full path provided in the 'file' column.\n filename:\n -\n plugin: callback\n callable: pathinfo\n source: file\n -\n plugin: extract\n index:\n - basename\n\n # Construct the destination URI using the file name.\n destination:\n plugin: concat\n delimiter: /\n source:\n - constants/destination_dir\n - '@filename'\n\n ##\n # Here's where we copy the file over and set the URI of the file entity.\n ##\n uri:\n plugin: file_copy\n source:\n - file # The source column in the CSV\n - '@destination' # The destination entry from above\n\ndestination:\n # These are Drupal 'image' entities we're making, not just plain 'file' entities.\n plugin: 'entity:file'\n type: image\nIt seems like a lot to take in at first, but there's a pattern to Drupal migrations. They always contain core identification information and three key sections: source, process, and destination. And these sections correspond exactly to Extract, Transform, and Load.
The first section of a migration contains metadata for Drupal about the migration itself. The id parameter is the machine name of this migration, and must not conflict with existing migrations. Note: use alphanumeric characters and underscores. Hyphens (-) will cause the migration to fail. The label is a human-readable string to identify this migration, and the migration_group is where this migration will be grouped in the GUI.
The source section configures a Drupal source plugin that will extract the data. A source plugin provides \"rows\" of data to processing plugins so that they can be worked on. In this case, we're using the csv source plugin, which very literally uses rows, however you can have source plugins that work with other data formats like XML and JSON. Look at the config from this section.
source:\n plugin: csv\n path: '/var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv'\n delimiter: ','\n header_row_count: 1\n ids:\n - file\n constants:\n destination_dir: 'fedora://csv_migration'\nWe're going to dive into the details of this step below, but in summary: the process section is where we extract the desired bits from that row, transform them as desired, and populate them into an associative array. This section is a series of named steps, that call one or more process plugins. These plugins are executed in sequence, with the results getting passed from one to the next, forming a pipeline. By the end of the step, you have transformed some element of data (perhaps through text manipulation, concatenation, etc...) into the form that Drupal is expecting. The resulting value gets associated with the name of the step.
If the name of a step is the same as a field or property name on the target entity, the migrated entity will have that value for that field or property. This is how you can apply metadata from the CSV to an entity. If the step name is not the name of a field or property on the target entity, the migrate framework assumes it's a temporary value you're using as part of more complex logic. It won't wind up on the entity when the migration is done, but it will be available for you to use within other process plugins. You can always spot when a temporary value is being used by the fact that it's prefixed with an @ and surrounded by quotes. You can also pass constants into process plugins, which are prefixed with constants/.
The destination section contains the configuration that describes what gets loaded into Drupal.
destination:\n plugin: 'entity:file'\n type: image\nIn the process section of the migration, we're copying the images over into a Drupal file system and setting the uri property on the corresponding File entity.
uri:\n plugin: file_copy\n source:\n - file\n - '@destination'\nfile_copy process plugin. But first, we have to know where a file is located and where it should be copied to. We know where the file resides, we have that in the CSV's file column. But we're going to have to do some string manipulation in order to generate the new location where we want the file copied. We're trying to convert something like /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/images/Free Smells.jpg to fedora://csv_migration/Free Smells.jpg. The URI we're constructing is a stream wrapper of the form scheme://path/to/file. Islandora uses Flysystem, which allows for integration with many different types of filesystems, both remote and local. With Flysystem, the scheme part of the URI is the name of a filesystem. By default, Fedora is exposed using the scheme fedora://. So by setting uri to fedora://csv_migration/Free Smells.jpg, we're saying \"put Free Smells.jpg in the csv_migration directory in Fedora.\"
Now, to perform this string manipulation in PHP, we'd do something like
$info = pathinfo($filepath);\n$filename = $info['basename'];\n$destination = \"fedora://csv_migration/\" . $filename;\nWe will mimic this exactly in the process section of our migration config. Just like we declare variables and call functions with PHP code, we can make entries in the process section to store the output of Drupal process plugins. We'll build up a destination \"variable\" using a filename \"variable\" and pass it into the file_copy process plugin.
To start, we'll get the filename using two process plugins, which do the same steps as the first two lines of the PHP above:
filename:\n -\n plugin: callback\n callable: pathinfo\n source: file\n -\n plugin: extract\n index:\n - basename\ncallback, lets you execute any PHP function that takes a single input and returns an output. It's not as flexible as making your own custom process plugin, but it's still pretty useful in a lot of situations. Here we're using it to call pathinfo(), telling it to use the file column in the CSV as input. We pass the resulting array from pathinfo() to the extract process plugin, which pulls data out of arrays using the keys you provide it under index. Now that we have the file name, we have to prepend it with fedora://csv_migration/ to make the destination URI. In our PHP code above, we used . to concatenate the strings. In the migration framework, we use the concat process plugin. You provide it with two or more strings to concatenate, as well as a delimiter.
destination:\n plugin: concat\n delimiter: /\n source:\n - constants/destination_dir\n - '@filename'\nIn our PHP code, we concatenated the $filename variable with a string literal. In our process plugin, we can provide the variable, e.g. the output of the filename process step, by prefixing it with an @. We can't, however, pass in fedora://csv_migration directly as a string. At first glance, you might think something like this would work, but it totally doesn't:
# Can't do this. Won't work at all.\n destination:\n plugin: concat\n delimiter: /\n source:\n - 'fedora://csv_migration'\n - '@filename'\nsource values as names of columns from the CSV or names of other process steps. Even if they're wrapped in quotes. It will never try to use the string directly as a value. To circumvent this, we declare a constant in the source section of the migration config. constants:\n destination_dir: 'fedora://csv_migration'\nThis constant can be referenced as constants/destination_dir and passed into the concat process plugin as a source.
There are a lot more process plugins available through the (core) Migrate and Migrate Plus modules, and they are documented on Drupal.org.
"},{"location":"technical-documentation/migrate-csv/#running-the-file-migration","title":"Running the File Migration","text":"Migrations can be executed via drush using the migrate:import command. You specify which migration to run by using the id defined in its YAML. You also need to set parameters to tell Drush who you are and what your site's URL is. Failing to do so will result in derivatives not being generated and malformed/improper RDF. So don't forget them! To run the file migration from the command line, make sure you're within /var/www/html/drupal/web (or any subdirectory) and enter
drush -y --userid=1 --uri=localhost:8000 migrate:import file\n--update flag. drush -y --userid=1 --uri=localhost:8000 migrate:import file --update\nmigration_group in their configuration. You can execute an entire group of migrations using the --group flag. For example, to run the entire group defined in this module drush -y --userid=1 --uri=localhost:8000 migrate:import --group migrate_islandora_csv\nmigrate:rollback command to delete all migrated entities. Like migrate:import, it also respects the --group flag and --uri parameter. So to rollback everything we just did: drush -y --uri=localhost:8000 migrate:rollback --group migrate_islandora_csv\nmigrate:reset command to put a migration back to Idle. For example, with the file migration, use drush -y --uri=localhost:8000 migrate:reset file\nMake sure you've run (and not rolled back) the file migration. It should tell you that it successfully created 5 files. You can confirm its success by visiting http://localhost:8000/admin/content/files. You should see 5 images of neon signs in the list.
Those five images are nice, but we need something to hold their descriptive metadata and show them off. We use nodes in Drupal to do this, and that means we have another migration file to work with. Nestled in with our nodes' descriptive metadata, though, are more Drupal entities, and we're going to generate them on the fly while we're making nodes. While we're doing it, we'll see how to use pipe delimited strings for multiple values as well as how to handle typed_relation fields that are provided by controlled_access_terms. Open up /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/config/install/migrate_plus.migration.node.yml and check it out.
# Uninstall this config when the feature is uninstalled\ndependencies:\n enforced:\n module:\n - migrate_islandora_csv\n\nid: node\nlabel: Import Nodes from CSV\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: modules/contrib/migrate_islandora_csv/data/migration.csv\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique id per row in the csv. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # We're tagging our nodes as Images\n model: Image\n\n # Everything gets created as admin\n uid: 1\n\n# Set fields using values from the CSV\nprocess:\n title: title\n uid: constants/uid\n\n # We use the skip_on_empty plugin because\n # not every row in the CSV has subtitle filled\n # in.\n field_alternative_title:\n plugin: skip_on_empty\n source: subtitle\n method: process\n\n field_description: description\n\n # Dates are EDTF strings\n field_edtf_date: issued\n\n # Make the object an 'Image'\n field_model:\n plugin: entity_lookup\n source: constants/model\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_models\n\n # Split up our pipe-delimited string of\n # subjects, and generate terms for each.\n field_subject:\n -\n plugin: skip_on_empty\n source: subject\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: entity_generate\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: subject\n\n # If you have multiple values of a complex\n # field, iterate over them using sub_process.\n # But sub_process requires structured data\n # i.e. an associative array, not a string\n # or list of strings. To turn strings into\n # associative arrays, use the custom\n # process plugin str_to_assoc.\n\n # Extract a list of names from the column\n # called photographer, and transform it into\n # an array of associative arrays.\n photographers:\n -\n source: photographer\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\n\n # Iterate over the array of associative arrays.\n # We create the taxonomy terms here so that we\n # can specify the bundle - other columns which\n # might feed into field_linked_agent may contain\n # corporate bodies or families. The resulting\n # array contains the resulting term id (tid)\n # under the key 'target_id'.\n # We also add a key-value pair\n # 'rel_type' => 'relators:pht'. Other columns\n # might use different relators.\n linked_agent_pht:\n plugin: sub_process\n source: '@photographers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n rel_type:\n plugin: default_value\n default_value: 'relators:pht'\n\n # Extract an array of names from the column\n # called provider\n providers:\n -\n source: provider\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\n # Generate/lookup taxonomy terms in the\n # corporate body vocab, and add the relator.\n linked_agent_prv:\n plugin: sub_process\n source: '@providers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: 'corporate_body'\n rel_type:\n plugin: default_value\n default_value: 'relators:prv'\n\n # Write to the linked agent field. In this case\n # we first want to merge the info from the\n # photographer and provider columns. Since we\n # already prepared our structured array using\n # the components of the typed_relation field as\n # keys ('target_id' and 'rel_type'), we can just\n # pass this array into field_linked_agent.\n field_linked_agent:\n plugin: merge\n source:\n - '@linked_agent_pht'\n - '@linked_agent_prv'\n\n# We're making nodes\ndestination:\n plugin: 'entity:node'\n default_bundle: islandora_object\nThe source section looks mostly the same other than some different constants we're defining - the string \"Image\" (no quotes needed) and the drupal ID of the user who will be assigned as the author. If values contained special characters such as colons, quotes would be needed.
If you look at the process section, you can see we're taking the title, description, and issued columns from the CSV and applying them directly to the migrated nodes without any manipulation.
title: title\n field_description: description\n field_edtf_date: issued\nsubtitle, we're passing it through the skip_on_empty process plugin because not every row in our CSV has a subtitle entry. It's very useful when you have spotty data, and you'll end up using it a lot. The method: process bit tells the migrate framework only skip that particular field if the value is empty, and not to skip the whole row. It's important, so don't forget it. The full YAML for setting field_alternative_title from subtitle looks like this: field_alternative_title:\n plugin: skip_on_empty\n source: subtitle\n method: process\nNow here's where things get interesting. We can look up other entities to populate entity reference fields. For example, all Repository Items have an entity reference field that holds a taxonomy term from the islandora_models vocabulary. All of our examples are images, so we'll look up the Image model in the vocabulary since it already exists (it gets made for you when you use islandora-playbook). We use the entity_lookup process plugin to do this.
field_model:\n plugin: entity_lookup\n source: constants/model\n entity_type: taxonomy_term\n # 'name' is the string value of the term, e.g. 'Original file', 'Thumbnail'.\n value_key: name\n bundle_key: vid\n bundle: islandora_models\nThe entity_lookup process plugin looks up an entity based on the configuration you give it. You use the entity_type, bundle_key, and bundle configurations to limit which entities you search through. entity_type is, as you'd suspect, the type of entity: node, media, file, taxonomy_term, etc... bundle_key tells the migrate framework which property holds the bundle of the entity, and bundle is the actual bundle id you want to restrict by. In this case we specify the vid (vocabulary id) has to be islandora_models - which is the machine name of the vocabulary we're interested in. In this plugin, source is the value to search for - in this case we're looking for the string \"Image\", which we've defined as a constant. And we're comparing it to the name field on each term by setting the value_key config.
This approach applies the same taxonomy term to all objects. If you want to assign a taxonomy term at the node level (that is, potentially a different term for each node) rather than to all the nodes being imported, you can use a configuration as illustrated next. In this example, we have a column in our CSV input file with the header 'model', which we define as the source of the field_model values:
field_model:\n plugin: entity_lookup\n # 'model' is the header of a field in our input CSV that contains the string value of the taxonomy term.\n source: model\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_models\nIf you're not sure that the entities you're looking up already exist, you can use the entity_generate plugin, which takes the same config, but will create a new entity if the lookup fails. We use this plugin to create subject taxonomy terms that we tag our nodes with. A node can have multiple subjects, so we've encoded them in the CSV as pipe delimited strings.
We can hack those apart easily enough. In PHP we'd do something like
$subjects = explode($string, '|');\n$terms = [];\nforeach ($subjects as $name) {\n $terms[] = \\Drupal::service('entity_type.manager')->getStorage('taxonomy_term')->create([\n ...\n 'vid' => 'subject',\n 'name' => $name,\n ...\n ]);\n}\n$node->set('field_subject', $terms);\nWith process plugins, that logic looks like
field_subject:\n -\n plugin: skip_on_empty\n source: subject\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: entity_generate\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: subject\nskip_on_empty process plugin, which we've already seen, followed by explode. The explode process plugin operates exactly like its PHP counterpart, taking an array and a delimiter as input. The combination of skip_on_empty and explode behave like a foreach loop on the explode results. If we have an empty string, nothing happens. If there's one or more pipe delimited subject names in the string, then entity_generate gets called for each name that's found. The entity_generate process plugin will try to look up a subject by name, and if that fails, it creates one using the name and saves a reference to it in the node. So entity_generate is actually smarter than our pseudo-code above, because it can be run over and over again and it won't duplicate entities. :champagne:"},{"location":"technical-documentation/migrate-csv/#complex-fields","title":"Complex Fields","text":"Some fields don't hold just a single type of value. In other words, not everything is just text, numbers, or references. Using Drupal 8's Typed Data API, fields can hold groups of named values with different types. Consider a field that holds an RGB color. You could set it with PHP like so:
$node->set('field_color', ['R' => 255, 'G' => 255, 'B' => 255]);\nYou could even have a multi-valued color field, and do something like this
$node->set('field_color', [\n ['R' => 0, 'G' => 0, 'B' => 0],\n ['R' => 255, 'G' => 255, 'B' => 255],\n]);\nIn the migrate framework, you have two options for handling these types of fields. You can build up the full array they're expecting, which is difficult and requires a custom process plugin. Or, if you only have one value going into a complex field, you can set each named component in the field with separate process pipelines.
In controlled_access_terms, we define a new field type of typed_relation, which is an entity reference coupled with a MARC relator. It expects an associative array that looks like this:
[ 'target_id' => 1, 'rel_type' => 'relators:ctb']\nThe target_id portion takes an entity id, and rel_type takes the predicate for the MARC relator we want to use to describe the relationship the entity has with the repository item. This example would reference taxonomy_term 1 and give it the relator for \"Contributor\".
If we have a single name to deal with, we can set those values in YAML, accessing field_linked_agent/target_id and field_linked_agent/rel_type independently.
field_linked_agent/target_id:\n plugin: entity_generate\n source: photographer\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n\n field_linked_agent/rel_type: constants/relator\nHere we're looking at the photographer column in the CSV, which contains the names of the photographers that captured these images. Since we know these are photographers, and not publishers or editors, we can bake in the relator constant we set to relators:pht in the source section of the migration. So all that's left to do is to set the taxonomy term's id via entity_generate. If the lookup succeeds, the id is returned. If it fails, a term is created and its id is returned. In the end, by using the / syntax to set properties on complex fields, everything gets wrapped up into that nice associative array structure for you automatically.
However, if you have multiple names, and/or multiple columns that contain values that will go into the same typed_relation field, you do need to build up a full array of structured data.
We start by extracting the values in the photographer column, and busting them into a list. (In this case, given the sample data, the lists will all have length 1). Then, we use a custom process plugin to make each value the value in an associative array (see example data below).
photographers:\n -\n source: photographer\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\nSo, if we started with a column containing
'Alice|Bob|Charlie'\nphotographers temporary variable would contain [\n ['name' => 'Alice'],\n ['name' => 'Bob'],\n ['name' => 'Charlie]\n]\nNext, we use the sub_process plugin. It takes an array of associative arrays (as seen above) and iterates over them. From within subprocess' process parameter, you can access only what's defined in that associative array. Here, when we do our entity_generate lookup, our source is name, the (only) key in that array.
linked_agent_pht:\n plugin: sub_process\n source: '@photographers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n rel_type:\n plugin: default_value\n default_value: 'relators:pht'\nsub_process, we cannot access the temporary variables or constants that we've created in the outer migration. This is why we use the default_value plugin when for the rel_type. It would have been simpler to define a constant as we did with 'Image', but we wouldn't be able to access it. The output of this pipeline is now formatted as the structured data expected by a typed_relation field: [\n ['target_id' => 42, 'rel_type' => 'relators:pht' ],\n ['target_id' => 43, 'rel_type' => 'relators:pht' ],\n ['target_id' => 44, 'rel_type' => 'relators:pht' ],\n]\nfield_linked_agent. We don't have to assign the sub-components of field_linked_agent here, because this is already the structured data it is looking for. field_linked_agent:\n plugin: merge\n source:\n - '@linked_agent_pht'\n - '@linked_agent_prv'\nClear as mud? Great. Now let's run that migration.
"},{"location":"technical-documentation/migrate-csv/#running-the-node-migration","title":"Running the node migration","text":"Like with the file migration, run drush -y --userid=1 --uri=http://localhost:8000 migrate:import node from anywhere within the Drupal installation directory will fire off the migration. Go to http://localhost:8000/admin/content and you should see five new nodes. Click on one, though, and you'll see it's just a stub with metadata. The CSV metadata is there, links to other entities like subjects and photographers are there, but there's no trace of the corresponding files. Here's where media entities come into play.
Media entities are Drupal's solution for fieldable files. Since you can't put fields on a file, what you can do is wrap the file with a Media entity. In addition to a reference to the file (binary), technical metadata and structural metadata for the file go on the Media entity (e.g. MIME type, file size, resolution). Media also have a few special fields that are required for Islandora, field_media_of and field_use, which denote what node owns the media and what role the media serves, respectively. Since the Media entity references both the file it wraps and the node that owns it, Media entities act as a bridge between files and nodes, tying them together. And to do this, we make use of one last process plugin, migration_lookup. Open up /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/config/install/migrate_plus.migration.media.yml and give it a look.
# Uninstall this config when the feature is uninstalled\ndependencies:\n enforced:\n module:\n - migrate_islandora_csv\n\nid: media\nlabel: Import Media from CSV\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: modules/contrib/migrate_islandora_csv/data/migration.csv\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique key per row in the csv. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # We're tagging our media as Original Files\n use: Original File\n\n # Everything gets created as admin\n uid: 1\n\nprocess:\n\n name: title\n uid: constants/uid\n\n # Make the media an 'Original File'\n field_media_use:\n plugin: entity_lookup\n source: constants/use\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_media_use\n\n # Lookup the migrated file in the file migration.\n field_media_image:\n plugin: migration_lookup\n source: file\n migration: file\n no_stub: true\n\n # Lookup the migrated node in the node migration\n field_media_of:\n plugin: migration_lookup\n source: file\n migration: node\n no_stub: true\n\ndestination:\n # These are 'image' media we're making.\n plugin: 'entity:media'\n default_bundle: image\n\nmigration_dependencies:\n required:\n - migrate_plus.migration.file\n - migrate_plus.migration.node\n optional: { }\nThe Breakdown
Compared to the other migrations, this one is very straightforward. There's no string or array manipulation in YAML, and at most there's only one process plugin per field. Title and user are set directly, with no processing required
name: title\n uid: constants/uid\nfield_media_use field is a tag that's used to denote the purpose of a file with regard to the node it belongs to. E.g. is this the original file? a lower quality derivative? thumbnail? etc... In many ways it bears a resemblance to DSID in Islandora 7.x. Like field_model with nodes, the vocabulary already exists in your Islandora install, so all you have to do is look it up with the entity_lookup plugin. # Make the media an 'Original File'\n field_media_use:\n plugin: entity_lookup\n source: constants/use\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_media_use\nfield_media_image and field_media_of fields are how the media binds a file to a node. You could use entity_lookup or entity_generate, but we've already migrated them and can very easily look them up by the id assigned to them during migration. But what's the benefit of doing so? The entity_lookup and entity_generate process plugins do the job fine, right? The main advantage of using migration_lookup and defining migrations whenever possible, is that migrated entities can be rolled back. If you were to hop into your console and execute
drush -y --uri=http://localhost:8000 migrate:rollback --group migrate_islandora_csv\nmigration_lookup to set entity reference fields."},{"location":"technical-documentation/migrate-csv/#running-the-media-migration","title":"Running the media migration","text":"Run drush -y --uri=http://localhost:8000 migrate:import media from anywhere within the Drupal installation directory. You should now be able to see the media files attached to the nodes you created earlier. At this point, you might want to create derivatives, such as thumbnails, using the appropriate Drupal actions on the main content admin window.
If you've made it all the way to the end here, then you've learned that you can migrate files and CSV metadata into Islandora using only YAML files. You've seen how to transform data with pipelines of processing plugins and can handle numeric, text, and entity reference fields. You can handle multiple values for fields, and even more complicated things like typed_relation fields. And as big as this walkthrough was, we're only scratching the surface of what can be done with the Migrate API.
There's certainly more you can do with Drupal 8's Migrate API. There's a plethora of source and processing plugins out there that can handle pretty much anything you throw at it. XML and JSON are fair game. You can also request sources using HTTP, so you can always point it at an existing systems REST API and go from there. If you can't make the Migrate API's existing workflow make the necessary changes to your data, you can expand its capabilities by writing your own process plugins. Reading the Drupal.org documentation on the Migrate API would be a good place to start.
But really the best thing to do is try and get your data into Islandora! We intend to create a boilerplate branch of this repository that will allow you to clone down a migration template, ready for you to customize to fit your data. And as you assemble it into CSV format, keep in mind that if you have more than just names for things like subjects and authors, that you can always make more CSVs. Think of it like maintaining tables in an SQL database. Each CSV has unique keys, so you can lookup/join entities between CSVs using those keys. And you can still pipe-delimit the keys like we did in our example to handle multi-valued fields.
In some repositories, these CSVs can be used to make bulk updates to metadata. Just make your changes to maintain the CSVs, then run the migration(s) again with the --update flag. This will not always be efficient, as you'll update every entity, even if it didn't change. But, by breaking down your CSVs per collection or object type, you could keep them small enough to use this process for a small repository.
There is also a tool for migrating directly from a Legacy Islandora to Islandora (migrate_7x_claw), using Solr, XPaths, and Fedora calls to pull files and objects directly into Islandora. It may be worth checking out, and/or using in conjunction with a CSV migration.
"},{"location":"technical-documentation/migration-islandora-workbench/","title":"Islandora Workbench","text":""},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench","title":"Islandora Workbench","text":"Islandora Workbench provides a command-line solution to create, update, and delete Islandora content from CSV data. The Islandora Workbench repository can be found on Github. Full details and documentation for Islandora Workbench are maintained separately (Islandora Workbench Documentation).
"},{"location":"technical-documentation/migration-islandora-workbench/#overview","title":"Overview","text":"Islandora Workbench is tailored towards end users with less technical knowledge or limited server access.
Workbench provides an alternative to Drupal's Migrate framework, as it does not need to be run on the Drupal server. Islandora Workbench uses Islandora's REST API and offers cross-platform support (Windows, Mac, Linux) to run on your computer, using your provided CSVs and files.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench-highlights","title":"Islandora Workbench highlights","text":"Islandora Workbench:
Using Islandora Workbench, you can assign both existing and new taxonomy terms to nodes. Within the CSVs you provide, the values of the taxonomy field/columns can:
If you need to create terms with multiple fields (such as an External URI) or which use term hierarchy, you can create terms in a separate task.
"},{"location":"technical-documentation/migration-islandora-workbench/#other-field-types","title":"Other Field Types","text":"There are multiple ways to create paged content with Islandora Workbench. More information on each option is available here. You may:
Click the video previews shown below to open the corresponding video in Youtube.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandoracon-2022-migration-strategies","title":"IslandoraCon 2022: Migration Strategies","text":"This video (August 2022) is an overview of how to harvest data from an existing Islandora 7 site using Workbench, how to sanitize and prep that data with custom Python tools, and how to pull that data into a new Islandora 2 site.
"},{"location":"technical-documentation/migration-islandora-workbench/#summer-of-islandora-workbench-introduction-to-islandora-workbench","title":"Summer of Islandora Workbench: Introduction to Islandora Workbench","text":"This video (July 29, 2021) is an introduction of Islandora Workbench.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench-demo","title":"Islandora Workbench Demo","text":"This video (Dec 17, 2020) is a demo of Islandora Workbench.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-online-islandora-migration-tools","title":"Islandora Online: Islandora Migration Tools","text":"This video (Aug 10, 2020) provides an overview of the Islandora Workbench and the two other options available to migrate data into an Islandora installation.
"},{"location":"technical-documentation/migration-migrate-api/","title":"Migrate API","text":""},{"location":"technical-documentation/migration-migrate-api/#migrate-api","title":"Migrate API","text":"Uses the Drupal Migrate API, which \"provides services for migrating data from a source system\" to Drupal 8, 9, or 10.
The \"source system\" can be almost anything:
Why use the Migrate API?
drush mim node --updateCheck out this video that demonstrates the Drupal Migrate API migration process: Islandora Webinar: Migrating from Islandora Legacy to Islandora (Nov 21, 2019)
"},{"location":"technical-documentation/migration-overview/","title":"Migration Overview","text":""},{"location":"technical-documentation/migration-overview/#migration-and-batch-loading","title":"Migration and Batch Loading","text":"This video (Aug 10, 2020) provides an overview of the various options available to migrate data into an Islandora installation.
The three main migration options are:
Why use the rest API?
Visit the REST API migration documentation section for more details.
"},{"location":"technical-documentation/migration-overview/#migrate-api","title":"Migrate API","text":"Why use the Migrate API?
drush mim node --updateTwo tools that use the Migrate API are migrate_islandora_csv and migrate_7x_claw.
Visit the Migrate API migration documentation section for more details.
"},{"location":"technical-documentation/migration-overview/#islandora-workbench","title":"Islandora Workbench","text":"Why use the Migrate API?
Visit the Islandora Workbench migration documentation section for more details.
"},{"location":"technical-documentation/migration-rest-api/","title":"REST API","text":""},{"location":"technical-documentation/migration-rest-api/#rest-api","title":"REST API","text":"Why use the REST API?
BONUS: It\u2019s just Drupal\u2019s REST API
"},{"location":"technical-documentation/migration-rest-api/#islandora-only-provides-two-additional-api-endpoints","title":"Islandora only provides two additional API endpoints","text":"Just be aware, you are writing everything yourself! (In other words you are making all of the migration decisions yourself.)
"},{"location":"technical-documentation/migration-rest-api/#videos","title":"Videos","text":"This video (Aug 10, 2020) provides an overview of the REST API and the two other options available to migrate data into an Islandora installation.
"},{"location":"technical-documentation/ppa-documentation/","title":"Updating adeb and adding it to LYRASIS PPA","text":""},{"location":"technical-documentation/ppa-documentation/#this-page-is-here-for-archival-purposes","title":"This page is here for archival purposes.","text":"If you are running Ubuntu 20.04 or higher, installing a specially-compiled imagemagick is no longer necessary. We are leaving this page available in case in the future we have a need to package our own deb and add it to the Lyrasis PPA.
Ubuntu removed JPEG 2000 support from ImageMagick since Vivid Vervet's 8:6.8.9.9-5 version. The PPA that we have created adds JPEG 2000 support back into ImageMagick for Ubuntu 16.04 and 18.04. More information on why JPEG 2000 support was removed can be found here, and more information on openjpeg in Ubuntu can be found here.
Review the Ubuntu Packaging Guide. Key items needed are in the Getting Set Up section:
sudo apt install gnupg pbuilder ubuntu-dev-tools apt-filepbuilder:pbuilder-dist bionic createdebuild:imagemagick is used an example. If you need to only change or patch the actual source code, then you will need to use quilt. More information on using quilt can be found in the Patches to Packages section. If you are only change dependencies, or information in the debian directory of a package, quilt is not required, and if used will cause build failures.
mkdir imagemagick-bionic-jp2cd imagemagick-bionic-jp2pull-lp-source imagemagick bioniccd imagemagick-6.9.7.4+dfsgdch (Make sure to change UNRELEASED to the Ubuntu release name. For example: bionic)debuild -SIndex: imagemagick-6.9.7.4+dfsg/debian/control\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control\n+++ imagemagick-6.9.7.4+dfsg/debian/control\n@@ -26,8 +26,7 @@ Build-Depends: debhelper (>= 10),\n libx11-dev, libxext-dev, libxt-dev,\n # for plugins\n ghostscript, libdjvulibre-dev, libexif-dev,\n- libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libjpeg-dev, libopenjp2-7-dev,\n libopenexr-dev, libperl-dev, libpng-dev, libtiff-dev,\n libwmf-dev,\n # libgraphviz-dev, incompatible license against fftw\n@@ -273,8 +272,7 @@ Depends: libmagickcore-6-headers (= ${so\n libmagickcore-6.q16-3 (= ${binary:Version}),\n libmagickcore-6.q16-3-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\n@@ -483,8 +481,7 @@ Depends: libmagickcore-6-headers (= ${so\n libmagickcore-6.q16hdri-3 (= ${binary:Version}),\n libmagickcore-6.q16hdri-3-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\nIndex: imagemagick-6.9.7.4+dfsg/debian/control.d/noquantum.in\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control.d/noquantum.in\n+++ imagemagick-6.9.7.4+dfsg/debian/control.d/noquantum.in\n@@ -26,8 +26,7 @@ Build-Depends: debhelper (>= 10),\n libx11-dev, libxext-dev, libxt-dev,\n # for plugins\n ghostscript, libdjvulibre-dev, libexif-dev,\n- libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libjpeg-dev, libopenjp2-7-dev,\n libopenexr-dev, libperl-dev, libpng-dev, libtiff-dev,\n libwmf-dev,\n # libgraphviz-dev, incompatible license against fftw\nIndex: imagemagick-6.9.7.4+dfsg/debian/control.d/quantum.in\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control.d/quantum.in\n+++ imagemagick-6.9.7.4+dfsg/debian/control.d/quantum.in\n@@ -78,8 +78,7 @@ Depends: libmagickcore-${IMVERSION}-head\n libmagickcore-${IMVERSION}.${QUANTUMDEPTH}-${CORESOVERSION} (= ${binary:Version}),\n libmagickcore-${IMVERSION}.${QUANTUMDEPTH}-${CORESOVERSION}-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\nIndex: imagemagick-6.9.7.4+dfsg/debian/rules\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/rules\n+++ imagemagick-6.9.7.4+dfsg/debian/rules\n@@ -98,7 +98,7 @@ STATIC_CONFIGURE_OPTIONS := \\\n --with-gs-font-dir=/usr/share/fonts/type1/gsfonts \\\n --with-magick-plus-plus \\\n --with-djvu \\\n- --with-openjp2 \\\n+ --with-openjp2 \\\n --with-wmf \\\n --without-gvc \\\n --enable-shared \\\nOnce the package is built successfully, you'll upload the changes file to Launchpad. For example:
dput ppa:lyrasis/imagemagick-jp2 imagemagick_6.9.7.4+dfsg-16ubuntu6.8_source.changesAfter the package is successfully uploaded to the PPA, you'll receive an email from Launchpad. Something like this:
Accepted:\n OK: imagemagick_6.9.7.4+dfsg.orig.tar.xz\n OK: imagemagick_6.9.7.4+dfsg-16ubuntu6.8.debian.tar.xz\n OK: imagemagick_6.9.7.4+dfsg-16ubuntu6.8.dsc\n -> Component: main Section: graphics\n\nimagemagick (8:6.9.7.4+dfsg-16ubuntu6.8) bionic; urgency=medium\n\n * Add jp2 support.\nAnd you will see it in the interface for the Lyrasis PPA:
"},{"location":"technical-documentation/resizing-vm/","title":"Resizing a VM","text":""},{"location":"technical-documentation/resizing-vm/#resize-vagrant-machine","title":"Resize vagrant machine","text":"To expand virtual machine's hard drive for testing of larger files. Once the VM has started, you'll need to halt the VM, download and run the script, tell it what size (in MB) and then start the VM. The last step vagrant ssh --command \"sudo resize2fs /dev/sda1\" is a check. It should return there was nothing to do. If you already provisioned you VM you can skip the 2 steps with provisioning in them.
# Skip this if you VM is already provisioned.\n$ vagrant up --no-provision <-- Exclude if already running and provisioned.\n\n$ vagrant halt\n\n# Download and run. This will default to the correct name (just press enter) then give the size.\n# Example: `350000` is equal to 350GB\n$ wget https://gist.githubusercontent.com/DonRichards/6dc6c81ae9fc22cba8d7a57b90ab1509/raw/45017e07a3b93657f8822dfbbe4fc690169cdabc/expand_disk.py\n$ chmod +x expand_disk.py\n$ python expand_disk.py\n$ vagrant up --no-provision\n\n# This step isn't needed but acts as a check to verify it worked.\n$ vagrant ssh --command \"sudo resize2fs /dev/sda1\"\n# Skip this if you VM is already provisioned.\n$ vagrant provision <-- Exclude if already provisioned.\nYou may need to remove the \"resized\" version. Assuming your VM location is ~/VirtualBox\\ VMs
$ rm -rf ~/VirtualBox\\ VMs/Islandora\\ CLAW\\ Ansible_resized\nIf your resources are restricted (hidden) you will need to have authorization to access them.
You can specify which types of authentication are allowed for which HTTP methods.
These are common to all HTTP methods against the REST API.
In the above screenshot we have 3 allowed methods. 1. basic_auth 1. jwt_auth 1. cookie
"},{"location":"technical-documentation/rest-authorization/#basic-authentication-basic_auth","title":"Basic authentication (basic_auth)","text":"To use basic authentication with a client like cURL use the -u username:password argument.
For example:
curl -u admin:islandora http://localhost:8000/node/3\nBy default JWTs are passed internally from Drupal to various microservices and Fedora.
To use a JWT yourself you need to enable the JWT Authentication Issuer module.
Once enabled this module makes a /jwt/token endpoint. You can perform a GET against this endpoint as an authenticated user to receive a JWT.
For example:
curl -i -u admin:islandora http://localhost:8000/jwt/token\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 22:08:37 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 620\nContent-Type: application/json\n\n{\n \"token\" : \"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE1NTE3MzczMTcsImV4cCI6MTU1MTc0NDUxNywiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.QUTrMiK_DyBxqQY4LnibLYtieEW3-MyjjQO9NSFI7bPylNm1S5ZY0uvzjDob3ckYgRN4uCyMFZO4BPytpQVA_jyeSuZyUA_10v33ItpoKyjrJ_S057iykNd_rWmxe8tT8T1fPypq_-Z7Th_PkyZWrYBqoBBVO1iVQt5txxfGWMqhxd2FgsXw6N-aR9sYOSc4UrLmFRmPP5Zk_CNIZP6NtBaM9JNr8CnWyPEFSAR75xfyH3ge5qjBqLlDS389k07pyJFB5rOT59txzLE9WLvpp9JK3oQv821Q1Bp-PMiASghXc0dBCHxM8o41BzLE88UstRA7agBAkUqG3hMpoNZqfA\"\n}\nYou can then take the same token and re-use it.
curl -H\"Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE1NTE3MzczMTcsImV4cCI6MTU1MTc0NDUxNywiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.QUTrMiK_DyBxqQY4LnibLYtieEW3-MyjjQO9NSFI7bPylNm1S5ZY0uvzjDob3ckYgRN4uCyMFZO4BPytpQVA_jyeSuZyUA_10v33ItpoKyjrJ_S057iykNd_rWmxe8tT8T1fPypq_-Z7Th_PkyZWrYBqoBBVO1iVQt5txxfGWMqhxd2FgsXw6N-aR9sYOSc4UrLmFRmPP5Zk_CNIZP6NtBaM9JNr8CnWyPEFSAR75xfyH3ge5qjBqLlDS389k07pyJFB5rOT59txzLE9WLvpp9JK3oQv821Q1Bp-PMiASghXc0dBCHxM8o41BzLE88UstRA7agBAkUqG3hMpoNZqfA\" http://localhost:8000/node/3?_format=jsonld\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 22:10:02 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/3>; rel=\"canonical\"\nLink: <http://localhost:8000/node/3/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=3>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/3/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/3/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/3>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=3>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/3?_format=json>; rel=\"alternate\"; type=\"application/json\"\nX-Drupal-Dynamic-Cache: HIT\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 858\nContent-Type: application/ld+json\n\n{\n \"@graph\": [\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/node\\\\/3?_format=jsonld\",\n \"@type\": [\n \"http:\\\\/\\\\/pcdm.org\\\\/models#Object\"\n ],\n \"http:\\\\/\\\\/purl.org\\\\/dc\\\\/terms\\\\/title\": [\n {\n \"@value\": \"Custom item\",\n \"@language\": \"en\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/author\": [\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/user\\\\/1?_format=jsonld\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/dateCreated\": [\n {\n \"@value\": \"2019-03-01T19:42:54+00:00\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#dateTime\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/dateModified\": [\n {\n \"@value\": \"2019-03-01T19:43:12+00:00\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#dateTime\"\n }\n ],\n \"http:\\\\/\\\\/purl.org\\\\/dc\\\\/terms\\\\/extent\": [\n {\n \"@value\": \"1 item\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#string\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/sameAs\": [\n {\n \"@value\": \"http:\\\\/\\\\/localhost:8000\\\\/node\\\\/1?_format=jsonld\"\n }\n ]\n },\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/user\\\\/1?_format=jsonld\",\n \"@type\": [\n \"http:\\\\/\\\\/schema.org\\\\/Person\"\n ]\n }\n ]\n}\nThis allows you to use a cookie stored in your web browser when you log in to Drupal to access these REST endpoint pages.
This is what allows you to access the URIs like http://localhost:8000/node/1?_format=json with your web browser.
You will need to use one of the configured authorization methods to create content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-create/#content-nodes","title":"Content (Nodes)","text":"The above setup shows a configuration where the JSON format is enabled for GET, PATCH, DELETE, and POST, with authentication types \"basic_auth\" and \"jwt_auth\" enabled for each method. Thus, with this configuration, you can perform a POST request against a node at the /node endpoint with a body in the JSON format.
To create a node, you need to provide two elements in your message body: the node type and any required field values.
For the Repository Item content type included in the Islandora Starter Site, these are:
A good way to make your first POST request is to perform a GET request against an existing node and erase all the extra content.
You can find more information about GET requests here.
Again we are using the json format.
\ud83c\udf4e For example curl -X GET 'http://localhost:8000/node/3?_format=json
Look for the type element
\"type\" : [\n {\n \"target_id\" : \"islandora_object\",\n \"target_type\" : \"node_type\",\n \"target_uuid\" : \"62189bec-3ef3-4196-b847-b17e5ce61fd5\"\n }\n]\nIn our example \"islandora_object\" is the machine name of the content type \"Repository Item\". If you have created a new type you will have a different target_id.
You will not need the target_uuid.
Next look for the title element
\"title\" : [\n {\n \"value\" : \"An example Islandora object\"\n }\n]\nLastly look for the field_model element
\"field_model\": [\n {\n \"target_id\": 24,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e7560b68-e95a-4e76-9671-2a3041cd9800\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/24\"\n }\n]\nYou can find the models by browsing the taxonomy terms available at http://localhost:8000/admin/structure/taxonomy/manage/islandora_models/overview
In my example installation, term 24 is an \"Image\", but let's create a collection which is term 23.
Note: Taxonomy terms may vary between instances and you should verify the correct number for your installation.
So the body of the request will be:
{\n \"type\": [\n {\n \"target_id\": \"islandora_object\",\n \"target_type\": \"node_type\"\n }\n ],\n \"title\": [\n {\n \"value\": \"Created a collection with POST\"\n }\n ],\n \"field_model\": [\n {\n \"target_id\": 23,\n \"target_type\": \"taxonomy_term\"\n }\n ]\n}\nNote: You must include an appropriate Content-type header for the format you're requesting
Other Note: You must include some authentication credentials to say who you are and so Drupal can check if you are allowed to create this object. Otherwise you will receive a 401 Unauthorized response.
If you do provide credentials but don't have permission, you will receive a 403 Forbidden response.
You can find more information about Authorization here
\ud83c\udf4e For example:
curl -i -X POST -u admin:islandora -H\"Content-type: application/json\" --data '{\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\"}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\"}]}' 'http://localhost:8000/node?_format=json'\n\nHTTP/1.1 201 Created\nDate: Tue, 05 Mar 2019 18:07:00 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nLocation: http://localhost:8000/node/3\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1564\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":3}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nThe parts of the above request are:
-i - return the response headers-X POST - send a POST request-u admin:islandora - use these basic authentication credentials-H\"Content-type: application/json\" - send the content-type header--data '{...}' - send the request body (seen above)'http://localhost:8000/node?_format=json' - the endpoint of the requestThe Drupal REST UI is supposed to have a way to upload files, but this seems to require the use of an X-CSRF-Token, which can only be retrieved using Cookie authentication and even then does not allow you to upload.
However there is a special REST endpoint created by Islandora, which is less configurable and is not part of the above-mentioned REST UI.
This endpoint is available at http://localhost:8000/node/{node id}/media/{media type}/{media use}
It only accepts PUT requests. If the media and file don't exist they are created, if they exist the file is updated with the new body.
The node id and taxonomy term id are used to search (via an entity query) for a matching media. If this media exists the body of the file is replaced with the new content, otherwise a new file and media are created to hold the content.
The tokens to this URI are as follows:
You can find the media use taxonomy terms at http://localhost:8000/admin/structure/taxonomy/manage/islandora_media_use/overview
The body of the request is the actual binary file to upload.
\ud83c\udf4e For example:
With a local file called my-image.png that I wanted to link to a node with ID 3.
I am using the taxonomy term \"Original file\", which on my machine is 16
> curl -i -X PUT -u admin:islandora -H\"Content-type: image/png\" --data-binary \"@my-image.png\" -H\"Content-Location: public://images/my-image.png\" 'http://localhost:8000/node/3/media/image/16'\n\nHTTP/1.1 100 Continue\n\nHTTP/1.1 201 Created\nDate: Tue, 05 Mar 2019 22:01:39 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLocation: http://localhost:8000/media/4\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 0\nContent-Type: text/html; charset=UTF-8\nThe parts of the above request are:
-i - return the response headers-X PUT - send a PUT request-u admin:islandora - use these basic authentication credentials-H\"Content-type: image/png\" - send the content-type header--data-binary \"@my-image.png\" - send the contents of the file located at my-image.png as binary-H\"Content-Location: public://images/my-image.png\" - store the file in the public scheme (ie. in Drupal) at the path images/my-image.png, to store the file in Fedora use the fedora// scheme (ie. fedora://images/my-image.png)'http://localhost:8000/node/3/media/image/16' - the endpoint of the request specifying the node, media type and taxonomy term.Deleting is as easy as getting resources and more difficult than creating resources.
If you can perform a GET request then you have the information required to perform a DELETE request.
Check out the examples below.
If you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-delete/#content-nodes","title":"Content (Nodes)","text":"You will need your node id, you can find more information in the GET documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
Our example node has a node id of 2
> curl -i -u admin:islandora -X GET 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 15:02:00 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/2>; rel=\"canonical\"\nLink: <http://localhost:8000/node/2/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=2>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/2/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/2/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/2>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=2>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/dc/dcmitype/Collection>; rel=\"tag\"; title=\"Collection\"\nLink: <http://localhost:8000/node/2?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1564\nContent-Type: application/json\n\n{\"nid\":[{\"value\":2}],\"uuid\":[{\"value\":\"413135a6-0bd1-4d6b-8bcb-059cf7784d83\"}],\"vid\":[{\"value\":2}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}%\nThen we switch GET to DELETE
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 15:02:30 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nAll subsequent requests to the above URI will return a 404 Not Found status code.
> curl -i -u admin:islandora -X GET 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 15:12:58 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 149\nContent-Type: application/json\n\n{\"message\":\"The \\u0022node\\u0022 parameter was not converted for the path \\u0022\\/node\\/{node}\\u0022 (route name: \\u0022rest.entity.node.GET\\u0022)\"}\nYou will need a media id as used in GET documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
With a media id of 1 for our example, I'll perform a GET
> curl -i -u admin:islandora -X GET 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 14:53:54 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/media/add>; rel=\"https://drupal.org/link-relations/add-page\"\nLink: <http://localhost:8000/media/add/image>; rel=\"https://drupal.org/link-relations/add-form\"\nLink: <http://localhost:8000/media/1>; rel=\"canonical\"\nLink: <http://localhost:8000/admin/content/media>; rel=\"collection\"\nLink: <http://localhost:8000/media/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/media/delete?media=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/media/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/media/1>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node/1>; rel=\"related\"; title=\"Media of\"\nLink: <http://pcdm.org/use#OriginalFile>; rel=\"tag\"; title=\"Original File\"\nLink: <http://localhost:8000/media/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nLink: <http://localhost:8000/media/1/source>; rel=\"edit-media\"\nLink: <http://localhost:8000/_flysystem/fedora/2019-03/Louis_Riel.jpg>; rel=\"describes\"; type=\"image/jpeg\"\nX-Drupal-Dynamic-Cache: HIT\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1937\nContent-Type: application/json\n\n{\"mid\":[{\"value\":1}],\"uuid\":[{\"value\":\"d8893926-ddb7-4125-b2da-30428af0fe3d\"}],\"vid\":[{\"value\":1}],\"langcode\":[{\"value\":\"en\"}],\"bundle\":[{\"target_id\":\"image\",\"target_type\":\"media_type\",\"target_uuid\":\"3860e653-201b-4509-89dd-628c446d81cb\"}],\"revision_created\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_user\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log_message\":[],\"status\":[{\"value\":true}],\"name\":[{\"value\":\"An image\"}],\"thumbnail\":[{\"target_id\":2,\"alt\":\"A portrait of Louis Riel\",\"title\":null,\"width\":800,\"height\":1333,\"target_type\":\"file\",\"target_uuid\":\"b0625129-c592-463a-93c3-3eff7cd3567e\",\"url\":\"http:\\/\\/localhost:8000\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-01T19:43:22+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_file_size\":[{\"value\":166613}],\"field_height\":[{\"value\":1333}],\"field_media_image\":[{\"target_id\":2,\"alt\":\"A portrait of Louis Riel\",\"title\":\"\",\"width\":800,\"height\":1333,\"target_type\":\"file\",\"target_uuid\":\"b0625129-c592-463a-93c3-3eff7cd3567e\",\"url\":\"http:\\/\\/localhost:8000\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"field_media_of\":[{\"target_id\":1,\"target_type\":\"node\",\"target_uuid\":\"8322e36e-f8ec-4fd9-919d-52aed7b17a52\",\"url\":\"\\/node\\/1\"}],\"field_media_use\":[{\"target_id\":16,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"08e01ff9-eb72-42f5-ae3a-8b21ba0c0bc3\",\"url\":\"\\/taxonomy\\/term\\/16\"}],\"field_mime_type\":[{\"value\":\"image\\/jpeg\"}],\"field_width\":[{\"value\":800}]}\nThen we replace GET with DELETE.
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 14:54:55 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nSubsequent requests to the media return 404 Not Found statuses.
> curl -i -u admin:islandora -X GET 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 15:13:45 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 153\nContent-Type: application/json\n\n{\"message\":\"The \\u0022media\\u0022 parameter was not converted for the path \\u0022\\/media\\/{media}\\u0022 (route name: \\u0022rest.entity.media.GET\\u0022)\"}\nYou'll need the file id, there is more information at the top of the GET requests for files documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
With a file id of 2 for our example, I can perform a test GET
> curl -i -u admin:islandora 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 14:40:40 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 617\nContent-Type: application/json\n\n{\"fid\":[{\"value\":2}],\"uuid\":[{\"value\":\"b0625129-c592-463a-93c3-3eff7cd3567e\"}],\"langcode\":[{\"value\":\"en\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"filename\":[{\"value\":\"Louis_Riel.jpg\"}],\"uri\":[{\"value\":\"fedora:\\/\\/2019-03\\/Louis_Riel.jpg\",\"url\":\"\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"filemime\":[{\"value\":\"image\\/jpeg\"}],\"filesize\":[{\"value\":166613}],\"status\":[{\"value\":true}],\"created\":[{\"value\":\"2019-03-01T19:43:35+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}]}\nIf this is the correct file, I can delete it.
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 14:43:22 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nSubsequent requests to the URI now return a 404 Not Found status.
> curl -i -u admin:islandora 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 14:43:33 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 157\nContent-Type: application/json\n\n{\"message\":\"The \\u0022file\\u0022 parameter was not converted for the path \\u0022\\/entity\\/file\\/{file}\\u0022 (route name: \\u0022rest.entity.file.GET\\u0022)\"}\nIf you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-get/#content-nodes","title":"Content (Nodes)","text":"The above setup shows that you can perform a GET request against a node at the /node/{id} endpoint. This is the same as the URL when viewed through a web browser. For example http://localhost:8000/node/2
It also defines the formats you can access. In this example json and jsonld are available.
To access a resource in a specific format use the _format= argument which is appended to the end of the URI after a ?.
To get the resource located at http://localhost:8000/node/3 in JSON, you can go to http://localhost:8000/node/3?_format=json.
\ud83c\udf4e For example:
You can do this in your web browser or with a command line client like cURL. (Note: response body here has been formatted, yours will look different)
vagrant@claw:~$ curl -i http://localhost:8000/node/3?_format=json\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 21:53:50 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/3>; rel=\"canonical\"\nLink: <http://localhost:8000/node/3/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=3>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/3/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/3/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/3>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=3>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/3?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nX-Drupal-Cache: MISS\nContent-Length: 1521\nContent-Type: application/json\n\n{\n \"nid\": [\n {\n \"value\": 1\n }\n ],\n \"uuid\": [\n {\n \"value\": \"8322e36e-f8ec-4fd9-919d-52aed7b17a52\"\n }\n ],\n \"vid\": [\n {\n \"value\": 1\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"type\": [\n {\n \"target_id\": \"islandora_object\",\n \"target_type\": \"node_type\",\n \"target_uuid\": \"62189bec-3ef3-4196-b847-b17e5ce61fd5\"\n }\n ],\n \"revision_timestamp\": [\n {\n \"value\": \"2019-03-01T19:43:12+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"revision_uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"revision_log\": [],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"title\": [\n {\n \"value\": \"Custom item\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:42:54+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:12+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"promote\": [\n {\n \"value\": true\n }\n ],\n \"sticky\": [\n {\n \"value\": false\n }\n ],\n \"default_langcode\": [\n {\n \"value\": true\n }\n ],\n \"revision_translation_affected\": [\n {\n \"value\": true\n }\n ],\n \"content_translation_source\": [\n {\n \"value\": \"und\"\n }\n ],\n \"content_translation_outdated\": [\n {\n \"value\": false\n }\n ],\n \"field_alternative_title\": [],\n \"field_description\": [],\n \"field_display_hints\": [],\n \"field_edtf_date\": [],\n \"field_edtf_date_created\": [],\n \"field_edtf_date_issued\": [],\n \"field_extent\": [\n {\n \"value\": \"1 item\"\n }\n ],\n \"field_identifier\": [],\n \"field_linked_agent\": [],\n \"field_member_of\": [],\n \"field_model\": [\n {\n \"target_id\": 24,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e7560b68-e95a-4e76-9671-2a3041cd9800\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/24\"\n }\n ],\n \"field_pid\": [],\n \"field_resource_type\": [],\n \"field_rights\": [],\n \"field_subject\": []\n}\nThe above steps are the same for media, it is just the URI that has changed.
For media you use /media/{id}
\ud83c\udf4e For example:
curl -u admin:islandora http://localhost:8000/media/2?_format=json\n\n{\n \"mid\": [\n {\n \"value\": 2\n }\n ],\n \"uuid\": [\n {\n \"value\": \"e75e609d-510c-4764-9280-665bb7026161\"\n }\n ],\n \"vid\": [\n {\n \"value\": 2\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"bundle\": [\n {\n \"target_id\": \"image\",\n \"target_type\": \"media_type\",\n \"target_uuid\": \"3860e653-201b-4509-89dd-628c446d81cb\"\n }\n ],\n \"revision_created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"revision_user\": [],\n \"revision_log_message\": [],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"name\": [\n {\n \"value\": \"1-Service File.jpg\"\n }\n ],\n \"thumbnail\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"default_langcode\": [\n {\n \"value\": true\n }\n ],\n \"revision_translation_affected\": [\n {\n \"value\": true\n }\n ],\n \"content_translation_source\": [\n {\n \"value\": \"und\"\n }\n ],\n \"content_translation_outdated\": [\n {\n \"value\": false\n }\n ],\n \"field_access_terms\": [],\n \"field_file_size\": [\n {\n \"value\": 166608\n }\n ],\n \"field_height\": [\n {\n \"value\": 1333\n }\n ],\n \"field_media_image\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"field_media_of\": [\n {\n \"target_id\": 1,\n \"target_type\": \"node\",\n \"target_uuid\": \"8322e36e-f8ec-4fd9-919d-52aed7b17a52\",\n \"url\": \"\\\\/node\\\\/1\"\n }\n ],\n \"field_media_use\": [\n {\n \"target_id\": 18,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e8c698a7-8c15-47c4-8567-1c27f8f0c19e\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/18\"\n }\n ],\n \"field_mime_type\": [\n {\n \"value\": \"image\\\\/jpeg\"\n }\n ],\n \"field_width\": [\n {\n \"value\": 800\n }\n ]\n}\nfield_media_of indicates which node(s) the media is associated with. The node can be referenced using the value of the target_id key (in this example, 1) or the target_uuid key (8322e36e-f8ec-4fd9-919d-52aed7b17a52).
Through the REST API you are accessing metadata about files, but not the actual binary object.
Based on the above configuration the path for a GET request is /entity/file/{id}
The file ID refers to the integer counter and not the UUID of a file.
Through the UI you can go to the File listing at http://localhost:8000/admin/content/files.
For the file you want to access, choose the USED IN link.
That URI has the format http://localhost:8000/admin/content/files/usage/{file id}
Alternatively the file ID can be found by viewing the media metadata (above section). In the above example if you look through the response from a GET request to the /media/{id} you will find a section like:
\"field_media_image\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\nNote: Depending on the \"type\" of file (image, video, audio, file) this field will have a different name.
This has both the url which is the URI to retrieve the binary content of the file as well as the target_id which is the file ID.
So we can retrieve the file with a GET request to the URI http://localhost:8000/sites/default/files/2019-03/1-Service%20File.jpg.
But more importantly we can retrieve the file metadata by making a request to http://localhost:8000/entity/file/3 with one of the allowed formats
\ud83c\udf4e For example:
> curl -i -X GET \"http://localhost:8000/entity/file/3?_format=json\"\n\nHTTP/1.1 200 OK\nDate: Tue, 05 Mar 2019 16:52:06 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nX-Drupal-Cache: HIT\nContent-Length: 634\nContent-Type: application/json\n\n{\n \"fid\": [\n {\n \"value\": 3\n }\n ],\n \"uuid\": [\n {\n \"value\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\"\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"filename\": [\n {\n \"value\": \"1-Service File.jpg\"\n }\n ],\n \"uri\": [\n {\n \"value\": \"public:\\\\/\\\\/2019-03\\\\/1-Service File.jpg\",\n \"url\": \"\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"filemime\": [\n {\n \"value\": \"image\\\\/jpeg\"\n }\n ],\n \"filesize\": [\n {\n \"value\": 166608\n }\n ],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ]\n}\nThe uri: url: is the path to your file, so for the above example of \"url\": \"\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\".
I can perform a GET against <my hostname>/sites/default/files/2019-03/1-Service%20File.jpg and get the contents of this file.
PATCH requests allow you to update resources in place via a REST call. In this case you send a few required fields and then any changed fields. PATCH requests are available for nodes and media entities, but not for files. PATCH requests are very consistent between node and media entities so we will just show one set of examples here.
Our example node is at http://localhost:8000/node/3.
If you perform a GET against another node or media you can see all of the fields, some of these are calculated (change based on others, i.e. \"changed\"), some are references to other entities (nodes, media, files in Drupal. i.e. \"field_model\") and others are pure text fields (i.e. \"title\").
The fields will be different between different node types and media types, but the syntax to update them is the same.
If you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
This with assume you have already created a node or media at some location.
"},{"location":"technical-documentation/rest-patch/#change-a-text-field","title":"Change a text field","text":"To change the title of a node, you need to supply the new title and the node_type. The node_type is required as this defines what fields are available to the node.
A successful PATCH request will return a 200 status code and the body will contain the newly updated body.
curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"title\": [{\"value\":\"Updated with a PATCH request\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 17:01:23 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1562\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":3}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T17:01:23+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nThis example is how to change a field that references some other entity. For this example we will use the field_model field, this is a reference to the taxonomy term that holds the \"model\" of the resource (ie. Image, Collection, Audio, Video, etc)
On our example installation taxonomy term 22 is \"Binary\", so to change a node from what it was to a Binary you would do.
> curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"field_model\": [{\"target_id\": 22, \"target_type\": \"taxonomy_term\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 17:51:47 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1562\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":4}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-11T17:36:10+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T17:51:47+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":22,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"e1f167e1-124d-4db4-96ab-30641ca4e21b\",\"url\":\"\\/taxonomy\\/term\\/22\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nTo patch an object and make it part of a collection, you need the id number of the collection object. In this example node 2 will be our collection.
target_type can be a confusing one, if you are ever unsure have a look at the returned values for an existing object.
> curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"field_member_of\": [{\"target_id\": 2, \"target_type\": \"node_type\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 18:01:40 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1669\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":4}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-11T17:36:10+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T18:01:40+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[{\"target_id\":2,\"target_type\":\"node\",\"target_uuid\":\"413135a6-0bd1-4d6b-8bcb-059cf7784d83\",\"url\":\"\\/node\\/2\"}],\"field_model\":[{\"target_id\":22,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"e1f167e1-124d-4db4-96ab-30641ca4e21b\",\"url\":\"\\/taxonomy\\/term\\/22\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nSignposting is a technique used in RESTful APIs where other relevant resources are exposed to clients as Link headers in GET and HEAD requests. These Link headers follow a standard format as specified in RFC8288. Drupal already makes use of this technique for content entities, and Islandora takes it even further by providing additional Link headers that enable the client to navigate the repository and discover additional information about various resources. Because the links are returned in response headers, they can be relied upon without having to parse the message body. This makes them consistent across all serialization formats that can be returned in a message body (XML, JSON, JSONLD, etc...).
As a general precaution, link headers for Drupal entities are not exposed to users that do not have the permissions to view the entity linked in the header. So making GET and HEAD requests anonymously will yield a different set of headers than what an authenticated user would see. For example, anonymous users don't have the view media permission, so they will not see the link headers for media associated with a node.
Other representations generated by different serializers available through Drupal's REST API are exposed as link headers with rel=\"alternate\" and type equal to the mimetype that will be received when dereferencing the link. For example, if an entity in Drupal has a JSONLD representation, then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/node/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"
Entity reference fields are exposed as link headers with rel=\"related\" and a title equal to the entity reference field's display label. For example, if http://example.org/node/1 has an entity reference field name \"Associated Content\" that references http://example.org/node/2, then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/node/2>; rel=\"related\"; title=\"Associated Content\"
Entity reference fields for taxonomy terms get special handling. The taxonomy terms used to tag content are exposed as link headers with rel=\"tag\" and a title equal to the taxonomy term's display label. If the term has an external URI in a controlled vocabulary, then that URI is provided. Otherwise, the local Drupal URI is provided. For example, if a piece of content is tagged with taxonomy/term/1, which has a display label of \"Example Term\", then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/taxonomy/term/1>; rel=\"tag\"; title=\"Example Term\"
If instead the term were to have the field_external_uri field with a value of http://exampletwo.org/vocab#term then the link header would look like
Link: <http://exampletwo.org/vocab#term>; rel=\"tag\"; title=\"Example Term\".
Media entities belonging to nodes are exposed as link headers with rel=\"related\" and a title equal to the display label of their field_media_use taxonomy term. For example, if a media is tagged as Original File indicating that it is the initial file uploaded, the link header returned in a GET or HEAD response for a node would look like
Link: <http://example.org/media/1>; rel=\"related\"; title=\"Original File\".
Files that are the source for media entities are exposed as Link headers in the GET and HEAD responses with rel=\"describes\". The endpoint to edit the contents of the source file is also exposed using rel=\"edit-media\". For example, if http://example.org/media/1 has the source file http://example.org/file.txt, then a GET or HEAD response would contain both
Link: <http://example.org/file.txt>; rel=\"describes\"Link: <http://example.org/media/1/source>; rel=\"edit-media\"After creating a node, adding it to a Collection, uploading a file and kicking off derivatives, the link headers returned for said node would look like the following. Note that non-Link headers have been removed for brevity:
vagrant@claw:~$ curl -I http://localhost:8000/node/1?_format=json\nHTTP/1.1 200 OK\n...\n# These are provided by Drupal core\nLink: <http://localhost:8000/node/1>; rel=\"canonical\"\nLink: <http://localhost:8000/node/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/1/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/1>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=1>; rel=\"https://drupal.org/link-relations/create\"\n# These are provided by Islandora\nLink: <http://localhost:8000/node/2>; rel=\"related\"; title=\"Member of\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nIf we were to inspect one of the Media associated with this node (which we would've gotten in the response above), the results would look like:
vagrant@claw:~$ curl -I http://localhost:8000/media/1?_format=json\nHTTP/1.1 200 OK\n...\n\n# These are provided by Drupal core\nLink: <http://localhost:8000/media/add>; rel=\"https://drupal.org/link-relations/add-page\"\nLink: <http://localhost:8000/media/add/image>; rel=\"https://drupal.org/link-relations/add-form\"\nLink: <http://localhost:8000/media/1>; rel=\"canonical\"\nLink: <http://localhost:8000/admin/content/media>; rel=\"collection\"\nLink: <http://localhost:8000/media/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/media/delete?media=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/media/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/media/1>; rel=\"https://drupal.org/link-relations/revision\"\n# These are provided by Islandora\nLink: <http://localhost:8000/node/1>; rel=\"related\"; title=\"Media of\"\nLink: <http://pcdm.org/use#OriginalFile>; rel=\"tag\"; title=\"Original File\"\nLink: <http://localhost:8000/media/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nLink: <http://localhost:8000/media/1/source>; rel=\"edit-media\"\nLink: <http://localhost:8000/_flysystem/fedora/2019-03/IF-Org-Chart_0.jpg>; rel=\"describes\"; type=\"image/jpeg\"\nSee the Drupal Documentation on PHPUnit in Drupal.
"},{"location":"technical-documentation/running-automated-tests/#setting-up-phpunit","title":"Setting up PhpUnit","text":"Before you can use phpunit, you must first install the following:
composer require --dev -W phpspec/prophecy-phpunit drupal/core-dev
In ISLE, you need to make the database port available to PHPUnit. To do that, edit docker-compose.yml and find the section including image: islandora/mariadb[version number]. Shortly below is a labels: heading; set the value of the traefik.enable: to \"true\". Apply the changes made to the docker_compose.yml using docker compose up -d.
Follow the Configure PHPUnit and Create a directory for HTML output sections in Drupal Documentation on running phpunit tests to make a phpunit.xml file.
If you place the phpunit.xml file in any directory other than [drupal root]/web/core, you need to change the 'bootstrap' in the <phpunit> tag near the top of the file to point to the relative or absolute location of the [drupal root]/web/core folder.
When setting the SIMPLETEST_DB database credentials in ISLE, * the default username and db_name are drupal_default * your db_password can be found in codebase/web/sites/default/settings.php
Unless you changed the default values, just swap out [password] for your actual db password in the following:
mysql://drupal_default:[password]@islandora.traefik.me:3306/drupal_default`.\nIf you are in the Drupal root directory (codebase on ISLE; the one containing web) and your phpunit.xml file is also in the Drupal root directory, use the following command to run phpunit for a single test file (here, Islandora's DeleteNodeWithMediaAndFile.php):
vendor/bin/phpunit web/modules/contrib/islandora/tests/src/Functional/DeleteNodeWithMediaAndFile.php
If your phpunit.xml is in a different directory, such as web/core, then use the -c flag to specify the path to the directory containing phpunit.xml:
vendor/bin/phpunit -c web/core web/modules/contrib/islandora/tests/src/Functional/DeleteNodeWithMediaAndFile.php
For any Drupal URI, the corresponding Fedora URIs is computed by the Crayfish Commons library using the Drupal-assigned UUID. In Drupal, a \"pseudo-field\" is available that will display the corresponding Fedora URI on the page of a node, taxonomy term, or media object. The presence of this URI does NOT guarantee the existence of the corresponding entity in Fedora.
"},{"location":"technical-documentation/testing-notes/#enabling-the-fedora-uri-pseudo-field","title":"Enabling the Fedora URI Pseudo-field","text":"To display the Fedora URI pseudo-field on a Drupal node, media, or taxonomy term, go to Manage > Configuration > Islandora and select all the bundles for which you would like the Fedora URI displayed. Once you have selected the bundles, and cleared the cache, the new pseudo-field will appear at the bottom of all display modes. You can alter where in the display the Fedora URI field appears, by going to the \"Manage Display\" page for the bundle. For example, for a Repository Item, you'd go to Manage > Structure > Content Types, and under the dropdown for \"Repository Item, select \"Manage Display\". In that list you should see Fedora URI which you can move around (or hide) as desired. This will need to be repeated in each Display mode (tab). Clearing cache may be necessary to refresh the node display.
Note: This information used to be stored in a service called Gemini, which kept track of corresponding minted Fedora URIs and their minted equivalents. Gemini was removed for Islandora 2.0.0.
"},{"location":"technical-documentation/testing-notes/#how-do-i-search-for-a-object-in-the-solr","title":"How do I search for a object in the Solr?","text":"http://localhost:8983/solr/#/islandora/queryExample
ss_search_api_id:\"entity:node/4:en\"\nhttp://localhost:8080/bigdata/#queryhttp://localhost:8080/bigdata/#namespaces), make sure islandora is selected.select ?p ?o where { <drupal_url> ?p ?o }\nExample:
select ?p ?o where { <http://localhost:8000/media/8?_format=jsonld> ?p ?o }\nselect ?s where { ?s <http://pcdm.org/models#memberOf> <drual_url_of_the_collection?_format=jsonld> }\nExample:
select ?s where { ?s <http://pcdm.org/models#memberOf> <http://localhost:8000/node/7?_format=jsonld> }\nselect ?s where { ?s <http://pcdm.org/models#fileOf> <drupal_url_of_the_object?_format=jsonld> }\nExample:
select ?s where { ?s <http://pcdm.org/models#fileOf> <http://localhost:8000/node/4?_format=jsonld> }\nThis section describes how to update Drupal and its modules using Composer. If you installed Islandora using the Islandora Playbook or ISLE, then your Drupal was installed by Composer, so it is best practice to continue using Composer for updates. The method on this section is not specific to Islandora, and does not (yet) include how to update Islandora Features.
How to upgrade Drupal in ISLE
For specific instructions on how to upgrade Drupal core and the Drupal modules installed within ISLE, please refer to the documentation page: Maintaining Your Drupal Site
"},{"location":"technical-documentation/updating-drupal/#what-is-composer","title":"What is Composer","text":"It is recommended by Drupal.org and the Islandora community to use Composer with Drupal for various tasks.
\"Composer is a dependency manager for PHP. Drupal core uses Composer to manage core dependencies like Symfony components and Guzzle.\" [Source]
"},{"location":"technical-documentation/updating-drupal/#always-create-backs-ups-db-and-files-before-updating","title":"Always create backs ups (DB and files) before updating","text":"Before updating either Drupal core or Drupal modules:
Warning: Always revert to a backup if you get a fatal error in the update process.
"},{"location":"technical-documentation/updating-drupal/#updating-drupal-core","title":"Updating Drupal Core","text":"Over time new versions of Drupal \u201ccore\u201d are released, and Islandora users are encouraged to install official Drupal core updates and security patches. On the other hand \u201calpha\u201d and \u201cbeta\" versions of Drupal core should only be installed by advanced users for testing purposes.
The Islandora community STRONGLY recommends that the \"Composer\" method of upgrading Drupal core be used with Islandora as mentioned here.
"},{"location":"technical-documentation/updating-drupal/#here-is-an-overview-of-the-steps-for-updating-drupal-core-using-composer","title":"Here is an overview of the steps for updating Drupal core using Composer","text":"Back Up
First make sure you have made database and file back ups.
1) First, verify that an update of Drupal core actually is available:
composer outdated \"drupal/*\"
If there is no line starting with drupal/core, Composer isn't aware of any update. If there is an update, continue with the commands below.
2) Assuming you are used to updating Drupal and know all the precautions that you should take, the update is as simple as:
composer update drupal/core --with-dependencies
If you want to know all packages that will be updated by the update command, use the --dry-run option first.
Alternate syntax for Islandora 8 needed
If you are running the older Islandora 8 codebase that predates the Islandora 2 release, note that Islandora 8 is configured to use a fork of drupal-composer/drupal-project which requires this specific composer syntax compared to other Drupal 8+ sites:
composer update drupal/core webflo/drupal-core-require-dev \"symfony/*\" --with-dependencies
In addition, if you are upgrading from 8.5 to 8.7, you need to replace \"~8.5.x\" with \"^8.7.0\" for drupal/core and webflo/drupal-core-require-dev in composer.json. [Source]
3) Apply any required database updates using drush updatedb, or use the web admin user interface.
drush updatedb
4) Clear the cache using drush cache:rebuild, or use the web admin user interface.
drush cache:rebuild
For stepwise update instructions visit this page: https://www.drupal.org/docs/8/update/update-core-via-composer#s-stepwise-update-instructions
"},{"location":"technical-documentation/updating-drupal/#updating-drupal-modules","title":"Updating Drupal Modules","text":"Islandora uses several general Drupal modules and some specialized Islandora Drupal modules, and over time new versions of these modules are released. There are two approaches to updating Drupal modules in Islandora: using Composer or updating modules individually. Islandora uses Composer to determine which Drupal module versions should be installed for each release of Islandora. Therefore if you update the Islandora specific Drupal modules using Composer you will also update any dependent general Drupal modules as well. The second method is to individually update Drupal modules.
For more information about how to update Drupal modules visit:
https://www.drupal.org/docs/8/extending-drupal-8/updating-modules
Back Up
First make sure you have made database and file back ups.
"},{"location":"technical-documentation/using-rest-endpoints/","title":"Islandora via REST","text":"Each node, media and file in Drupal has its own URI and we can GET the resources, some in a variety of formats.
We can also create nodes, media and files in Drupal by using PUT and/or POST requests.
We can update the field content by using a PATCH request and lastly we can DELETE a node, media or file resource.
To perform these actions, you will need the RESTful Web Services module enabled.
To configure your setup via the UI, you also need the RESTful UI module enabled.
Then you can configure your REST services at https://<yourmachine>/admin/config/services/rest
This screenshot shows the setup for resources, you can see the various HTTP methods and what formats they will respond in and what authentication methods they support.
Islandora uses semantic versioning for all non-Legacy code in the github.com/Islandora organization. This allows us to be compatible with Composer and with Drupal's release naming conventions for contributed modules.
"},{"location":"technical-documentation/versioning/#semantic-versioning","title":"Semantic Versioning","text":"Semantic Versioning is a common versioning standard. Versions have the form: Major version, Minor version, or Patch.
1.2.3 => 1.2.4 - Just a bug fix, should be a drop-in replacement.1.2.3 => 1.3.0 - Adds in new features, should be a drop-in replacement to get new functionality.1.2.3 => 2.0.0 - Major changes, may require a migration or changes to your set-up.The following Islandora components use semantic versioning:
Drupal submodules, which are included in several of the above modules, share versions with their parents.
Drupal module versions switched from 8.x-1.x to 2.x
In October 2021, Islandora switched from the \"core compatibility\" based numbering scheme (8.x-1.x) to a pure semantic versioning scheme for its Drupal modules. In accordance with Drupal's requirements, this transition required us to bump the major version, from 1.x to 2.x, despite there not being any major API-breaking changes to the code itself.
"},{"location":"technical-documentation/versioning/#implications-for-release-process","title":"Implications for Release Process","text":"Committers should now create (i.e. \"tag\") new versions of components when new bug fixes, features, or API changes are successfully added. This means that \"releases\" (new versions) will be happening individually, continually, and far more frequently than before. See Releasing Islandora.
"},{"location":"technical-documentation/versioning/#module-interdependencies","title":"Module Interdependencies","text":"When Islandora components require other Islandora components in their composer.json files, we prefer the version specification syntax ^2 to point to the latest-released compatible version within the specified major version.
In Drupal, blocks are snippets of content that can be placed on pages within your website's layout. Some examples of blocks are your site's menu, breadcrumbs, search block, etc... You have a lot of control over when and where you see blocks using Drupal's administrative interface. You can create and delete blocks, as well as move them around in different locations on the page (called 'regions' in Drupal theme terminology). You can also control on which pages the blocks are visible, allowing you to conditionally show/hide them as appropriate. Blocks are made available to place by enabling the Drupal modules that provide them, and there's lots out there to choose from. Placing blocks is an essential skill for any site administrator, so let's jump right in and get our hands dirty. There are two methods discussed here: the \"Block Layout\" interface, and using \"Contexts\". There is a third option, \"Display Suite,\" that is not discussed here.
"},{"location":"tutorials/blocks/#using-block-layout","title":"Using Block Layout","text":"Using the admin toolbar, navigate to Admin > Structure > Block layout (admin/structure/block). You will see a table listing all of the available regions provided by the current theme for block placement. You can switch between available themes using the tabs above the table. For each region, enabled blocks are listed and can be re-arranged. Take a look at the \"Main Content\" region for example.
If you wanted to move the content above its tabs, you could drag the \"Main page content\" block above the \"Tabs\" block. If you scroll all the way down to the bottom and click the \"Save blocks\" button, now when you go to view content you should see something like this
If you want to add a new block to a region, hit the \"Place block\" button for that region and you'll be brought to a modal that will let you select which block to place.
After selecting your block, you will be brought to its configuration form.
Here you can give the block a different title, change its region using a dropdown, and control its visibility settings by configuring various conditions. In our case, we're adding the \"Development\" block to the \"Footer\" region. And since we want to see this block everywhere, we'll leave the visibility settings alone for now. Click the \"Save block\" button, and now if we go view some content we should see the \"Development\" menu in the footer of our site.
If we want to limit this eyesore to only the front page of our site, we can revisit the visibility settings of the block. Navigate to the block we just placed on the block placement page, and click its \"Configure\" button. Here you can limit the block to show only for certain roles or content types. You can also explicitly set a white or black list of pages, which is what we'll do.
Click the Pages tab and you'll have a text area you can list pages in. We're going to enter <front>, which is a special value that means \"The front page of your site\". Underneath the text area, you can select whether you are showing or hiding the block for this list of pages. We want to show this only on the front page, so we'll leave it set at \"Show for the listed pages\". Click \"Save block\" and go back to a piece of content. You shouldn't see the development menu.
But go to the frontpage and...
"},{"location":"tutorials/blocks/#using-context","title":"Using Context","text":"The visibility conditions in the block placement user interface are simple and effective. They cover the majority of use cases, however, there are times when you need even more control. This is where the Context module really shines. Any condition available to the Context module can be used to control block visibility. And you can even combine conditions using Boolean (AND/OR) logic.
For example, in the Islandora Starter Site, we provide a context specifically for showing a list of members of a collection. All of our content is the same content type (Repository Item), and Collections are distinguished by being tagged with the \"Collection\" taxonomy term. Using the admin toolbar, if you navigate to Admin > Structure > Context (admin/structure/context), you should see a list of all contexts available to the site.
Click the \"Configure\" button on the \"Collection\" context.
You can see in its \"Condition\" section, we have the \"Node has term\" condition, which checks for content tagged with the term provided by the user. In this case, it's set to look for the \"Collection\" tag. Underneath, in the \"Reaction\" section, you can see that the \"Block\" reaction has been selected, and its configuration looks a lot like the core block placement UI. The only difference here is that when the \"Include blocks from block layout\" box is checked, you are only placing the blocks you want to add in addition to an already existing block layout. If you uncheck the box, you are creating an entirely new block layout from scratch, and will have to replace basic things like breadcrumbs and menus.
Here you can see we've added only the members block to this block layout. If we go and navigate to a collection, you'll see its list of members underneath its content.
If we edit the collection and change its model tag from \"Collection\" to anything else, you'll see the block disappears!
As you can see, block placement can be very flexible, and allow you to customize your site in a very granular fashion. Between using core block placement and the context module, there's no block you can't get into the right place on your site.
"},{"location":"tutorials/blocks/#video-walkthrough-blocks","title":"Video Walkthrough: Blocks","text":"Click the image below to open the Blocks video tutorial on the Islandora Youtube channel.
See more videos from the Drupal 101 series here.
Islandora Quick Lessons (2020)
Learn more with this video on Creating a Custom Block.
"},{"location":"tutorials/create-a-resource-node/","title":"Create a Resource Node","text":""},{"location":"tutorials/create-a-resource-node/#overview","title":"Overview","text":"This tutorial will walk you through adding a piece of digital content (an image and some metadata) into an Islandora repository, creating an Islandora Resource Node. At the end, you will have metadata in a Drupal node, a file stored in Fedora, and derivatives automatically created.
Video version available
The material in this tutorial is presented in our video, Adding Content.
"},{"location":"tutorials/create-a-resource-node/#tools","title":"Tools","text":"For this tutorial you will need an instance of Islandora Starter Site. You can also use the online sandbox or one of the sandbox VMs.
Online Sandbox
login credentials for online sandbox
"},{"location":"tutorials/create-a-resource-node/#step-1-create-an-islandora-node","title":"Step 1: Create an Islandora Node","text":"To add content to Islandora, start by creating a node. When logged in to Drupal as an administrator, navigate with the top menu bar to Manage >> Content >> Add content or click on Add content under the Tools menu.
Next, select a content type that is configured with Islandora behaviours. In the Islandora Starter Site, there is an Islandora content type. It is called Repository Item.
At the top of the form, the Model field is a mandatory Islandora field. Select Image from the Model drop box. This will enable image display behaviours, and cause image derivatives to be generated once we upload a file.
The rest of the form contains a large number of metadata fields, grouped into collapsible sections. Fill out the Title (mandatory) and whichever other fields are applicable to you.
Keep it simple
For this tutorial, do not attempt to fill out any fields with an autocomplete symbol - a small circle at the right-hand-side of the text box.
Then, click Save. You have now created a node, and since it is of an Islandora content type, we call it a \"resource node.\"
The new node page displays, with the title, any metadata that we entered, and a Fedora URI which you can click on to see if this new node has been synced into Fedora (this only works if you have permission to access Fedora). Congratulations, you have created a resource node!
"},{"location":"tutorials/create-a-resource-node/#step-2-upload-an-original-file","title":"Step 2: Upload an Original File","text":"Alas, your node has no files. To upload a file, click on the node's Media tab.
Concept: Media tab
The \"Media\" tab shows Media that we say \"belong to\" or are \"media of\" that node. The idea of media \"belonging to\" nodes, and the \"Media\" tab, are part of Islandora but not part of standard Drupal.
The Media tab shows nothing listed, because this node has no files. So click on Add Media. You will be taken to a list of media types to choose from.
We want to add a jpg image, so click on Image. You are now presented with a form to create a media.
Images vs TIFFs
Drupal considers any type of image that can be viewed natively in the browser as an Image. For other image types that require special viewers, such as TIFFs, you would have to choose File. Learn more about Media in Islandora.
This form contains fields for metadata about the file, including \"technical metadata\". There are three important parts of the form:
With the mandatory fields filled out (do not edit the \"Media of\" field), click Save, and a new Media will be created attached to your resource node. You will land on the administrator's list of Media (i.e. all media on your site, not just those attached to the resource node), and the new media object you created is at the top of the list.
Select the Media you just created, and you will see a page containing that Media's information and a rendering of the image that you uploaded.
"},{"location":"tutorials/create-a-resource-node/#step-3-verify-derivatives-were-created","title":"Step 3: Verify derivatives were created","text":"Now return to the node you created, and you will see the image along with its (minimal) descriptive metadata. The image you are viewing is not the one you uploaded, but a lower-resolution service file that was created and stored in the Drupal public filesystem.
If you open the node's Media tab, you should see new Media have been added - these are derivatives have been created automatically, including a service file, a thumbnail, and a FITS technical metadata file.
You now have an Islandora resource node, which is a special case of a Drupal node, and it has Media attached to it including an Original File, Thumbnail, Service File, and FITS Technical Metadata file.
"},{"location":"tutorials/create-a-resource-node/#learn-more","title":"Learn More","text":"Views are powerful content filters that enable you to present Islandora (and other) content in interesting and exciting ways. For more documentation on views:
Islandora Starter Site ships with some views already created and turned on. The Islandora home page displays content items that have been added to Islandora. This view is named Frontpage, and it lists items that meet the following filter criteria:
This view will display all content items added to Islandora, as the checkbox Promoted to front page is on by default.
As you develop your Islandora Website it is likely that you will need to change the default behaviour of the Frontpage View. As an example, the following describes how to edit the Frontpage page view to only show content items and not collections.
For this example, we added six collection items to Islandora. In total there are eight items in the repository. In addition to the six collection items, there is one audio item and one image item.
To edit the front page view, hover over the view (Frontpage view) and select Edit view when displayed.
Select Add under the filter criteria section.
We do not want to display collections, so we need to add a filter criteria that does not filter for the Islandora model type 'Collection'.
Select Model from the list and then Apply (all displays).
Select Islandora Model to select filters on Islandora model types and select Apply and continue.
Select the operator Is none of and the Collection model (autocomplete should work here to help you). To finish click Apply (all displays).
Save the view. Now the 'Frontpage' View does not display collections.
For this example, we create a new view that only shows collections. It will be created as a Block (also see the tutorial on Configuring Blocks) that will only display on the front page. We will add the new collection list block below the existing frontpage view that lists items.
Name the view and select Create a block. Give the block a title and decide how you want it to display (Grid, Table, List, Paging). To progress, select Save and edit.
5. Customize the view format and sorting as required. 6. Add a filter criteria to only show the Islandora model type of 'Collection' and Save the view.
7. To place the view on front page, the new block must be added to the 'Main page content' area (using 'Block layout') and set to display on the front page. 1. Navigate to Administration >> Structure >> Block layout (/admin/structure/block). Under Main content select Place block
\nRestrict the block to only display on the frontpage by adding the text '' to the Page vertical tab. Then select Save block.
4. Review the block placement and move if required.
8. The 'Collection list' now only displays on the front page. It displays below the Main page content.
Islandora Quick Lessons
Learn more with videos on Basic Views and Advanced Views.
"},{"location":"tutorials/how-to-create-collection/","title":"How to create and add to a Collection","text":"This how-to demonstrates creating a collection and adding items to it in the Islandora Starter Site. For more about collections, see Concept: Collection.
"},{"location":"tutorials/how-to-create-collection/#introduction","title":"Introduction","text":"In the Islandora Starter Site, nodes that have \"Collection\" in their Model field will show a view of their member (\"child\") objects. A member is any object that is \"Member of\" that object, and can be added via the \"Children\" tab. This is the mechanism in the Islandora Starter Site, and your individual instances may vary.
Collections and Deleting
Collections and their members are independent of each other, and removing something from a collection does not delete it. Similarly, deleting a collection does not delete its members.
"},{"location":"tutorials/how-to-create-collection/#creating-a-collection","title":"Creating a Collection","text":"When logged in, click Manage > Content on the admin toolbar. Then, click on Add content.
Then click on 'Repository Item' to give your collection the default metadata profile for Islandora.
Fill out the form.
At the top of the form, select \"Collection\" from the Model dropdown list.
When done filling out the form, click Save.
The collection has been created. Now let's add some members to this collection.
"},{"location":"tutorials/how-to-create-collection/#add-existing-items-to-a-collection","title":"Add Existing Items to a Collection","text":"To populate a collection with existing items, return to any existing content and click on its Edit tab. This brings up the form for this item.
Scroll down to the bottom of the form to find the System section. In the Member of field, start typing in the name of the collection this item should belong to. Select the name of the collection you want from the autocomplete. It is important that you select it from the dropdown, not just type in the correct title, because selecting it causes the node id to appear beside the title and allows Drupal to create a relationship.
The correct collection is now selected. Click Save when ready.
To confirm, return to the collection and verify the new item appears in the collection's list of members.
"},{"location":"tutorials/how-to-create-collection/#add-a-new-item-as-a-member-of-a-collection","title":"Add a New Item as a Member of a Collection","text":"To create an item and add it as a member to a collection in one step, visit a collection and click on its Children tab. From the Children tab, you can manage the members of a collection and perform actions on them.
Click on the +Add Child button, and then select Repository Item as your new item's Content Type. Only content types that have the field_member_of field will be available from this list.
You are taken to the creation form for a Repository Item, but if you scroll down to the \"System\" section, you should see the widget for \"Member Of\" is already filled out for you with the appropriate collection.
Click 'Save' at the end of the form to create the new item and add it as a member of the collection.
Islandora Quick Lessons
Learn more with this video on Making a Collection.
"},{"location":"tutorials/switch-homepage-to-twig/","title":"Format Homepage with TWIG","text":""},{"location":"tutorials/switch-homepage-to-twig/#twig-debugging","title":"TWIG Debugging","text":"It's helpful to identify which TWIGs are available to use and where they're stored but not required use TWIGs to format the homepage.
# Copy the default service.\n$ cp web/sites/default/default.services.yml web/sites/default/services.yml\n\n# fix permissions (just in case)\n$ chown nginx:nginx web/sites/default/services.yml\n\n# I use nano to edit but you can pick whichever editor you want.\n# For this example we'll install the editor\n$ apk add nano\n\n# Now open the newly created service file and set these 3 values under the TWIG config section.\n$ nano web/sites/default/services.yml\n\n...yml\ntwig.config:\n debug: true\nauto_reload: true\ncache: true\n# Now save and exit (in NANO it's CTRL + x)\nCopy the default TWIG into your theme's template directory.
$ cp web/themes/contrib/bootstrap/templates/node/node.html.twig web/themes/contrib/solid/templates/node--6--full.html.twig\n\n# Clear cache\n$ drush cr\nX next to the loaded TWIG file. Please note that the file name corresponds to the node number. To use the URL alias instead of the node ID requires additional work. Here's a tutorial on this topic. <!-- FILE NAME SUGGESTIONS:\n x node--6--full.html.twig\n * node--6.html.twig\n * node--page--full.html.twig\n * node--page.html.twig\n * node--full.html.twig\n * node.html.twig\n-->\nNow edit the TWIG file (web/themes/contrib/solid/templates/node--6--full.html.twig) to say whatever you want, and it should show up immediately without needing to clear cache.
"},{"location":"tutorials/switch-homepage-to-twig/#clean-up","title":"Clean up","text":"Don't forget to turn off TWIG debugging in config file (web/sites/default/services.yml). This will likely have unexpected consequences on production system performance.
twig.config:\n debug: false\n auto_reload: false\nThis page is about controlling who can view or edit your Islandora content (metadata and binaries).
Islandora recommends using Drupal's access control features. Contributed modules such as those described below, can provide additional flexibility and configurability. However, these only apply when content is accessed through Drupal, and are not applied to data in Fedora, the Triplestore, or Solr.
Known strategies for implementing access control in Drupal include:
Exposed Endpoints
If access control is a concern, you will want to lock down access to the following services that are part of the Islandora Suite:
as anyone accessing data through those services are bypassing Drupal's access control. In all out-of-the-box sandboxes and demo instances, these services are wide open.
"},{"location":"user-documentation/access-control/#access-control-in-core-drupal","title":"Access control in Core Drupal","text":"Summary: Core Drupal's access control features include the \"published\"/\"unpublished\" states and some basic permissions that can be granted globally or at the bundle level.
Drupal's concepts of \"published\" and \"unpublished\" for nodes and media apply to Islandora just as they do in Drupal. Usually, published content is visible to the entire world (without any authentication), while unpublished content can only be accessed by selected users - usually administrators, other privileged roles, and the user who created the content. This is configurable through the Drupal Permissions interface.
All Drupal permissions are granted to roles, not individual users. Read more documentation on Users, roles, and permissions.
The extent of the configurability of Drupal Core's access control (excluding revisions) is provided by the following permissions:
View permissions (apply to all content or all media regardless of bundle) - View all published content - View own unpublished content - View (published) media - View own unpublished media \"Editing\" permissions (can be granted for a specific bundle, or for all nodes or media): - Create new content - Edit (your) own content - Edit any(one's) content - Delete (your) own content - Delete any(one's) content Administrative permission (includes all the above and more, give to trusted users only): - Administer contentThese are the basic access control options built into Drupal Core. There are many contributed modules that do access control, some are described below. Before using access control modules, please see Comparison and Overview of Access Control modules on drupal.org.
Contributed modules are required for the following cases:
Given the [Collection] \u2192 Node \u2192 Media \u2192 File structure of Islandora resources, it is reasonable to wonder if access to a Node, Media, or File is influenced by the permissions on its \"parent\" resource.
There is not yet a mechanism for Drupal's access control to be \"inherited\" from parent resources to child resources. An exception is non-public files, as described below.
"},{"location":"user-documentation/access-control/#sidebar-access-to-files-in-drupal","title":"Sidebar: Access to files in Drupal","text":"Drupal's access control applies to files except those stored in Drupal's public filesystem.
If you upload a file to Drupal, it's usually stored in a field on an entity. In Islandora, that entity is a Media. Drupal's access checks mean that a user must have permission to view the Media, and (if configured) the file field on that media, to see the file. However, file fields can be configured with various forms of file storage, and the file storage determines whether access control checks are applied at all. Files stored in Drupal's public filesystem are always public, viewable by anyone in the world.
Drupal's access control applies to files accessed through Drupal where the file storage is:
Direct access to file storage
Files stored using Flysystem are actually located on a separate service. It is the administrator's responsibility to implement appropriate access control on the target storage system, as there is a way to link directly to files in Fedora, S3, or Dropbox. These direct links bypass Drupal and therefore Drupal's access control.
The filesystem used by a field is usually configured through \"Manage Fields\" on the appropriate entity. However, it is possible for a field to be configured to use a certain filesystem, but for actual files created through back-end processes to use a different one. For example, derivatives generated by Islandora as configured in the Islandora Starter Site are written to the public filesystem, despite the field configuration which points to Fedora.
To determine the actual location of a file, right-click the existing file in the Media Edit page and select \"copy location\". A file that is saved in flysystem will include /_flysystem/ in the URL. These links (you can test it!) only work if you have permission to view the media. A file with /sites/default/files/ is probably publicly accessible (this is the public filesystem on most Drupal instances).
In the Islandora Starter Site, All Derivatives Are Public
Out of the box using the Islandora Starter Site, derivatives are created for all Islandora media (as long as the conditions of the contexts are met), and they are stored in the public Drupal filesystem. This happens even if the media and/or node are not published.
"},{"location":"user-documentation/access-control/#access-control-in-solr","title":"Access control in Solr","text":"The Search API Module, which connects Drupal to Solr, provides configuration so that access considerations are respected by the Drupal solr results display. The \"Processors\" configuration (e.g. at admin/config/search/search-api/index/default_solr_index/processors) provides checkboxes for the following options:
Both are enabled out of the box in the Islandora Starter Site. This will ensure that queries through Drupal never show content that the active user shouldn't see, as well as preventing information about unpublished entities from ever being entered into Solr.
Solr Admin Client
Anyone with access to the Solr Admin Client may see the full contents of the index, regardless of permissions. If this is not desired, you should restrict access to the Solr endpoint and GUI. By default, the Islandora Playbook exposes Solr at http://localhost:8983/solr/. ISLE-DC on the other hand can expose or not expose individual services, including Solr. Since December 2021 this has been available as an option in the .env file and since that time, Solr (and some other services) are not exposed by default.
\"The Group module allows you to create arbitrary collections of your content and users on your site and grant access control permissions on those collections.\"
Opinion
Group is one of the more hefty modules, and is difficult to learn. It is good if you have static or semi-static groups of users who need access to a static or semi-static group of content, but sharing an item with an arbitrary group of users is cumbersome.
"},{"location":"user-documentation/access-control/#permissions-by-term-contributed-module","title":"Permissions By Term (contributed module)","text":"\"The Permissions by Term module extends Drupal by functionality for restricting view and edit access to single nodes via taxonomy terms. [...] Taxonomy term permissions can be coupled to specific user accounts and/or user roles.\" By default, this module only affects nodes. To enable Permissions by Term for Media and other entities, enable the \"Permissions by Entity\" submodule.
The Islandora Starter Site includes an empty vocabulary called \"Islandora Access\", which is intended to hold such taxonomy terms. However, permissions_by_term or a similar mechanism to control access must be installed and configured on your own.
This module is known to work, and appears to be supported well by the Drupal community.
Warning
This module is known in the Islandora community to cause performance degradation when large numbers of nodes are involved. Seth Shaw describes this in a blog post, Content Access Control Solutions Investigation.
Examples Wanted
Do you have experience setting up Permissions By Term? We'd love some illustrative examples here.
"},{"location":"user-documentation/access-control/#field-permissions-contributed-module","title":"Field Permissions (contributed module)","text":"\"The Field Permissions module allows site administrators to set field-level permissions to edit, view and create fields on any entity.\"
"},{"location":"user-documentation/access-control/#other-contributed-modules","title":"Other contributed modules","text":"Workflow Participants allows for granting permissions on individual nodes or media to individual users, as an extension to the Workflow suite of modules. However, it is not well-supported and the \"manage workflow participants\" permission should not be given to untrusted users as it may grant users the ability to add participants even to content they are otherwise not able to see or edit.
"},{"location":"user-documentation/access-control/#access-control-in-fedora","title":"Access control in Fedora","text":"In Fedora, it is possible to control access to resources using Access Control Lists (ACLs) per the Fedora API Specification. ACLs are inherited through Fedora's LDP Containment relationships.
Islandora does not create customized ACLs when syncing content into Fedora.
Islandora does not create Fedora resources in hierarchies that use LDP containment relationships.
At this time, access control in Fedora is NOT reflective of access control in Drupal. In the sandboxes and demo installations, all resources in Fedora are open for the world to view. This includes unpublished nodes and media, as well as the files uploaded into Fedora. Access controls configured in Drupal are not synced in any way to Fedora.
Sites concerned with access control wish to \"lock down\" their Fedora to only be accessible through Drupal.
"},{"location":"user-documentation/access-control/#access-control-in-the-triplestore","title":"Access control in the Triplestore","text":"Anyone with access to Blazegraph can access all the repository information within Blazegraph. If this is not desired, you should restrict access to the Blazegraph endpoint and GUI. By default, the Islandora Playbook exposes Blazegraph at http://localhost:8080/bigdata/. ISLE-DC on the other hand can expose or not expose individual services, including Blazegraph. Since December 2021 this has been available as an option in the .env file and since that time, Blazegraph (and some other services) are not exposed by default.
Meta-Issue: Access Restrictions and Embargoes
"},{"location":"user-documentation/accessibility/","title":"Accessibility","text":"Accessibility is the ability of a site to be used fully by all users, including those using screen reader technologies or keyboard navigation.
"},{"location":"user-documentation/accessibility/#drupal-documentation","title":"Drupal documentation","text":"Islandora's accessibility features are currently provided by Drupal and contributed Drupal modules. These pages within the Accessibility section of the Drupal documentation can provide guidance:
Much of the accessibility of a website is dependent on how specific HTML tags and attributes are used, thus falls largely into the realm of Drupal Themes. The Olivero theme and the Claro admin theme were designed by the Drupal community with accessibility as a guiding principle.
"},{"location":"user-documentation/accessibility/#automatic-alt-text","title":"Automatic alt-text","text":"When creating image media, alt-text is a required attribute. An Islandora function currently automatically populates the alt-text, if that media is the \"media of\" a node, with the respective node's title.
This is not considered good accessibility as the node's title will likely be already on the page, and it doesn't add anything to describing the image itself. Please fill out the alt text with something meaningful when adding image content to Islandora.
"},{"location":"user-documentation/advanced-search/","title":"Islandora advanced search","text":"Advanced Search adds additional functionality beyond the basic Solr search. It enables the use of Ajax with search blocks, facets, and search results.
"},{"location":"user-documentation/advanced-search/#requirements","title":"Requirements","text":"Use composer to download the required libraries and modules.
composer require drupal/facets \"^1.3\"\ncomposer require drupal/search_api_solr \"^4.1\"\ncomposer require drupal/search_api \"^1.5\"\nHowever, for reference, islandora_advanced_search requires the following drupal modules:
To download/enable just this module, use the following from the command line:
composer require islandora/islandora\ndrush en islandora_advanced_search\nYou can set the following configuration at Administration >> Configuration >> Advanced Search Settings (admin/config/search/advanced):
"},{"location":"user-documentation/advanced-search/#configuring-solr","title":"Configuring Solr","text":"Please review Configure Search before continuing. The following assumes you already have a working Solr and the Drupal Search API setup.
"},{"location":"user-documentation/advanced-search/#configure-collection-search","title":"Configure collection search","text":"To support collection based searches you need to index the field_member_of for every repository item as well define a new field that captures the full hierarchy of field_member_of for each repository item.
Add a new Content solr field field_descendant_of to the solr index at admin/config/search/search-api/index/default_solr_index/fields.
Then under admin/config/search/search-api/index/default_solr_index/processors enable Index hierarchy and set up the new field to index the hierarchy.
The field can now be used limit a search to all the descendants of a given object.
Re-Indexing
You may have to re-index to make sure the field is populated.
"},{"location":"user-documentation/advanced-search/#configure-views","title":"Configure views","text":"The configuration of views is outside the scope of this document, please read the Drupal Documentation, as well as the Search API Documentation.
"},{"location":"user-documentation/advanced-search/#collection-search","title":"Collection search","text":"It will be typical that you require the following Relationships and Contextual Filters when setting up a search view to enable Collection Search searches.
Here a relationship is set up with Member Of field and we have two contextual filters:
field_member_of (Direct descendants of the Entity)field_descendant_of (All descendants of the Entity)Both of these filters are configured the exact same way.
These filters are toggled by the Advanced Search block to allow the search to include all descendants or just direct descendants (documented below).
"},{"location":"user-documentation/advanced-search/#paging","title":"Paging","text":"The paging options specified here can have an effect on the pager block (documented below).
"},{"location":"user-documentation/advanced-search/#sorting","title":"Sorting","text":"Additionally, the fields listed as Sort Criteria as Exposed will be made available in the pager block (documented below).
"},{"location":"user-documentation/advanced-search/#configure-facets","title":"Configure facets","text":"The facets can be configured at admin/config/search/facets. Facets are linked to a Source which is a Search API View Display so it will be typically to have to duplicate your configuration for a given facet across each of the displays where you want it to show up.
To be able to display exclude facet links as well as include links in the facets block we have to duplicate the configuration for the facet like so.
Both the include / exclude facets must use the widget List of links that allow the user to include / exclude facets
The excluded facet also needs the following settings to appear and function correctly.
The URL alias must match the same value as the included facet except it must be prefixed with ~ character that is what links to the two facets to each other.
And it must also explicitly be set to exclude:
You may also want to enable Hide active items and Hide non-narrowing results for a cleaner presentation of facets.
"},{"location":"user-documentation/advanced-search/#configure-blocks","title":"Configure blocks","text":"For each block type:
There will be one block per View Display. The block should be limited to only appear when the view it was derived from is also being displayed on the same page.
This requires configuring the visibility of the block as appropriate. For collection based searches be sure to limit the display of the Facets block to the models you want to display the search on, e.g:
For any valid search field, you can drag / drop and reorder the fields to display in the advanced search form on. The configuration resides on the block so this can differ across views / displays if need be. Additionally, if the View the block was derived from has multiple contextual filters you can choose which one corresponds to direct children, this will enable the recursive search checkbox.
"},{"location":"user-documentation/content-models/","title":"Content models in Islandora","text":""},{"location":"user-documentation/content-models/#resource-nodes","title":"Resource Nodes","text":"This section describes the Islandora concept of a Resource Node. For a step-by-step demonstration, see the tutorial Create a resource node.
A resource node holds the descriptive metadata for an Islandora object, as well as groups together the various files that are part of the object for preservation or display, such as the original file and various derivative files generated from it.
The model for exactly what constitutes an object in Islandora is flexible and can be adapted to the needs of specific users. For example, the Islandora Starter Site configuration considers an object as a resource node of the type \"Repository Item\" which contains descriptive metadata about the object. Attached to that Node are one or more Media, each representing a file that is part of this object, such as \"Original File\", \"Thumbnail\", \"Preservation Master\", etc. With this model, every original file uploaded into Islandora has its own resource node.
Multi-file Media configurations also attach Media to a parent node, but allow for that node to be represented by multiple \"Original File\"s. In this model, a Media contains the original file as well as any derivative files created from it (thumbnail, service file, etc.).
For an example of where these two different approaches could apply, the basic configuration might make sense for a book that has rich page-level metadata, so that each page would be its own Node with its own metadata record; the multi-file media configuration might be a better fit for a book that does not have page-level metadata (except an ordering or page numbers), so that each Media would represent one page, and all pages (Media) would be attached to a single parent Node/metadata record for the entire book.
As we learned in the introduction, objects in an Islandora repository are represented as a combination of resource nodes, media, and files in Drupal. Because of this, their metadata profile, display, form (and much more) are configurable through the Drupal UI. This gives repository administrators a huge degree of control over their repository without any need for coding. Much more so than ever before. And since we're using a core Drupal solution for modeling our resource nodes and media, compatibility with third-party modules is virtually guaranteed. This opens up a plethora of solutions from the Drupal community that will save you untold time and effort when implementing your repository with Islandora.
"},{"location":"user-documentation/content-models/#properties","title":"Properties","text":"Resource nodes, as Drupal nodes, have some common basic properties regardless of content type. These properties are not fields. This means that they cannot be removed and have limited configurability. Their name, what type of data they hold, etc... are all baked in. Here's an example of the basic properties on nodes:
nid: 1\nuid: 1\ntitle: \"I am an Islandora object\"\ncreated: 1550703004\nchanged: 1550703512\nuuid: 02932f2c-e4c2-4b7e-95e1-4aceab78c638\ntype: islandora_object\nstatus: 1\nAs you can see, it's all system data used at the Drupal level to track the basics.
Property Value nid The local ID for the node uid The ID of the Drupal user who created the node uuid The global ID for any entity title The title for the node created Timestamp of when the node was created changed Timestamp of when the node was last updated type Content type (e.g. which group of fields are present on the node) status Published, unpublished, etc...Compared to Islandora Legacy
These node properties are analogous to following Islandora Legacy object properties:
Islandora Legacy Islandora owner uid dc.title title PID uuid status statusThe small amount of configurability available for these properties is found on the content type editing form where a user can choose to change the label of the title field, whether to display author information on the node's page, etcetera. These settings will only apply to nodes of that particular content type.
To view all of a node's property and field values administrators can use the 'Devel' tab's 'Load' section:
"},{"location":"user-documentation/content-models/#fields","title":"Fields","text":"In addition to the basic node properties identified above, resource nodes (like all Drupal nodes) can have fields. Most of what we would think of as descriptive metadata is stored as fields. Resource nodes use 'content types' to define a specific set of required and optional fields it has; we can think of content types as metadata profiles for our objects. For example, you might have a content type for a set of repository objects that have very specialized metadata requirements but another content type for generic repository objects that share a more general set of metadata fields. A resource node's content type is set on its creation and is immutable. The section on metadata describes in more detail how fields on Islandora objects work.
Configuring fields (adding, editing, removing) is usually done through the Manage > Content types interface, as is described in the tutorial, Create/Update a Content Type.
Islandora has a notion of a content model, which is used to identify what type of content is being represented by a node (e.g. an image, a video, a collection of other items, etc...). This is done using a special field, Model, which accepts taxonomy terms from the Islandora Models vocabulary. By applying a term from the Islandora Models vocabulary to a node, Islandora will become aware of how to handle the node in response to certain events, like choosing a viewer or generating derivatives.
Compared to Islandora Legacy
Content models in Islandora Legacy were immutable and contained restrictions as to what types of datastreams could be associated with an object. Islandora imposes no such restrictions. Content models can be changed at any time, and they in no way dictate what types of media can be associated with a node.
"},{"location":"user-documentation/content-models/#media","title":"Media","text":"All resource nodes can be linked to any number of media. The media associated with a resource node can be managed using the \"Media\" tab when viewing a node. Much like the \"Members\" tab, Actions can be performed in bulk using the checkboxes and Actions dropdown.
See the media section for more details.
"},{"location":"user-documentation/content-models/#display-modes","title":"Display modes","text":"Drupal uses \"display modes\" (also called \"view modes\") as alternative ways to present content to users. You may be familiar with the \"full\" and \"teaser\" versions of nodes, which are rendered using two corresponding display modes. Islandora makes use of display modes to control how media content is displayed. Islandora Starter Site provides two display modes for Media, one which renders the OpenSeadragon viewer and the other which renders the pdf.js viewer. These two display modes can be enabled by using \"Display hints\" in the node edit form, or you can configure Islandora to use a specific display mode for all media based on the file's Mime type. Both methods make use of Contexts.
To set the display mode on the resource node's edit form, select the display mode you want to use for that node in the Display hints field:
Due to the associated Context configurations (\"OpenSeadragon\" and \"PDFjs Viewer\") that are shipped with the Islandora Starter Site, the selected display mode will then be used when the resource node's page is rendered.
At a global level, there are a couple of ways to tell Drupal to use the PDFjs viewer to render the content of the media field whenever the media has a Mime type of application/pdf.
The first way is to edit the \"PDFjs Viewer\" Context. By default, this Context tells Drupal to use the PDFjs viewer if the node has the term \"PDFjs\" (yes, that's a taxonomy term):
If you add the Condition \"Node has Media with Mime type\" and configure it to use application/pdf as the Mime type, like this:
Context will use whichever Condition applies (as long as you don't check \"Require all conditions\"). That is, if the \"PDFjs\" display hint option in the node edit form is checked, or if the node's media has a Mime type of application/pdf, the media content will be rendered using the PDFjs viewer.
The second way to use the media's Mime type to render its content with the PDFjs viewer is to create a separate Context that will detect the media's Mime type and use the configured display mode automatically. To do this, create a new Context. Add a \"Node has Media with Mime type\" condition and specify the Mime type, and then add a \"Change View mode\" Reaction that selects the desired display mode:
Finally, save your Context. From that point on, whenever the media for a node has the configured Mime type, Drupal will render the media using the corresponding display mode.
The node-level and global approaches are not exclusive to one another. One Context can override another depending on the order of execution. Contexts are applied in the order they are displayed on the Contexts page, which is editable through a drag-and-drop interface. Whichever Condition appears last in the list of Contexts between the node-level Condition (which in this case is the \"Node has term\" condition) the global Condition (which is \"Node has Media with Mime type\"), that one will override the other. An example of having the display mode specified in the node edit form intentionally override the display mode based on Mime type is to have media with the image/jp2 mime-type configured to use the OpenSeadragon viewer, but to manually select the OpenSeadragon display mode for nodes with JPEG media (for example, a very large JPEG image of a map, where the OpenSeadragon's pan and zoom features would be useful).
Islandora has a notion of membership, which is used to create a parent/child relationship between nodes. Membership is denoted using another special field, \"Member Of\". This is used to create the link between members and their parent collection, pages and their book (\"paged content\"), or members of a compound object and the compound object itself.
Any two nodes can be related in this way, though typically, the parent node has a content model of Collection or Paged Content (see their respective pages for more details). The \"Member Of\" field can hold multiple references, so it is possible for a single child to belong to multiple parents, but may also complicate the creation of breadcrumbs.
Compared to Islandora Legacy
In Islandora Legacy, there was a distinction between belonging to a collection and belonging to a compound object. In Islandora, this distinction is not present. Since all nodes can have members , essentially every node has the potential to be a compound object or collection.
Child v. Member
Islandora uses the \"child\" and \"member\" descriptor for resource nodes that store a reference to another resource node in the \"Member Of\" field interchangeably. Administrators will more often see the \"member\" terminology more often while front-end users will usually see \"child\" terminology.
For any node, its Children tab can be used to see all its members. You can also perform Actions in bulk on members using the checkboxes and the Actions dropdown as well as clicking on the Reorder Children tab to adjust the order in which they display.
"},{"location":"user-documentation/content-models/#more-information","title":"More information","text":"The following pages expand on the concepts discussed above:
This document was originally developed by Alex Kent and has been adapted for general use by the Islandora community.
[^1] In the Islandora Starter Site, this is the field_model field, which is populated by taxonomy terms in the islandora_models taxonomy vocabulary provided by the islandora_core_feature submodule of Islandora/islandora
[^2] In the Islandora Starter Site, this is the field_member_of field.
The conventional Islandora Legacy definition of an object is a file loaded in the repository with associated derivatives. In Islandora Legacy, objects (video files, audio files, PDFs, etc.) are loaded through the user interface, and Datastreams are generated automatically. These consist of access and display copies, the metadata, OCH/HOCR, technical metadata, and more. All of these Datastreams are directly connected to the object and accessed through the admin interface.
In Islandora, the traditional Islandora Legacy objects (video files, audio files, etc. that were represented in different content models) are now Drupal nodes. Islandora object nodes are a special kind of Drupal node, distinct from nodes that exist for other content types such as a blog post, an article, a page (like the About page on a site), and others. These Islandora objects are still loaded through the interface and described with the data entry form, and derivatives are still generated. However, the Datastreams are no longer connected to the original object in the same immutable way. Each of these Datastreams can be manipulated through Drupal by non-developers. You can create a variety of ways to view this metadata and information related to the objects. Doing so requires knowledge of Drupal 8, but this essentially means that there are many ways to view the metadata and access the related objects in Islandora.
In Islandora it is therefore helpful to think of objects as resource nodes. The term reflects the new nature of objects in Islandora. A resource node does not just refer to the individual object file, but encompasses multiple elements that all relate to each other, even if they are no longer directly connected like objects in Islandora Legacy.
The typical elements of a resource node:
These resource nodes are what the librarian, student, archivist, technician, or general non-developer creates through the data entry form. It is possible to configure all elements of a resource node in Islandora through Drupal. This fact allows control over how one accesses the node and how nodes are displayed and discovered online by non-developers. It also allows a repository to take full advantage of all third-party Drupal modules, themes, and distributions available.
"},{"location":"user-documentation/content-types/","title":"Creating and updating content types","text":""},{"location":"user-documentation/content-types/#overview","title":"Overview","text":"Since metadata in Islandora is stored as fields in Nodes, the standard Drupal Content Types system provides our 'ingest forms'. For more information about Content Types in general, please see Content Types in Drupal. If you are already familiar with Drupal Field UI, you\u2019re already well-equipped to create and modify your own ingest forms in Islandora.
This page will address how to create and modify ingest forms by editing fields and form display settings on Content Types via the graphical user interface (GUI). This page will also cover editing the RDF mapping to accommodate changes to fields.
Islandora forms are Drupal forms, and for help working with forms via the API, please check out the Further Reading section for links to more advanced Drupal documentation.
"},{"location":"user-documentation/content-types/#before-you-start","title":"Before you start","text":"If you have deployed your Islandora with the Islandora Starter Site configuration, you will already have a Repository Item content type available, with pre-configured fields and repository behaviours.
There are multiple tabs with different options to configure your Content Type:
Changes not displaying?
If you make changes under Manage display and don't see them reflected in your Node, double check that you have edited the right view mode
This example adds a new field where a user can indicate if the repository item needs to be reviewed:
The new field has been added:
It appears in the ingest form when creating a new repository object. To test this, go to Content >> Add content >> Repository item:
RDF Mappings
New fields, except for Typed Relation fields, are not automatically indexed in Fedora and the triple-store. Update the Content Type's RDF Mapping to enable indexing the field (see below).
Search
New fields will not automatically be searchable. They need to be added to the Solr index configuration. See the 'Setup and Configure Search' page for more information.
Context
To add new behavior based on the results of this new field, check out Context.
"},{"location":"user-documentation/content-types/#change-the-form-display","title":"Change the form display","text":"To change where in the form a field is displayed, go to the Admin menu, return to Structure >> Content Types, and find the Repository Item content type again. Select Manage form display from the dropdown menu or select the Manage form display tab.
When creating a new Repository Item, the new field appears at the top, as a set of radio buttons.
"},{"location":"user-documentation/content-types/#change-the-content-display","title":"Change the content display","text":"Finally, change how the results of this example field are displayed. Initially the new field shows up at the bottom of repository object pages:
In the Admin menu, return to Structure >> Content Types and find the Repository Item content type again. Select Manage display from the dropdown menu or select the Manage display tab.
To create your own custom content type from scratch, please refer to this guide on Drupal.org.
Your custom content types can contain whatever fields you like, but there are two mandatory fields that all Islandora content types should contain:
In order for a custom content type to be considered an Islandora Object, it needs to have the field \"Member of\" ('field_member_of'). This allows it to be included in contexts that have the \"Node is an Islandora node\" condition. Nodes that have this field will automatically be synced to Fedora and indexed by the triple store if you are using the context provided by the Islandora Starter Site. Having this field present in your content type also gives you tabs for adding children and media when viewing an item of that content type.
The other mandatory field is \"Model\" ('field_model'). This is used in several of the contexts that the Islandora Starter Site provides. This field determines how Islandora objects are displayed, and how media derivatives are created.
RDF mapping aligns Drupal fields with RDF ontology properties. For example, the title field of a content model can be mapped to dcterms:title and/or schema:title. In Islandora, triples expressed by these mappings get synced to Fedora and indexed in the Blazegraph triplestore. RDF mappings are defined/stored in Drupal as a YAML file (to learn more about YAML, there are several tutorials on the web. Currently, Drupal 8 does not have a UI to create/update RDF mappings to ontologies other than Schema.org. This requires repository managers to update the configuration files themselves. Consider using the RDF mappings included in the Islandora Starter Site as templates by copying and modifying one to meet your needs.
The Drupal 8 Configuration Synchronization export (e.g. http://localhost:8000/admin/config/development/configuration/single/export) and import (e.g. http://localhost:8000/admin/config/development/configuration/single/import) can be used to get a copy of the mappings for editing in a text editor before being uploaded again. Alternatively, a repository manager can update the configuration on the server and use Features to import the edits.
An RDF mapping configuration file has two main areas: the mapping's metadata and the mapping itself. Most of the mapping's metadata should be left alone unless you are creating a brand-new mapping for a new Content Type or Taxonomy Vocabulary. A partial example from islandora_default's islandora_object (Repository Item) is included below:
langcode: en\nstatus: true\ndependencies:\n config:\n - node.type.islandora_object\n enforced:\n module:\n - islandora_demo\n module:\n - node\nid: node.islandora_object\ntargetEntityType: node\nbundle: islandora_object\ntypes:\n - 'pcdm:Object'\nfieldMappings:\n title:\n properties:\n - 'dc:title'\n field_alternative_title:\n properties:\n - 'dc:alternative'\n field_edtf_date:\n properties:\n - 'dc:date'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_description:\n properties:\n - 'dc:description'\nThe required mapping metadata fields when creating a brand-new mapping include the id, status, targetEntityType, and bundle. (uuid and _core, not seen in the example above but may be present in exported copies, will be added by Drupal automatically.) bundle is the machine name for the Content Type or Taxonomy Vocabulary you are creating the mapping for. targetEntityType is node for Content Types or taxonomy_term for Taxonomy Vocabularies. The id configuration is a concatenation of target entity type and bundle ('node' and 'islandora_object' in the example above). The id is also used to name the configuration file: e.g. rdf.mapping.node.islandora_object.yml is rdf.mapping. plus the id (node.islandora_object) and then .yml.
The mapping itself consists of the types' and fieldMappings configurations.
All the mappings use RDF namespaces instead of fully-qualified URIs. For example, the type for islandora_object is entered in the RDF config as pcdm:Object instead of http://pcdm.org/models#Object. The available namespaces are defined in module hooks (hook_rdf_namespaces) but can also be entered manually in a configuration interface. Repository managers wanting to add additional namespaces need to go to Configuration > Search and Metadata > JSONLD and enter their desired namespaces in the \"Additional RDF Namespaces\" box.
Namespaces currently supported (ordered by the module that supplies them) include:
The types corresponds to the rdf:type predicate (which corresponds to JSON-LD's @type) and can have multiple values. This type value will be applied to every node or taxonomy term using the mapped content type or vocabulary.
In some cases a repository may want a node or taxonomy term's rdf:type to be configurable. For example, the Corporate Body Vocabulary (provided by the Controlled Access Terms Default Configuration module) has schema:Organization set as the default type in the RDF mapping. However, more granular types may apply to one organization and not another, such as schema:GovernmentOrganization or schema:Corporation. The alter_jsonld_type Context reaction allows Content Types and Taxonomy Vocabularies to add a field's values as rdf:types to its JSON-LD serialization (the format used to index a node or taxonomy term in Fedora and the triple-store).
fieldMappings specifies the fields to be included, their RDF property mappings, and any necessary data converters (the datatype_callback). One field can be mapped to more than one RDF property by adding them to the field's properties list. The datatype_callback is defined by the 'callable' key and the fully qualified static method used to convert it to the desired data format. For example, fields of the Drupal datetime type need to be converted to ISO 8601 values, so we use the Drupal\\rdf\\CommonDataConverter::dateIso8601Value function to perform the conversion.
Islandora Quick Lessons
Learn more with this video on Customizing a Form.
"},{"location":"user-documentation/context/","title":"Context","text":""},{"location":"user-documentation/context/#context-in-islandora","title":"Context in Islandora","text":"Context is a Drupal contrib module that allows configuration of \"if this, then that\" logic using an administrative user interface. Context defines \"conditions\" and \"reactions\" to enact the \"if this [condition], then that [reaction]\" logic. Some Islandora Legacy repositories use the community-contributed Islandora Context module to apply this sort of logic to Islandora objects. An example Context from Islandora Legacy is
If an object's namespace is 'customnamespace', render the block containing the right's statement \"Please contact Special Collections before reusing this item\".
The Context project page on drupal.org says \"You can think of each context as representing a 'section' of your site\", but that doesn't tell the whole story, since it gives the impression that Context is all about defining subsites. Context in Drupal 8 (and above) is much more powerful than in Drupal 7 due to how it interacts with the rest of Drupal (specifically, through Drupal 8's new plugin API). Because of this increased flexibility and power, and because Context provides a ready-made user interface for site administrators to combine conditions and reactions, Islandora uses Context to drive the logic underlying many important aspects of Islandora, including which derivatives get generated and how objects are displayed. In Islandora Legacy, many of these things are managed (and often hard-coded) within solution packs and utility modules.
"},{"location":"user-documentation/context/#an-example","title":"An example","text":"Let's look at the example of how Context can determine how an object is displayed. Drupal 8 has the idea of \"view modes\", which allow site builders to choose what happens when an object is viewed by the user (it has nothing to do with Drupal Views). In the node edit form for Islandora objects, there is a checkbox that, if checked, tells Drupal to render the image using the OpenSeadragon viewer:
This functionality is accomplished via the \"OpenSeadragon\" Context, which, as its configuration suggests, checks as its condition whether the node as the \"OpenSeadragon\" tag and if so, reacts by using the view mode \"OpenSeadragon\":
"},{"location":"user-documentation/context/#context-and-actions","title":"Context and actions","text":"Context in Drupal 8 uses Drupal Actions to execute the \"then that\" part of their logic. An implication of this is some configuration options relevant to the outcome of the \"then that\" reaction are configured in the Action admin interface (located at admin/config/system/actions), not within the Context admin interface (located at admin/structure/context). While this makes for a clunky admin experience, the advantage is that it allows the reuse of a single piece of Drupal code in multiple Contexts.
One important group of functionality in Islandora repositories that admins might want to control is what types of media get persisted to Fedora. Islandora allows administrators to choose what types of media get persisted to Fedora and what types of media get persisted to the Drupal public file system. For example, you may want to persist your thumbnails to Fedora along with your \"Preservation Master\" and \"Original File\" files. This can be accomplished using Context.
By default, Islandora stores thumbnails in Drupal's public file system, not in Fedora. Thumbnails are automatically generated by a Context (the \"Image Service File\" Context to be specific), but if you want to change the file system where your thumbnails are stored to the Fedora file system, you need to change that setting in the \"Image - Generate a thumbnail from a service file\" Action, not in the Context that uses that action:
This configuration should be done as part of your initial Islandora setup, since there is currently no way to move thumbnails that were persisted to Drupal's public file system to Fedora, or vice versa. But, at least site admins can choose one or the other using this Action.
You can create custom Actions and then use them in Contexts. For example, if you want to present a message to the user every time they upload a media tagged as \"Preservation Master File\", create an Action of type \"Display a message to the user\" and enter your message:
Then, create a Context that uses this Action as its Reaction, and as its Condition, uses \"Media has term\" with a value of \"Preservation Master File\":
Now, when a user creates a Media and tags it as a \"Preservation Master File\", they will be shown the message you created in your Action:
"},{"location":"user-documentation/context/#tips-on-using-islandora-specific-contexts","title":"Tips on using Islandora-specific contexts","text":"Islandora provides some Conditions and Reactions that aren't available on other Drupal sites. When using them, you may find the following tips useful:
,) after the first term in the autocomplete field and continue typing.Most Islandora repository administrators will not need to alter or configure any Contexts. But since much of Islandora's underlying functionality is governed by this powerful Drupal contrib module, administrators should become comfortable using it, and Actions, to customize and refine their repositories. Also, since Islandora objects are ordinary Drupal nodes, Islandora objects can take advantage of standard Drupal 8 Context functionality out of the box.
"},{"location":"user-documentation/extending/","title":"Extending Islandora: The Islandora Cookbook","text":"Because newer Islandora is very tightly integrated with Drupal, many use cases that required custom Islandora modules in Islandora Legacy and earlier can now be solved with a combination of configuration and modules from the Drupal community. Other use cases may still require custom code, but are not common enough, or complete enough, for that custom code to be a part of the core software.
Both scenarios are covered by the Islandora Cookbook below, which contains an annotated list of custom Islandora tools and modules developed by our community, a list of Drupal contributed modules that may be useful in an Islandora context, and a set of recipes which each provide a detailed outline for solving a particular use case.
Islandora is also compatible with most Drupal themes.
"},{"location":"user-documentation/extending/#introduction","title":"Introduction","text":"In the spirit of Islandora Awesome, Islandora Cookbook is a curated list of great modules and other tools for Islandora. Because Islandora is more tightly integrated with Drupal, most of these 'recipes' require only Drupal modules as ingredients.
We offer this list for discovery, but do not officially provide support for any of these modules.
"},{"location":"user-documentation/extending/#recipes","title":"Recipes","text":"Below are modules and tools that might be useful to solve common use cases, presented without specific recipes.
"},{"location":"user-documentation/extending/#islandora-contributed-modules","title":"Islandora contributed modules","text":"Warning! - All modules are under development.
Permissions by Term
By default, Drupal allows you only to restrict access to Drupal nodes by coupling node content types to user roles. The Permissions by Term module extends Drupal by functionality for restricting view and edit access to single nodes via taxonomy terms. Since Islandora objects can have taxonomy terms, this can be used to control access at the node and collection level. The submodule Permissions by Entity extends this control to the media level.
"},{"location":"user-documentation/extending/#setting-a-date-and-time-for-publication","title":"Setting a date and time for publication","text":"Moderation Scheduler
Moderation Scheduler gives content editors the ability to schedule Islandora nodes to be published at specified dates and times in the future. Can be combined with Views Bulk Edit to set the same scheduled publication date/time on multiple nodes at once.
"},{"location":"user-documentation/extending/#tombstoning","title":"Tombstoning","text":"Tombstones
A module that informs users who attempt to view deleted content that the resource has been removed instead of showing a generic 404 page. Fielded so that the date, reason, and alternative links can be given, along with a citation for the deleted object.
"},{"location":"user-documentation/extending/#displays","title":"Displays","text":""},{"location":"user-documentation/extending/#image-slideshow","title":"Image slideshow","text":"Views Slideshow
Views Slideshow can be used to create a slideshow of any content (not just images) that can appear in a View. Powered by jQuery, it is heavily customizable: you may choose slideshow settings for each View you create. It can be used to create an easy, adjustable slideshow of images from an Islandora repository. Views Slideshow can be seen in action with Islandora objects on the front page here
"},{"location":"user-documentation/extending/#ingest","title":"Ingest","text":""},{"location":"user-documentation/extending/#batch-uploading-via-csv","title":"Batch uploading via CSV","text":"Migrate Islandora CSV
This repository is a tutorial that will introduce you to using the Drupal 8 Migrate tools to create Islandora content. Whether you will eventually use CSVs or other sources (such as XML or directly from a 7.x Islandora) this tutorial should be useful as it covers the basics and mechanics of migration.
This repository is also a Drupal Feature that, when enabled as a module, will create three example migrations ready for you to use with the Migrate API.
"},{"location":"user-documentation/extending/#alternative-way-to-batch-upload-via-csv","title":"Alternative way to batch upload via CSV","text":"Islandora Workbench
Command-line tool for ingesting (and updating) nodes and media from anywhere - you don't need to access to the Drupal server's command line. Provides robust data validation, flexible organization of your input data (can use CSV, Google Sheets, or Excel files) plus creation of taxonomy terms on the fly.
"},{"location":"user-documentation/extending/#search","title":"Search","text":""},{"location":"user-documentation/extending/#custom-search-weighting","title":"Custom search weighting","text":"Search Overrides
This module provides a method for users with the necessary permissions to manually override the results being returned by Search API Solr. They will be able to choose a specific search term, and pick which nodes should be at the top, and also choose to exclude nodes so they will not be shown in the results. Currently, only nodes are supported.
"},{"location":"user-documentation/extending/#other","title":"Other","text":""},{"location":"user-documentation/extending/#batch-editing","title":"Batch editing","text":"Views Bulk Edit
A powerful tool that turns Views into a means of batch editing nodes, including Islandora repository objects. Once installed, create a view, add the fields that you would like to edit, add a Views bulk operations (Edit) field to the view, and select which actions you would like to have available. The Modify field values action will allow you to batch edit the value for the same field across multiple objects. A demonstration of a simple bulk-editing view with a few fields and actions can be found here
Content Type Clone
A tool that allows you to clone an existing content type. Can be used to copy and easily make your own version of the Repository Item Content Type with fewer or edited fields, without starting over. Has options to also copy all nodes from the old type to the new, and to delete from the old type when copying.
"},{"location":"user-documentation/extending/#gather-user-feedback","title":"Gather user feedback","text":"Content Feedback
The Content Feedback module allows users and visitors to quickly send feedback messages about the currently displayed content, and can be applied to Islandora nodes. All content feedback messages are listed and grouped by status in an administrative feedback list.
"},{"location":"user-documentation/extending/#sending-your-content-to-archiveorg","title":"Sending your content to Archive.org","text":"Wayback Submit to Archive.org
A tool for automatically submitting the contents of your site to Archive.org. The Wayback Submit module will submit all node types on schedule, according to criteria set by the site admin (only certain node types, only certain views, etc).
"},{"location":"user-documentation/extending/#other-resources","title":"Other resources","text":"This tutorial contains are step-by-step instructions for adding a facet on a vocabulary reference field. After a search, a block containing the configured facets appears letting the user filter their search.
This is a walk-through. For a more in-depth discussion of creating facets, see Configure Advanced Search
This tutorial currently works best with the \"standard\" install of the Islandora and Islandora Default modules which can be obtained from the playbook (as opposed to the install profile, which handles facet blocks differently).
"},{"location":"user-documentation/faceting/#step-1-add-vocabulary-bundles","title":"Step 1: Add vocabulary bundles","text":"Steps 1 and 2 add the field to the Solr index.
Now that you've added the field, you need to rebuild your Solr index.
Step 4 adds and configures the facet itself.
This step adds the facets in a single block.
At this point, searching for content that has facet values should cause the block to appear. For more in-depth overview of search, see Configure Advanced Search
Facets aren't necessarily searchable
While this will create facets, the values that appear won't work (won't necessarily bring back any content) if you type them in the search box. This is because the fulltext search box uses only fulltext fields, and facets, as mentioned above, requires string fields.
If you want to be able to search for taxonomy term values and bring up the related nodes, you could either include the full rendered item for your content type, or you may wish to repeat step 2 for each entity reference field, and set the new fields to fulltext so that searching for term values brings back node results.
"},{"location":"user-documentation/file-viewers/","title":"File Viewers","text":""},{"location":"user-documentation/file-viewers/#what-are-viewers","title":"What are viewers?","text":"Viewers allow site builders to display files in interactive JavaScript-based widgets, that provide functionality like zooming in/out, turning pages, playing/pausing, viewing in full screen, etc.
In Drupal, a common way to implement a viewer is through a module that provides a Drupal field formatter that interfaces with the appropriate JavaScript library. The field formatter will work with specific types of Drupal fields (e.g. file fields or image fields, some may even provide their own fields). Some viewer modules in Islandora also provide a block, that can display appropriate files based on the context.
Viewers that are known to work with Islandora include:
The simplest Drupal-y way of making a viewer appear is to configure a Media to render. You can do this by configuring a View Mode that shows the desired file field, displayed in a field formatter that invokes the desired viewer.
In the Starter Site:
Note
Formerly, this field was called \"Display Hints\". That field name has been retired in order to reduce confusion, since this uses a different mechanism. This mechanism no longer relies on Node View Modes, or EVA views. However, the basic EVA view still persists in the starter site as it is part of the Islandora Core Feature. Again, it will first look for a Service File, then fall back to the Original File.
"},{"location":"user-documentation/file-viewers/#changing-a-viewer-for-all-media-of-a-media-type","title":"Changing a Viewer for all media of a media type","text":"With the above configuration:
field_media_file, field_media_image... as appropriate), select a different field formatter and configure it how you like it. Suppose you have a new viewer available, for example, for zip files. You could either:
Either would work! The choice is yours to make. They're honestly both good.
Should you choose the latter:
Both OpenSeadragon and Mirador provide blocks that act as multi-page viewers. To configure one of these viewers:
node/%node/book-manifest-original; in the block, we enter this as node/[node:nid]/book-manifest-original. When the block is rendered on a node page, such as node/18, then the nid (18) will be passed into the view.Islandora uses Flysystem and the associated Drupal module to persist binary files to Fedora instead of keeping a copy in both Drupal and Fedora.
"},{"location":"user-documentation/flysystem/#background","title":"Background","text":"\"Flysystem is a filesystem abstraction library for PHP\" which allows applications to read from and write to a variety of data source beyond the local file system, such as an SFTP server, Amazon S3, and Zip files provided an Adapter is available to support it. Flysystem Adapters extend a single class implementing League\\Flysystem\\FilesystemInterface although some separate adapter traits for common actions and properties, such as the StreamedCopyTrait, are available.
The Drupal Flysystem module extends Flysystem to work within the Drupal filesystem structure. Drupal flysystem plugins include a Flysystem adapter (if not provided by default or in another library) and a class implementing Drupal\\flysystem\\Plugin\\FlysystemPluginInterface which instantiates the Flysystem adapter based on the Drupal site's configuration.
The Drupal Flysystem module uses flysystem stream wrappers to define filesystem descriptors which are configured in the site's settings.php file. The configurations including the filesystem prefix, adapter (driver), and any adapter-specific configurations such as API endpoints and authorization information.
Islandora implements a Flysystem adapter and a Drupal Flysystem plugin. The Flysystem adapter acts as an intermediary between the Flysystem filesystem API and Fedora, translating requests and responses between them. The adapter interacts with Fedora using an instance of the Chullo Fedora API it receives from the Drupal Flysystem plugin. The Drupal Flysystem plugin's main responsibility is to instantiate the Chullo Fedora API object with the proper authentication and pass it to the Flysystem adapter. To authenticate with Fedora the plugin adds a handler to the Chullo's Guzzle client which adds a JWT authentication header to each request.
The Fedora Flysystem adapter does not use Gemini to map the relationship between Drupal URIs and Fedora URIs, so they are indexed separately using the \"files_in_fedora\" Context which triggers the \"Index Fedora File in Gemini\" and \"Delete Fedora File in Gemini\" actions as appropriate.
"},{"location":"user-documentation/flysystem/#configuration","title":"Configuration","text":"The fedora file system is configured in the site's settings.php file. An example configuration can be seen in the islandora-playbook web server role's drupal tasks:
$settings['flysystem'] = [\n 'fedora' => [\n 'driver' => 'fedora',\n 'config' => [\n 'root' => 'http://localhost:8080/fcrepo/rest/',\n ],\n ],\n];\nhttp://localhost:8080/fcrepo/rest/test.tif and will be accessible from the Drupal URL http://localhost:8000/_flysystem/fedora/test.tif. The driver value fedora corresponds to the plugin's machine name. The config section contains all the adapter-specific configurations. In this case, the only thing configured for the site is the Fedora REST end-point. (Change this value to match your own Fedora's location, if needed.) The JWT is configured separately. Other examples of Drupal Flysystem configurations can be seen in the module's README.
Islandora is configured to have all Media use the Fedora file system by default in the islandora_core_feature. For example, the field storage uri_scheme setting for field_media_image (and the other media types) is \"fedora\". This can also be viewed in the UI on the field's \"Field settings\" page; e.g. http://localhost:8000/admin/structure/media/manage/image/fields/media.image.field_media_image/storage, look for \"Upload destination\" and see that \"Flysystem: fedora\" is selected.
However, there are methods for saving files that can explicitly set a different filesystem than the default. Migrations can explicitly set which file system a file is saved to and Islandora can emit events that also specify which file system a derivative should be saved to.
"},{"location":"user-documentation/flysystem/#derivatives","title":"Derivatives","text":"As hinted in the previous section, Islandora, by default saves derivatives to the Drupal public file system.
For example, if I upload a TIFF to a repository item as a File Media with the term \"Original File\", the \"Image Original File\" (image_original_file) Context is triggered. This fires the 'image_generate_a_service_file_from_an_original_file' action which emits an event using the 'public' scheme (file system).
To make Islandora save future derivatives to Fedora instead of to Drupal, change the corresponding action's \"File system\" setting ('scheme' in the corresponding config file) to 'fedora' instead of 'public'.
"},{"location":"user-documentation/glossary/","title":"Glossary","text":"The following glossary of terms addresses an Islandora context. When comparing new Islandora and Fedora to older versions it may also be helpful to reference the Islandora 7 Glossary.
"},{"location":"user-documentation/glossary/#alpaca","title":"Alpaca","text":"Islandora's event-driven middleware based on Apache Camel that handles communication between various components of Islandora, for instance synchronizing Drupal data with a Fedora repository and the Blazegraph triple store.
"},{"location":"user-documentation/glossary/#ansible","title":"Ansible","text":"Open source software for provisioning, configuration management and application deployment. In the context of Islandora, Ansible can be used to install and maintain the Islandora software stack more conveniently and efficiently on a server or group of servers. The configuration and installation instructions are captured in a human-readable list of tasks, called 'Playbook'. The Islandora Playbook for Ansible is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#apache","title":"Apache","text":"Refers to the Apache Software Foundation, a not-for-profit organization supporting various open source software development projects. The Islandora software stack consists of different components that are developed under the umbrella of the Apache Software Foundation, for instance Apache ActiveMQ, Apache Camel, the Apache HTTP server (webserver), Apache Karaf, Apache Solr, and Apache Tomcat.
Can in a narrower sense refer to the Apache HTTP server.
"},{"location":"user-documentation/glossary/#api","title":"API","text":"See Application Programming Interface
"},{"location":"user-documentation/glossary/#application-programming-interface","title":"Application Programming Interface","text":"Also API; a connection between computers or between computer programs. It is a type of software interface, offering a service to other pieces of software.
"},{"location":"user-documentation/glossary/#blazegraph","title":"Blazegraph","text":"Blazegraph is an open source triplestore and graph database. Islandora ships Blazegraph as part of the software stack. Metadata about Resource nodes is synchronized between the Drupal database and Blazegraph. Data in the Blazegraph triple store can be queried using SPARQL.
"},{"location":"user-documentation/glossary/#bundle","title":"Bundle","text":"A bundle is the generic name for a sub-type of a Content Entity type in Drupal. To illustrate: Node and Taxonomy Term are both names of Content Entity types, and both have sub-types (\"bundles\"). The bundles of Node are called \"Content Types\" and the bundles of Taxonomy Term are called \"Vocabularies\". Each bundle includes its own configurations of what fields are present on the bundle and how they are entered and displayed. A bundle is thus part of the configuration of your site. Some Content Entity Types, such as User, do not have bundles.
"},{"location":"user-documentation/glossary/#cantaloupe","title":"Cantaloupe","text":"Cantaloupe is an image server written in Java. It implements the IIIF Image API, which means it handles deep zooming of large images and other image manipulations. It is required to serve images to some viewers such as Mirador and OpenSeadragon.
"},{"location":"user-documentation/glossary/#checksum","title":"Checksum","text":"Checksums are a sequence of numbers and letters to check data for errors. If you know the checksum of an original file, you can use a checksum utility to confirm your copy is identical. Checksums can be used to check the Fixity of a file.
"},{"location":"user-documentation/glossary/#claw","title":"CLAW","text":"CLAW (CLAW Linked Asset WebDataFrameWork) was the development code name for the software released in June 2019 as Islandora 8, now called Islandora.
"},{"location":"user-documentation/glossary/#collection","title":"Collection","text":"A collection is a way of grouping related resources together, much like a directory on a computer. Collections can contain any number of related resource Nodes and sub-collections.
"},{"location":"user-documentation/glossary/#configuration","title":"Configuration","text":"See also: Configuration entity
Contrast: Content
In Drupal, your configuration is the total set of configuration entities that are live in your site. Configuration is usually managed through the Drupal GUI, and it can also be exported and imported. When it is active in your site, configuration lives in the Drupal database. When it is exported or serialized, configuration appears as a set of YAML (.yml) files, one file per configuration entity. Configuration can be overridden in the settings.php file.
"},{"location":"user-documentation/glossary/#configuration-entity","title":"Configuration entity","text":"See also: Configuration
Contrast: Content entity
A Drupal configuration entity (or \"config entity\") is an individual piece that makes up your site's configuration. It is usually represented as a single YAML (.yml) file, though the actual (\"live\") configuration lives in the database. A config entity usually represents the results of saving a single form in the administration interface, and may contain multiple (usually related) individual settings. Each configuration entity can be exported or imported as a \"single item\" through the Configuration Synchronization GUI, or with the Devel module's \"config editor\" can be edited individually. However, config entities are often interrelated and manual editing is usually not recommended.
"},{"location":"user-documentation/glossary/#content","title":"Content","text":"See also: Content Entity
Contrast: Configuration
In Drupal, your content is the total set of things that have been created or uploaded \"as content\" in your website. This includes all content entities - the actual nodes, media, files, taxonomy terms, etc, but does not include anything that is configuration. Content can be exported and imported, but only between sites with exactly the same configuration.
Sometimes, \"Content\" is used to refer to Nodes but not other content entities. This is the case when creating a new View and one of the options is to make a view of \"Content\".
"},{"location":"user-documentation/glossary/#content-entity","title":"Content entity","text":"See also: Content
Contrast: Configuration entity
In Drupal, content entities are the actual nodes, media, taxonomy terms, users, comments, and files that you've created on your site. For example, you may have 223 nodes, 534 media, 1000 taxonomy terms, 14 users, and 535 files in your site - those counts represent the numbers of content entities present in your site. \"Node\", \"Media\", \"Taxonomy term\" etc. are the high-level \"types\" of content entities. Some of these types have sub-types which are called bundles.
Content entities should not be confused with content types, which are bundles of nodes, and are part of a site's configuration.
"},{"location":"user-documentation/glossary/#content-model","title":"Content model","text":"Deprecated concept used in Islandora Legacy; see Islandora Model.
"},{"location":"user-documentation/glossary/#content-type","title":"Content type","text":"A type of Node. Content types are the \"bundles\" of Nodes, which are a type of Content Entity.
The standard Drupal Content types are 'Article' and 'Basic page'. Islandora Starter Site adds 'Repository Item' as a Content type, defining metadata fields typically used to describe digital resources. You can easily create your own content types.
"},{"location":"user-documentation/glossary/#context","title":"Context","text":"An \"if-this-then-that\" configuration created using the Drupal Context contrib module. Islandora extends the capabilities of Context by adding custom Conditions, custom Reactions, and by evaluating context at specific times to allow Contexts to be used for derivatives, indexing, and display.
"},{"location":"user-documentation/glossary/#crayfish","title":"Crayfish","text":"A collection of Islandora microservices. Some microservices are built specifically for use with a Fedora repository, while others are just for general use within Islandora.
"},{"location":"user-documentation/glossary/#datastream","title":"Datastream","text":"Deprecated terminology, refers to how Fedora 3/Islandora Legacy stored files as part of a resource ('object') in the Fedora repository. Replaced by Drupal Media entities, which 'wraps' Files in an intermediate structure. This allows Fields to be attached to files, for instance for storing technical metadata.
"},{"location":"user-documentation/glossary/#derivative","title":"Derivative","text":"A version of a file which is derived from an uploaded file. For example, a thumbnail generated from an uploaded image. Islandora uses microservices to generate derivatives. See the concept page for Derivatives.
"},{"location":"user-documentation/glossary/#docker","title":"Docker","text":"Docker is a platform that use OS-level virtualization to deliver software in packages called containers. Islandora uses Docker as part of ISLE, a suite of Docker containers that run the various components of Islandora.
"},{"location":"user-documentation/glossary/#drupal","title":"Drupal","text":"Drupal is an open source web content management system (CMS) written in PHP. Known for being extremely flexible and extensible, Drupal is supported by a community of over 630,000 users and developers. Drupal sites can be customized and themed in a wide variety of ways. Drupal sites must include Drupal Core and usually involve additional, Contributed code.
"},{"location":"user-documentation/glossary/#drupal-core","title":"Drupal Core","text":"The files, themes, profile, and modules included with the standard project software download.
"},{"location":"user-documentation/glossary/#drupal-roles","title":"Drupal Roles","text":"Roles are a way of assigning specific permissions to a group of users. Any user assigned to a role will have the same permissions as all other users assigned to that role. This allows you to control which users have permission to view, edit, or delete content in Drupal. Islandora provides a special role called fedoraAdmin that is required to have actions in Drupal reflected in Fedora.
"},{"location":"user-documentation/glossary/#entity","title":"Entity","text":"A Drupal term for an item of either content or configuration data. Examples include Nodes (content items), Blocks, Taxonomy terms, and definitions of content types; the first three are content entities, and the last is a configuration entity. In common usage, the term often refers to Drupal content entities like Nodes or Taxonomy terms.
"},{"location":"user-documentation/glossary/#fedora-repository-software","title":"Fedora (Repository Software)","text":"Fedora is a digital asset management architecture upon which institutional repositories, digital archives, and digital library systems might be built. Fedora is the underlying architecture for a digital repository, and is not a complete management, indexing, discovery, and delivery application.
The Fedora repository functions as the standard smart storage for Islandora.
"},{"location":"user-documentation/glossary/#ffmpeg","title":"FFmpeg","text":"FFmpeg is a cross-platform audio and video processing software. In Islandora, FFmpeg is provided by the Crayfish microservice, Homarus.
"},{"location":"user-documentation/glossary/#field","title":"Field","text":"Data of a certain type that is attached to a content entity. For instance, on a Resource Node content type, you might have fields for a title, description, display hints, subjects, and other metadata.
"},{"location":"user-documentation/glossary/#field-instance","title":"Field instance","text":""},{"location":"user-documentation/glossary/#field-type","title":"Field type","text":""},{"location":"user-documentation/glossary/#fits","title":"FITS","text":"File Information Tool Set, a set of software components for identifying, validating and extracting of technical metadata for a wide range of file formats.
"},{"location":"user-documentation/glossary/#fixity","title":"Fixity","text":"Also file fixity; digital preservation term meaning that a digital file remains unchanged ('fixed') over time. Fixity checking verifies that a file has not been corrupted or manipulated during a transfer process or while being stored. Typically, a fixity checking process computes checksums or cryptographic hashes for a file and compares the result to a reference value stored earlier. The Riprap microservice and the contributed Riprap Islandora module support fixity checking and error reporting in Islandora.
"},{"location":"user-documentation/glossary/#flysystem","title":"Flysystem","text":"Flysystem is a filesystem abstraction library for PHP. Islandora uses Flysystem to swap about different backend filesystem applications. Islandora provides a custom Flysystem adapter for Fedora.
"},{"location":"user-documentation/glossary/#glam","title":"GLAM","text":"Acronym for \"galleries, libraries, archives, and museums\".
"},{"location":"user-documentation/glossary/#greenfield","title":"Greenfield","text":"An installation without legacy constraints. Usually refers to a brand new system where users load new content, as opposed to migrating content from a previous system.
"},{"location":"user-documentation/glossary/#imagemagick","title":"Imagemagick","text":"Imagemagick is an open-source image processing library. In Islandora, Imagemagick is provided by the Crayfish Microservice, Houdini.
"},{"location":"user-documentation/glossary/#hocr","title":"hOCR","text":"hOCR is an open standard for representing OCR (Optical Character Recognition) results, including text positioning, as HTML. hOCR can be produced by Tesseract, and can be displayed as an overlay on an image by Mirador.
"},{"location":"user-documentation/glossary/#homarus","title":"Homarus","text":"Homarus is a microservice wrapper for FFMpeg. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#houdini","title":"Houdini","text":"Houdini is a microservice wrapper for Imagemagick. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#hypercube","title":"Hypercube","text":"Hypercube is a microservice wrapper for Tesseract. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#iiif","title":"IIIF","text":"The International Image Interoperability Framework. Generally pronounced \"triple-eye-eff.\" A set of open standards and APIs that help archives, libraries, and museums make the most of their digitized collections with deep zoom, annotation capabilities, and more, and also the community of users and developers that support the framework.
"},{"location":"user-documentation/glossary/#iiif-manifest","title":"IIIF Manifest","text":"Defined in the IIIF Presentation API, it is a document that includes \"The overall description of the structure and properties of the digital representation of an object.\" In Islandora, it lists one or more files, in order, that can be displayed in a viewer such as Mirador or OpenSeadragon.
"},{"location":"user-documentation/glossary/#ingest","title":"Ingest","text":"To ingest an object is to add an entry for it in Islandora. This can be done through the Drupal graphical user interface or one of the Drupal APIs (REST, Migrate API). The third-party contributed software Islandora Workbench uses the Drupal REST API for convenient bulk ingest.
In the context of digital repositories, ingest refers to the process by which the repository software imports and subsequently processes an object, creating derivatives automatically, and running any other processing that is configured to occur when an object is added. This would be distinguished by software which simply stores objects after import (with or without associated files) and performs no processing. The Islandora GUI and the documentation sometimes use other terms such as 'import' or 'add resource node'. In such contexts, these terms generally refer to the ingest process.
"},{"location":"user-documentation/glossary/#islandora-8-8x-20","title":"Islandora 8 (8.x, 2.0)","text":"Islandora 8, 8.x, 2.0, and CLAW are all deprecated names for the current version of Islandora. They referred to Islandora's use of Drupal 8, and being a major shift away from Islandora Legacy (formerly known as Islandora 7 or 7.x as it runs on Drupal 7).
"},{"location":"user-documentation/glossary/#islandora-install-profile","title":"Islandora Install Profile","text":"The Islandora Install Profile (in GitHub as Islandora Install Profile Demo, is a Drupal install profile that was developed by Born Digital, an Islandora vendor. It defines an Islandora with additional modules, themes, and configurations that were not defined in the Islandora Starter Site (formerly Islandora Defaults). The Install Profile and the Starter Site share the same function (though they approach it differently) and it is not possible to use both.
"},{"location":"user-documentation/glossary/#islandora-starter-site","title":"Islandora Starter Site","text":"The Islandora Starter Site is a way to install Drupal that provides a functional Islandora \"out of the box.\" It was created from Islandora Defaults [now defunct] by discoverygarden inc, an Islandora vendor. The Islandora Install Profile and the Starter Site share the same function (though they approach it differently) and it is not possible to use both.
"},{"location":"user-documentation/glossary/#islandora-model","title":"Islandora model","text":""},{"location":"user-documentation/glossary/#islandora-playbook","title":"Islandora playbook","text":"A set of human-readable YAML files, containing instructions for automatically configuring a server environment and installing the different components of the Islandora software stack. The instructions recorded in Playbook are executed by Ansible. The Islandora Playbook for Ansible is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#isle","title":"ISLE","text":"ISLE, or ISLandora Enterprise, is a community initiative to ease the installation and maintenance of Islandora by using Docker. ISLE is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#json-ld","title":"JSON-LD","text":"JSON-LD (JavaScript Object Notation for Linked Data) is a method of encoding linked data using JSON.
"},{"location":"user-documentation/glossary/#linked-data","title":"Linked data","text":"In computing, linked data is structured data which is interlinked with other data so it becomes more useful through semantic queries. Linked data typically employs the Resource Description Framework for data modelling.
"},{"location":"user-documentation/glossary/#manifest","title":"Manifest","text":"See IIIF Manifest.
"},{"location":"user-documentation/glossary/#matomo","title":"Matomo","text":"Matomo, formerly called Piwik, is a software for tracking visits to websites. It is an open source alternative to Google Analytics and allows the generation of website usage reports.
"},{"location":"user-documentation/glossary/#media","title":"Media","text":"Media are a Drupal Content entity type, which allows to manage Media items (Files) like images, documents, slideshows, YouTube videos, tweets, Instagram photos, etc. The Media module provides a unified User Interface where editors and administrators can upload, manage, and reuse files and multimedia assets. In the context of Islandora, Media entities 'wrap' files and provide a place to store file-specific metadata.
See https://www.drupal.org/docs/8/core/modules/media/overview for more information on the Drupal foundations, and refer to https://islandora.github.io/documentation/user-documentation/media/ for how Islandora uses Media.
"},{"location":"user-documentation/glossary/#memento","title":"Memento","text":"Protocol specification that allows a web client to request an earlier/historic state web resource (if available). Fedora implements the Memento protocol to store and serve versions of content in a Fedora repository.
"},{"location":"user-documentation/glossary/#mirador","title":"Mirador","text":"Mirador is a javascript-based zoomable image Viewer. It is related to (and more fully-featured than) OpenSeadragon. It has the ability to do zooming, display multiple pages, and display positioned text (e.g. hOCR or attributions). To render an image through Mirador, it must be provided a IIIF Manifest and the images must be served through a IIIF-friendly image server such as Cantaloupe.
"},{"location":"user-documentation/glossary/#microservice","title":"Microservice","text":"A software development technique \u2014 a variant of the service-oriented architecture (SOA) structural style \u2014 that arranges an application as a collection of loosely coupled services. In a microservices' architecture, services are fine-grained and the protocols are lightweight.
"},{"location":"user-documentation/glossary/#module","title":"Module","text":"Software (usually PHP, JavaScript, and/or CSS) that extends site features and adds functionality. Drupal modules conform to a specific structure allowing them to integrate with the Drupal architecture.
"},{"location":"user-documentation/glossary/#node","title":"Node","text":"Usually refers to a piece of Drupal Content of the type 'Node'. This includes actual pages, articles, and Resource nodes. Nodes must belong to a specific node bundle, called a \"Content Type\".
"},{"location":"user-documentation/glossary/#oai-pmh","title":"OAI-PMH","text":"The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed for harvesting metadata descriptions of records in an archive so that services can be built using (aggregated) metadata from many archives. Islandora allows to publish metadata in a way conformant to OAI-PMH, acting as a so-called OAI-PMH endpoint.
"},{"location":"user-documentation/glossary/#ontology","title":"Ontology","text":"In computer science and information science, an ontology encompasses a representation, formal naming and definition of the categories, properties and relations between concepts, data and entities. In the narrower context of the Resource Description Framework (RDF), an ontology is a formal, machine-readable description of the 'vocabulary' that can be used in a knowledge graph. An RDF ontology for instance specifies classes of things or concepts (e.g. the class of all book authors) and properties of classes/class instances (e.g. an author's name, birthdate, shoe size; also the fact that an author has written something that is in the class of books).
"},{"location":"user-documentation/glossary/#open-source","title":"Open Source","text":"Open source describes a method of software development that promotes access to the end product's source code. Islandora is an open source product with an active development community, operating under the GPL license (2.0) for Drupal components and the MIT license for non-Drupal components.
"},{"location":"user-documentation/glossary/#openseadragon","title":"OpenSeadragon","text":"OpenSeadragon is javascript-based zoomable image Viewer. It has the ability to do zooming and display multiple pages. To render an image through OpenSeadragon, it must be provided in a IIIF Manifest.
"},{"location":"user-documentation/glossary/#pr","title":"PR","text":"See Pull request
"},{"location":"user-documentation/glossary/#pull-request","title":"Pull request","text":"Also PR; sometimes also known as merge requests; technical term from distributed version control systems for software code like Git. Code contributors can request that the maintainer of a code repository 'pulls' the code change into the repository after approval.
"},{"location":"user-documentation/glossary/#rdf","title":"RDF","text":"See Resource Description Framework
"},{"location":"user-documentation/glossary/#repository-item","title":"Repository Item","text":"A type of content entity that comes \"out of the box\" with the Islandora Starter Site. See also: Resource Node
"},{"location":"user-documentation/glossary/#resource-description-framework","title":"Resource Description Framework","text":"Also RDF; family of World Wide Web Consortium (W3C) specifications originally designed as a data model for metadata. It has come to be used as a general method for conceptual description or modeling of information that is implemented in web resources. The data is modelled as a set of statements, also known as triples. A collection of RDF statements intrinsically represents a directed graph. Data represented according to the RDF specifications can be serialized in different ways, for instance using JSON-LD.
"},{"location":"user-documentation/glossary/#resource-node","title":"Resource Node","text":"A Resource node is a generic Islandora term for a Drupal Node that represents a single conceptual item or object stored in an Islandora repository. It acts as a stand-in for all files and metadata associated with that item, and is the place where the item 'lives' as a visitable URI.
The term 'Resource node' is specific to Islandora. Typically, Resource nodes in an Islandora installation will use a specific Content type for the digital assets stored in the repository.
For example, a video stored in Islandora will have a Resource node, with metadata stored in Fields. Attached to the Resource node is a Media entity, which encapsulates the preservation-grade file. The Resource node may be linked to further Media, for instance for a thumbnail, web-friendly derivative, and technical metadata associated with the resource node. The Resource node may also belong to one or more collections.
"},{"location":"user-documentation/glossary/#taxonomy-term","title":"Taxonomy term","text":"A Drupal Content Entity of the type 'taxonomy term'. Taxonomy terms belong to vocabularies which define what fields are available and how they behave. Drupal generally uses terms contained in taxonomies or vocabularies to classify content (tag or category). Taxonomy terms are used in Islandora to establish locally controlled vocabularies for describing resources, for instance for standardised spellings of names or subject terms.
"},{"location":"user-documentation/glossary/#tesseract","title":"Tesseract","text":"Tesseract is an open-source OCR (Optical Character Recognition) software. It can perform OCR in multiple languages. It can produce OCR (plain text) and hOCR (HTML, which includes positional data). In Islandora, Tesseract is provided by the Crayfish Microservice, Hypercube.
"},{"location":"user-documentation/glossary/#theme","title":"Theme","text":"Software and asset files (images, CSS, PHP code, and/or templates) that determine the style and layout of the site. The Drupal project distinguishes between core and contributed themes.
"},{"location":"user-documentation/glossary/#vagrant","title":"Vagrant","text":"Vagrant is an open-source software product for building and maintaining portable virtual software development environments (virtual machines). The Islandora Playbook includes a 'vagrantfile', a set of instructions that allows users to create a local virtual machine environment which will subsequently run Ansible to execute the configuration and installation steps recorded in the Islandora Playbook.
"},{"location":"user-documentation/glossary/#vbo","title":"VBO","text":"See Views Bulk Operations.
"},{"location":"user-documentation/glossary/#view","title":"View","text":"Drupal Views let you query the database to generate lists of content, and format them as lists, tables, slideshows, maps, blocks, and many more. The Views UI module, part of Drupal Core, provides a powerful administrator interface for creating and editing views without any code. There is a large ecosystem of extension modules for Views.
Views power many of the Islandora features, including viewers, IIIF Manifests, and search.
"},{"location":"user-documentation/glossary/#view-mode","title":"View Mode","text":"A View Mode is a way that a piece of Drupal content can be rendered. View modes let you create alternate configurations for what fields get displayed, in what order, and rendered in what field formatters. View modes are created under Manage > Display Modes > View Modes, but are configured at the bundle level (after first enabling that view mode to have its own configuration). If the requested view mode does not have a custom configuration, then the \"Default\" view mode will be used.
In Views, you can choose to show \"Rendered entities\" (usually as opposed to \"Fields\"). Here, you can select which view mode to use to render the results.
"},{"location":"user-documentation/glossary/#viewer","title":"Viewer","text":"A Viewer is any tool that allows Drupal to embed, display, or play back a particular object in a web-accessible format. Viewers are typically projects unto themselves. To use a viewer within Drupal usually involves a Library containing the viewer's code, as well as a Drupal Module that makes the viewer code appear within Drupal. Usually a viewer displays a single binary file, but some viewers (e.g. Mirador and OpenSeadragon) can display an entire manifest (ordered list of files).
"},{"location":"user-documentation/glossary/#views-bulk-operations","title":"Views Bulk Operations","text":"Also called VBO; a Drupal Module for performing bulk/batch operations on Nodes selected by a View definition.
"},{"location":"user-documentation/glossary/#virtual-machine-image","title":"Virtual Machine Image","text":"The Virtual Machine Image allows you to mount a fully working version of Islandora on your local machine as a separate virtual machine.
"},{"location":"user-documentation/glossary/#vocabulary","title":"Vocabulary","text":"A Drupal configuration entity that holds taxonomy terms. The vocabulary defines what fields are available on each term and how the terms behave. Vocabularies are the \"bundles\" of taxonomy terms.
"},{"location":"user-documentation/glossary/#weight","title":"Weight","text":"Drupal field that stores an integer value on an entity, allowing to represent the relative order of the entity in relation to other entities of the same type or subtype. Used by Islandora to store the order of components in compound objects, for instance pages in paged content items (books, serials).
"},{"location":"user-documentation/glossary/#yaml","title":"YAML","text":"YAML is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. Software applications like Drupal or Ansible store configuration information in YAML files for easy transportability of a configuration.
Some definitions adapted from Wikipedia and Drupal.org
"},{"location":"user-documentation/iiif/","title":"IIIF (International Image Interoperability Framework)","text":"IIIF is a set of specifications that provides interoperability of image-based collections across platforms. What this means for a repository platform like Islandora at a general level is that image-based objects such as (still) images and paged content can be managed by Islandora but viewed in external applications, and that Islandora can bring in image-based content from elsewhere to supplement locally managed content. If this intrigues you, see the section \"Looking under the hood (and beyond)\" below.
At a practical level, because Islandora supports several of the IIIF specifications, we can:
Display thumbnails for all pages of a book or newspaper issue within image viewers
If you're not using one of our provisioning tools, you will need to:
The Islandora Starter Site uses a Context to automatically use the IIIF Presentation API with the Mirador viewer for showing paged content.
To use this Context, give your book or newspaper (or other paged content) a model of \"Paged Content\" or \"Publication Issue\". To double-check this, in the Mirador Block - Multipaged items Context, you should see those terms used in the \"Node has term\" condition (you can register more than one term there, and having one of these on your node will activate this Context). Now, when you view a paged content Islandora node, you will see service files of all of its child pages (assuming you have added some child pages to the object) in the Mirador viewer as illustrated above.
If you are using the Mirador viewer, it enables a lot of features out of the box, including the strip of thumbnails at the bottom, and there is little to configure.
"},{"location":"user-documentation/iiif/#openseadragon-viewer-optional","title":"OpenSeadragon Viewer (optional)","text":"However if you are using the OpenSeadragon viewer, a Context can be set up as above, to show the OpenSeadragon viewer. In the Starter Site, there is currently a OpenSeadragon Block - Multipaged Items Context that is not enabled. You may wish to enable this and disable the Mirador Block - Multipaged items Context.
You can change how individual or paged content images are arranged in the OpenSeadragon viewport by doing the following:
admin/config/media/openseadragonIf you want to see the raw output of the IIIF API implementations in Islandora, visit a node that is displaying the OpenSeadragon viewer (doesn't matter if it's a single image or a paged content node like a book), and tack \"manifest\" onto the end of the URL, like http://myrepo.org/node/23/manifest and hit enter. You will see the raw JSON that IIIF-compliant viewers use to render the content. To see the output for a paged content item that lists all its children, tack /book-manifest on the end of the URL.
The really neat thing is, IIIF-compliant viewers don't need to be embedded in Islandora websites. If a viewer on another website knows the URL of a IIIF manifest like the ones that Islandora can produce, that viewer can display the content described in the manifest. Some implementations of IIIF viewers that show off the potential to combine content from multiple IIIF servers include:
These two examples have nothing to do with Islandora, but illustrate the potential for IIIF to build tools that extend beyond a given repository platform.
"},{"location":"user-documentation/iiif/#resources","title":"Resources","text":"To find resources on how to customize the IIIF interface check out IIIF's \"Guides to finding and working with IIIF materials\".
There is also a list of awesome IIIF resources that includes examples, tutorials, Digital Asset Management (DAMs), Image servers, Exhibits, Annotations, discovery, community involvement, Online training courses, and more.
"},{"location":"user-documentation/jwt-authentication/","title":"JWT authentication","text":"Islandora uses JWT tokens to authenticate communication between its components. RSA private public key pair is used to sign and verify JWT tokens. The process of issuing JWT tokens using RSA private key is handled by the Drupal jwt module.
The private public RSA pair needed by JWT authentication mechanism is generated in the web server. By default, claw playbook places the keys in /opt/islandora/auth. Crayfish and Tomcat/Karaf need the public key to verify the JWT token. By default, they are put in the following locations: /var/www/html/Crayfish/public.key, /etc/tomcat8/public.key. If you are deploying Crayfish and Karaf/Tomcat components to different servers, ensure that web server public.key files are in the expected locations.
Note that the connection need to be over SSL or an encrypted channel for this communication to be secure. Otherwise, a third party can capture your token and get access to your servers.
The JWT tokens expiration time is configurable via Islandora core settings: http://localhost:8000/admin/config/islandora/core. Currently, it is recommended to set the JWT Expiry to the maximum expected time for a job, including batch jobs.
The purpose of this page is to provide a guided reading list to anyone who wants to get up to speed on the basics of linked data within the Islandora community. Those who make their way through the readings will be able to talk competently about linked data and better understand the design decisions made in Islandora. The list starts with the fundamentals of linked data (RDF, SPARQL, serializations and ontologies) and moves toward more advanced topics specific to the use cases of a Fedora 4 based digital repository system.
"},{"location":"user-documentation/linked-data/#reading-list","title":"Reading list","text":""},{"location":"user-documentation/linked-data/#basics-of-linked-data","title":"Basics of linked data","text":"This section seeks to give the reader a foundational understanding of what linked data is, why it is useful, and a very superficial understanding of how it works.
This section is all about RDF, the Resource Description Framework, which defines the way linked data is structured.
This section takes a look at SPARQL, the query language that allows you to ask linked data very specific questions. The queryable nature of linked data is one of the things that makes it so special. Try some SPARQL queries on DBpedia's endpoint to get some hands-on experience.
RDF data can be translated into many different formats. RDF/XML is the original way that RDF data was shared, but there are much more human-friendly serialization formats like Turtle which is great for beginners. JSON-LD is the easiest format for applications to use, and is the serialization format that Islandora uses internally. Make sure to check out the JSON-LD Playground for an interactive learning experience.
Ontologies and vocabularies are created by communities of people to describe things, and once created, anyone can use an ontology or vocabulary to describe their resources. This section goes over some of the more popular ontologies & vocabularies in use.
One isn't limited to the ontologies & vocabularies that already exist in the world, anyone is free to create their own. This section goes over ontologies that exist to help those trying to create their own ontologies.
Most ontologies are very specific to certain use cases, and digital repository systems are no different. This section covers ontologies that are of specific interest to users of Islandora, or any Fedora 4 based digital repository system.
In Islandora, the JSON-LD Module transforms nodes (or media, or taxonomy terms) into the RDF that is synced into Fedora and the Triplestore. It uses RDF mappings, a concept defined by the RDF Module, and exposes them through the REST API at ?_format=jsonld.
A quick overview of JSON-LD, the RDF module, and the REST API.
"},{"location":"user-documentation/linked-data/#the-json-ld-syntax","title":"The JSON-LD syntax","text":"JSON-LD is a syntax which can be used to express RDF (like Turtle, or RDF XML), that is written in JSON, because devs like JSON and it's web-friendly. The JSON-LD syntax was designed for including Linked Data within HTML of web pages (similar to microdata or RDFa). Instead of nesting the RDF predicates within existing HTML tags as RDFa does, JSON-LD lets you put a solid blob of Linked Data inside a <script> tag. JSON-LD can also function as a standalone document, which is how we're using it.
The RDF Module is part of Drupal Core, but has no official documentation. The RDF Module embeds RDFa, a form of linked data, within the Drupal-generated HTML when you load the web page for a node, media, or taxonomy term. Official line is that this will allow Google to provide \"rich snippets\" such as star-ratings, contact info, and business hours. As an example of Drupal-provided RDFa:
<h1 class=\"page-header\">\n<span property=\"schema:title\">My cat</span>\n</h1>\nproperty=\"schema:title\" is markup generated by Drupal's RDF module that identifies the value \"My cat\" as the schema.org title of this page. A node's fields (such as field_tags) and properties (such as author) can be mapped to RDF according to a bundle-specific \"mapping\" that is stored within Drupal. In Drupal8-ese, RDF mappings are configuration entities. Drupal doesn't have a good UI for editing RDF mappings, but you can create, read, and update them as YAML files using Drupal's Configuration Synchronization interface (see section below on How to Edit an RDF Mapping).."},{"location":"user-documentation/linked-data/#rest-api","title":"REST API","text":"The pattern of using ?_format= to get a different representation of content is provided by the RESTful Web Services (rest) module. It allows other services to interact with Drupal entities through HTTP requests (GET, POST, PATCH, and DELETE). Which operations are allowed, and with what formats (such as xml, json, and jsonld) is configured at admin/config/services/rest/. Note that only jsonld uses RDF mappings; the json and xml formats expose a structured object based on how Drupal sees the entity. Access to these alternate formats through the REST API corresponds to permissions on the entity, so anyone with access content permission can view the JSON-LD version of that content. This is new as of Drupal 8.2.
For more information on interacting with Drupal entities via REST requests, see An Introduction to RESTful Web Services in Drupal 8.
"},{"location":"user-documentation/linked-data/#json-ld-module","title":"JSON-LD module","text":"Using the RDF mapping configurations provided by the RDF module, the JSON-LD Module exposes the RDF-mapped entity in JSON-LD, through the REST API, at node/[nid]?_format=jsonld (for nodes; for media and terms, at media/[mid]?_format=jsonld and taxonomy/term/[tid]?_format=jsonld).
types (which maps to rdf:type - see below, under Structure of an RDF Mapping).islandora module uses this hook to trigger any \"Map URI to Predicate\" and \"Alter JSON-LD Type\" reactions that are configured in Contexts. The Islandora Starter Site provides two Contexts - \"All Media\" and \"Content\" - that configure these to occur on Media and Repository Item nodes.?_format=jsonld to be part of, or not part of, the URIs of Drupal objects. On an out-of-the-box islandora-playbook, this string is stripped, but by default on a fresh install of the jsonld module, it is not.{\n\"@graph\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/8\",\n\"@type\":[\n\"http:\\/\\/pcdm.org\\/models#Object\"\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/title\":[\n{\n\"@value\":\"lasmomias de uninpahu\",\n\"@language\":\"fa\"\n}\n],\n\"http:\\/\\/schema.org\\/author\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/en\\/user\\/1\"\n}\n],\n\"http:\\/\\/schema.org\\/dateCreated\":[\n{\n\"@value\":\"2019-06-04T14:32:05+00:00\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#dateTime\"\n}\n],\n\"http:\\/\\/schema.org\\/dateModified\":[\n{\n\"@value\":\"2019-06-04T17:02:51+00:00\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#dateTime\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/description\":[\n{\n\"@value\":\"mpermmbklmh\",\n\"@language\":\"fa\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/created\":[\n{\n\"@value\":\"2015-10-15\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#string\"\n},\n{\n\"@value\":\"2015-10-15\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#date\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/extent\":[\n{\n\"@value\":\"1 item\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#string\"\n}\n],\n\"http:\\/\\/pcdm.org\\/models#memberOf\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/7\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/type\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/3\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/subject\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/27\"\n}\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/en\\/user\\/1\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Person\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/7\",\n\"@type\":[\n\"http:\\/\\/pcdm.org\\/models#Object\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/3\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Thing\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/27\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Thing\"\n]\n}\n]\n}\nIf using the Islandora Starter Site, the RDF mappings are set in the config files with names like rdf.mapping.[...].yml. The starter site config files will override configs set by modules. If you are building a site from scratch (not using the Islandora Starter Site, there are relevant configs in the following folders:
[drupal modules directory]/islandora/modules/islandora_core_feature/config/install/ (media and taxonomy terms)[drupal modules directory]/controlled_access_terms/modules/controlled_access_terms_defaults/config/install/ (the default corporate_body, family, geo_location, person, resource_type and subject vocabularies)[drupal web root]/core/profiles/standard/config/install/ (articles, pages, comments, and tags).Once loaded by modules, configuration .yml files are not live so editing them will not change the existing configuration. However, for modules that are Features, it is possible to re-import the changed configuration files at admin/config/development/features (todo: link to further reading on Features).
Once loaded, RDF mappings can be customized for the needs of a particular site through Drupal's Configuration Synchronization UI at admin/config/development/configuration. They can be exported, modified, and re-imported one-at-a-time by choosing the \"Single Item\" option on the Export/Import tabs. You can also create new RDF mappings (e.g. for a custom content type) and load them through this interface, by copying an existing mapping and changing the appropriate values.
Contributed module for RDF Mappings
A custom module rdfui exists, and is installed-but-not-enabled on boxes provisioned by the islandora-playbook. We don't use it because it is very rudimentary and limited to the schema.org vocabulary. We have an open ticket to develop a UI to support RDF mappings to any ontology. Contributions welcome.
ldp, ebucore, pcdm, are premis are registered in islandora.module using hook_rdf_namespaces(). To register your own namespaces, you will need to create a custom module that implements that hook.Below is an example of an RDF mapping as a .yml (YAML) file. It is the RDF mapping (current at time of writing) of the Repository Item (islandora_object) bundle, provided by the Islandora Starter Site and exportable as rdf.mapping.node.islandora_object.yml).
types specifies the rdf:type of the resource or content model. field_model, a required field of Islandora objects, also gets mapped to rdf:type through an arcane back-end process.fieldMappings specifies fields attached to that bundle and their RDF property mappings. One field can be mapped to more than one RDF property. It is a simple flat list.mapping_type:: There are several mapping types which are provided out of the box. - rel - standing for relationship, expresses a relationship between two resources - property - the default, or if a relationship is not provided, expresses the relationship between a resource and some literal text.
datatype_callback: This is a custom function that transforms the output of the field. There are some provided to us by Drupal and some added by Islandora, such as: - Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value which converts dates to ISO format - Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough which converts a referenced entity to the URI on the entity (which is configurable with the link_field argument An example usage of the Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough is as follows:
field_subject:\n properties:\n - 'dcterms:subject'\n datatype_callback:\n callable: 'Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough'\n arguments:\n link_field: 'field_authority_link'\nfield_authority_link of that subject's taxonomy term entity."},{"location":"user-documentation/linked-data/#sample-rdf-mapping","title":"Sample RDF mapping","text":"langcode: en\nstatus: true\ndependencies:\n config:\n - node.type.islandora_object\n module:\n - node\nid: node.islandora_object\ntargetEntityType: node\nbundle: islandora_object\ntypes:\n - 'pcdm:Object'\nfieldMappings:\n field_alternative_title:\n properties:\n - 'dc:alternative'\n field_edtf_date:\n properties:\n - 'dc:date'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_edtf_date_created:\n properties:\n - 'dc:created'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_edtf_date_issued:\n properties:\n - 'dc:issued'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_description:\n properties:\n - 'dc:description'\n field_extent:\n properties:\n - 'dc:extent'\n field_identifier:\n properties:\n - 'dc:identifier'\n field_member_of:\n properties:\n - 'pcdm:memberOf'\n mapping_type: rel\n field_resource_type:\n properties:\n - 'dc:type'\n mapping_type: rel\n field_rights:\n properties:\n - 'dc:rights'\n field_subject:\n properties:\n - 'dc:subject'\n mapping_type: rel\n field_weight:\n properties:\n - 'co:index'\n title:\n properties:\n - 'dc:title'\n created:\n properties:\n - 'schema:dateCreated'\n datatype_callback:\n callable: 'Drupal\\rdf\\CommonDataConverter::dateIso8601Value'\n changed:\n properties:\n - 'schema:dateModified'\n datatype_callback:\n callable: 'Drupal\\rdf\\CommonDataConverter::dateIso8601Value'\n uid:\n properties:\n - 'schema:author'\n mapping_type: rel\nDrupal 8 recognizes files (such as images, audio files, video files, etc.) but wraps each file in an intermediate structure called a \"media\" to allow us to attach fields to files. Drupal uses a media's fields to store information about the media's file, such as file size, width and height (for images), alt text (for images), creation date, and so on.
Compared to Islandora Legacy
In Islandora Legacy, this sort of technical metadata would have been stored as datastream properties or as additional metadata-specific datastreams. In Islandora, each datastream holds its technical metadata using an associated media entity.
Fedora will store the media's file as a binary resource and use the media's properties as the binary resource's description.
For example, creating a media with a file named test.jpg (in November 2019) will create a Fedora binary resource of the file accessible at /fcrepo/rest/2019-11/test.jpg with the media's fields accessible at /fcrepo/rest/2019-11/test.jpg/fcr:metadata.
Islandora Core Feature and the Islandora Starter Site make use of the media types provided by the Standard Drupal installation (Video, Audio, etc). The file extensions allowed by each media type have been configured at the Drupal level. It is possible to create your own media types, and/or to edit the allowed field types and functionality of the existing media types. However, with Islandora Starter Site, the Image media type only allows .png, .gif, .jpg or .jpeg files. Many large images such as TIFFs (.tiff files) and JP2s (.jp2) must be added in the File media type instead of Image.
"},{"location":"user-documentation/media/#media-revisions","title":"Media revisions","text":"The metadata associated with a file can be updated by clicking the Edit tab while on the media's page and clicking Save.
The Create new revision checkbox is selected by default which will prompt Fedora to make a new version of the media's metadata before updating its resource record. A message can be added to the revision which is stored in Drupal but is not currently saved in Fedora.
"},{"location":"user-documentation/media/#using-the-media-form-to-replace-an-existing-file-does-not-behave-as-expected","title":"Using the Media form to replace an existing file does not behave as expected.","text":"The media edit form allows a user to remove a file and replace it with a new one. However, because of the relationship Islandora creates between a file and its media, the effects of removing a file and uploading a new one are not intuitive.
First, the remove button removes the file's reference on the media but does not delete the file, which, if in Fedora, will remain in Fedora.
Second, because Drupal does not want users to reuse a file path uploading a file, even with the same name in the same month, Drupal will rename the file for you. This will result in a new binary resource in Fedora, rather than replacing the existing resource, so the file's URI in Fedora will change
Third, the metadata synced from the Media into Fedora at [old Fedora URI]/fcr:metadata will remain in Fedora, but the metadata in the Media will also be synced into Fedora at [new Fedora URI]/fcr:metadata. There will be no way for the metadata in Fedora describing the file at the old Fedora URI to be edited.
Removing Media with Actions
To completely delete a file and its metadata from Fedora and Drupal, run the \"Delete media and file(s) action\" after selecting the Media in the general media list (/admin/content/media). This will cause the paths to the file and its metadata in Fedora to return 404s.
Replacing Media via REST
It is possible to use Islandora's REST interface to replace Media and Files.
"},{"location":"user-documentation/media/#media-ownership","title":"Media ownership","text":"Islandora objects can have any number of media associated with them. Media store a reference to the resource node they belong to using a special field, \"Media Of\". By changing this field's value, you can change which resource node owns the media, and therefore, where it gets displayed or managed.
Compared to Islandora Legacy
The direction of the relationship between objects and datastreams is reversed when compared to Islandora 7. Generally speaking, objects are unaware of their datastreams, and it's a Drupal view that lists datastreams for an object.
"},{"location":"user-documentation/media/#media-use","title":"Media use","text":"Islandora media express their intended use with a special \"Media Use\" field, which accepts taxonomy terms from the \"Media Usage\" vocabulary. Because the Media Usage vocabulary is an ordinary Drupal vocabulary, Islandora site administrators can create additional terms, and in turn, these local terms can be used to identify media that have some custom local purpose. However, most of the default set of \"Media Use\" terms are taken from the PCDM Use Extension vocabulary:
Compared to Islandora Legacy
Terms from the Media Usage vocabulary are very similar to DSIDs in Islandora Legacy. The only difference is that a DSID is immutable, but a media's usage can be changed at any time through the media's edit form.
"},{"location":"user-documentation/media/#derivatives","title":"Derivatives","text":"Islandora generates derivatives based on the Media Use term selected for a Media and the Model of the node that owns it. All of this is configurable using Context.
By default, derivatives are generated from Media with the \"Original Files\" term selected. When an Original File is uploaded, if the node that owns it has an \"Image\" model, image derivatives are created. If it's a \"Video\", then video derivatives are generated, etc...
By default, derivatives are created with a path determined by the current year, current month, the associated resource node' identifier assigned by Drupal, and the type of derivative. For example, a resource node found at node/2 with a service file media generated in October 2019 will have the path \"2019-10/2-Service File.jpg\". Naming conventions can be configured by editing each derivative action's File path settings.
Within a node's media tab, you can see all of its media, including derivatives, listed along with their usage. For example, from the Original File, a lower quality \"Service File\" and a smaller \"Thumbnail Image\" file were generated.
For more information on how to configure derivatives, see the section on Context.
"},{"location":"user-documentation/media/#multi-file-media","title":"Multi-File Media","text":"An alternate method of using Media to store files and derivatives is known as \"multi-file\" or multifile media. This method allows a single resource node to have multiple \"Original Files\" or \"Preservation Masters\". This method was implemented by UPEI's Research Data Management project, to represent datasets that may include multiple important files that require equal preservation treatment and derivatives (such as a spreadsheet and accompanying data dictionary), while being described by a single set of descriptive metadata.
There are no multi-file media bundles that currently ship with Islandora. The following recipe describes how to set up multi-file media:
A video tutorial explaining how to configure multi-file Media is available under the following URL: https://www.youtube.com/watch?v=3U6tBvD8oJY
"},{"location":"user-documentation/metadata-harvesting/","title":"Metadata harvesting","text":""},{"location":"user-documentation/metadata-harvesting/#oai-pmh","title":"OAI-PMH","text":"The Open Archives Initiative Protocol for Metadata Harvesting, commonly referred to as OAI-PMH, is a specification for exposing repository metadata for harvesting. OAI-PMH specifies six services which can be invoked over HTTP(s). The full specification details the services:
Service URL on localhost:8000 Identify http://localhost:8000/oai/request?verb=Identify ListMetadataFormats http://localhost:8000/oai/request?verb=ListMetadataFormats ListSets http://localhost:8000/oai/request?verb=ListSets GetRecord http://localhost:8000/oai/request?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:localhost:node-1 ListIdentifiers http://localhost:8000/oai/request?verb=ListIdentifiers&metadataPrefix=oai_dc ListRecords http://localhost:8000/oai/request?verb=ListRecords&metadataPrefix=oai_dcThese services are provided by the OAI-PMH Drupal REST OAI-PMH module Drupal module. The Islandora Starter Site provides some default configuration so that repository content can be harvested. This configuration used to be part of Islandora OAI-PMH, a submodule of Islandora Defaults.
For OAI-PMH functionality, the Islandora Starter Site provides:
/admin/config/services/rest/oai-pmh) to connect this view with the REST OAI-PMH module.The REST OAI-PMH module indexes (caches) set membership, so new Items may not appear immediately in their respective sets. Indexing will happen automatically during cron runs but can be triggered manually at /admin/config/services/rest/oai-pmh/queue.
The OAI-PMH module makes use of one of two modules to provide metadata mappings: the RDF module or the Metatag module. By configuring OAI-PMH to use the RDF module (appears as \"OAI Dublin Core (RDF Mapping)\" and is enabled by default in the Islandora Starter Site), the OAI-PMH module will use the RDF mapping as configured for your content type (the same mapping that is used for Fedora and Blazegraph, e.g. rdf.mapping.node.islandora_object.yml).
However,
Field values not showing up in OAI-DC record?
If you want the value of a field to be emitted in the OAI-DC record, you must assign a Dublin Core predicate for that field in your content type's RDF mapping. If you are wondering why a field is not showing up in the OAI-DC record, the content type's RDF mapping is the first thing to check.
The REST OAI-PMH module does not support metadata formats other than OAI-DC, but it supports some alternate methods of defining mappings to OAI-DC. Consult that module's documentation for more information.
"},{"location":"user-documentation/metadata-harvesting/#creating-additional-metadata-formats","title":"Creating additional metadata formats","text":"This involves creating a new plugin.
The Drupal rest_oai_pmh module's DefaultMap plugin provides a basic model to follow for creating a plugin.
Exact implementation of your plugin will depend on your data model. The rest_oai_pmh module by default expects a flat list of fields and field values to output. This means that if your data model uses anything like Typed Relation field types, Paragraphs, or other complex nested entity modeling, you will need to add custom logic to build the values to emit via OAI-PMH for those fields.
"},{"location":"user-documentation/metadata/","title":"Metadata in Islandora","text":"Metadata vs Fields in the Starter Site
To learn about the default out-of-the-box metadata fields, see Starter Site Metadata Configuration. This page describes technical details about how metadata is handled as fields in Drupal and provides a deeper understanding of, and tools for modifying, your metadata configuration.
1-minute synopsis: In Islandora, metadata is stored in Drupal, in fields attached to entities (nodes or media). This allows us to interact with metadata (add, edit, remove, display, index in a search engine...) almost entirely using standard Drupal processes. If exporting this metadata to Fedora and/or a triplestore, the values are serialized to RDF using mappings that can be set for each bundle.
Drupal 8 Terminology
In Drupal 8, Fields can be attached to bundles (sometimes called entity sub-types -- e.g. Content types, Media types, Vocabularies) or entities (e.g. Users). For more on Fields, see \"2.3 Content Entities and Fields\" and \"6.3 Adding Basic Fields to a Content Type\" in the Official Drupal Guide.
As described in the resource nodes section, Islandora digital objects are comprised of Drupal nodes for descriptive metadata, Drupal media for technical metadata, and Drupal files for the binary objects. This section describes how Islandora uses and extends Drupal fields to manage descriptive metadata.
"},{"location":"user-documentation/metadata/#content-types","title":"Content Types","text":"In Drupal, Nodes come in different subtypes called Content Type. These let you define a type of content (\"Article\" and \"Basic Page\" are Drupal defaults and \"Repository Item\" is an Islandora specific example), the set of metadata fields that are attached to that content, and how those fields can be edited and displayed. Each Content Type is essentially a metadata profile that can be used for a piece of web content, or to describe a digital resource. You can create your own Content Types for your Islandora project or use a pre-defined one like Repository Item from the Islandora Starter Site. We will go over the metadata specific aspects of Content Types below, but for a fuller walk-through of creating a Content Type see here.
Not all Content Types in your Drupal site need be Islandora Resource Nodes. Making a Content Type a Resource Node will associate Islandora specific behaviours (such as syncing to Fedora or causing derivatives to be generated) with it. The decision to make a content an Islandora resource node is left to the discretion of the site manager. In Islandora, a \"resource node\" is usually considered a descriptive record for \"a thing\", and is conceptually similar to an \"Islandora Object\" in 7.x, i.e. a \"Fedora Object\" in Fedora 3.x and below. Read more on configuring a Content Type to be treated as a Resource Node.
"},{"location":"user-documentation/metadata/#fields","title":"Fields","text":"The administrator will define the fields that are associated with a specific Content Type. The same fields can be applied to different Content Type, but the field display and editing configurations are unique to each Content Type. The names and definitions of these fields are specific to Drupal and do not have to correspond to an outside metadata schema. You will give each field a Label, Machine Name, and a specific Field Type, like Text, Integer, EDTF, or Entity Reference (see below). Specific to the Field Type you will then define the maximum length of the field, the number of values it can contain, and what taxonomies it might link to.
Fields can be added under Administration >> Structure >> Content types >> Your Content Type's Name >> Manage fields (/admin/structure/types/your_type/fields). This tab will list all Fields, their Label, Machine Name, Field Type, and give you the option to make what edits to the definition of that field that you can.
Certain decisions must be made when fields are created, and before any content is added, because they can not be changed later. Field Type can not be changed, so you wouldn't be able to change a text field to a taxonomy field after creation. The field's machine name also can't be changed. The number of values allowed in a field or its maximum length or type of item to reference (in the case of Entity reference fields) can not be changed after content has been added. You can, however, always add new fields to a Content Type, even after content has been added.
7.x Migration Note: What About My MODS XML?
Even when using the Islandora Starter Site, there is no \"official\" metadata schema in Islandora. Where Islandora 7.x used MODS, and took advantage of its hierarchical/extensible structure, Drupal Fields are a flat structure working with distinct, individual elements. You can base your fields on those in MODS, or any other schema, but that structure is up to you. The Metadata Interest Group has developed a sample MODS-Drupal-RDF mapping, which provides a structure upon which you can build your Drupal fields. It is used by the Repository Item Content Type in the Islandora Starter Site.
You Cannot Change The Content Type Of A Node
Once a node is created, its Content Type cannot be changed. Just as you are unable to change many aspects of a Field once it has been created, once a node has been created it is now permanently of that Content Type and the fields associated with it. At that point your only option would be to create a new node of the intended content type, map the field values (programmatically or by copy-paste), and update any media or children that refer to the old node to refer to the new one.
The Islandora Starter Site provides a Repository Item Content Type that can be used as a structure to build your collection around, or it can be used as a sample to see how fields in Content Type work. It pre-defines fields, including Alternative Title and Date Issued that could be of use in many digital repositories. The full list of fields and their field types can be seen in the screenshot below.
Titles Aren't Conventionally-Configurable Fields
The field title is built-in to each Content Type by default, and can be referenced in views, templates, and indexing like other fields, but it cannot be configured like other fields. The only aspect you can change about title is its label. It has a built-in maximum length of 255 characters which cannot be changed. If your content requires longer titles we recommend you create a separate \"long_title\" field to store the full title and reserve the default title field for a display title. There is a contributed module called Node Title Length, which allows an administrator to configure the length of the title field in the core node table. However, this only works on nodes (not media or other entities) and involves meddling in a core Drupal database schema, which makes some people uneasy.
"},{"location":"user-documentation/metadata/#content-entry-formmanage-form-display","title":"Content Entry Form/Manage Form Display","text":"After creating the Fields for a Content Type you'll be able to manage the form used by content creators to create Nodes of that Content Type. On the Manage form display tab you'll be able to edit this form by arranging the order of the fields, choose what Widget will define the entry options for a field, and then set certain settings for that Widget. Fields are arranged by dragging the cross to the left of the Label. They can also be removed from the form, but not the Content Type, by dragging them to the bottom of the list under the Disabled heading. Widgets are defined by Field Type, so an Entity reference field could use auto complete, a select list, or even checkboxes, and are chosen from a drop-down list. The widget settings are accessed through the gear on the far right of a row and may allow you to set the size of an entry field, whether the field Label is displayed, or if you use placeholder text.
"},{"location":"user-documentation/metadata/#content-displaymanage-display","title":"Content Display/Manage Display","text":"The Manage display tab is where you will make decisions about how to display the metadata. Order is arranged as above, and can again be dragged to the Disabled section to hide the field from display. You can choose whether a field's label is displayed above the value, in-line, or hidden.
"},{"location":"user-documentation/metadata/#vocabularies","title":"Vocabularies","text":"See also: MIG Presentation on Taxonomies by Kristina Spurgin, 2021-07-19
In Drupal, Taxonomy Vocabularies (or simply Vocabularies) are also entity subtypes that define a set of fields and their configurations. Whereas instances of content types are called nodes, items in a vocabulary are called taxonomy terms (or simply terms). Traditionally, taxonomy terms are used to classify content in Drupal. For instance, the Article Content Type includes a field field_tags that can refer to terms in the Tags vocabulary.
There are two ways that users can interact with taxonomies: they can be \"closed,\" e.g. a fixed list to pick from in a dropdown, or \"open,\" e.g. field_tags where users can enter new terms, which are created on the fly. This is not set on the vocabulary itself, but in the configuration of the field (typically on a node). Terms within vocabularies have an ordering, and can have hierarchical structure, but do not need to.
Islandora (through the Islandora Core Feature) creates the 'Islandora Models' vocabulary which includes the terms 'Audio', 'Binary', 'Collection', 'Compound Object', 'Digital Document', 'Image', 'Newspaper', 'Page', 'Paged Content', 'Publication Issue', and 'Video'. Islandora Starter Site provides contexts that cause certain actions (e.g. derivatives to happen, or blocks to appear) based on which term is used.
The Controlled Access Terms module provides additional vocabularies:
Each of these vocabularies has its own set of fields allowing repositories to further describe them. The Repository Item Content Type has fields that can reference terms in these vocabularies. See 'Entity Reference fields' in the 'Field Types' section below.
The vocabularies provided by default are a starting point, and a repository administrator can create whatever vocabularies are desired.
Large Taxonomy Vocabularies
The Drupal Taxonomy UI is known to break down when your vocabularies get large (e.g. over 20,000 terms). Jonathan Hunt created the CCA Taxonomy Manager module for SFU to solve this problem.
"},{"location":"user-documentation/metadata/#field-types","title":"Field Types","text":"Fields are where descriptive and administrative metadata about Drupal entities is stored. There are different types of fields including boolean, datetime, entity reference, integer, string, text, and text_with_summary. These field types also have widgets (controlling how data is entered) and formatters (controlling how data is displayed). The Drupal 8 documentation on FieldTypes, FieldWidgets, and FieldFormatters includes a list of the core field types with brief definitions, along with a list of core widgets and formatters. Custom field types can be created to represent data in ways not provided by these core options.
More field types, formatters, and widgets are available in various modules. For example, the controlled_access_terms module provides two additional field types designed specifically for use with Islandora: ETDF, and Typed Relation. These and the Entity Reference field type are described in more detail below, since they are of particular interest for Islandora users.
"},{"location":"user-documentation/metadata/#authority-link","title":"Authority Link","text":"The Authority Link data type configures fields to hold two associated values:
Within Islandora, this data type is used by a metadata field in Taxonomy Vocabularies called Authority Sources to capture equivalent representations of terms from external authority sources.
Tip
The term external authority source refers to both controlled vocabularies like Art & Architecture Thesaurus or FAST as well as Name Authority Files like Library of Congress Name Authority File or VIAF.
For instance, if you are creating a term called Red squirrels within the default Taxonomy Vocabulary Subject, you may want to include the URI for Tamiasciurus from the FAST (Faceted Application of Subject Terminology) vocabulary. If you configured the field Authority Sources to list FAST (Faceted Application of Subject Terminology) as an external source authority option, you can select this source and add the associated URI (http://id.worldcat.org/fast/1142424).
"},{"location":"user-documentation/metadata/#configurations-for-authority-sources-field","title":"Configurations for Authority Sources field","text":"Each Taxonomy Vocabulary can have different external source authority options for its Authority Sources field. To configure the Authority Sources field to change these options, navigate to Home-->Administration-->Structure-->Taxonomy-->Edit Taxonomy Vocabulary Name-->Taxonomy Vocabulary Name and select \"Edit\" for the Authority Sources field. Then enter your options in the Authority Sources text box, entering one value per line in the format key|label. The key is the stored value (typically an abbreviation representing the authority source). The label will be used in displayed values and editing forms.
By default, this field is repeatable. To change this, edit the \"Field settings\" and change Allowed numbers of values from \"Unlimited\" to \"Limited\" and enter the number of allowable values. This will apply to every instance of the Authority Sources field across your Taxonomy Vocabularies. You cannot change the repeatability of Authority Sources after data has been entered in the field.
"},{"location":"user-documentation/metadata/#entity-reference","title":"Entity Reference","text":"Entity Reference fields are a special type of field built into Drupal core that creates relationships between entities. The field's configuration options include (but are not limited to):
The Repository Item content type, provided by the Islandora Starter Site, includes several entity reference fields that reference vocabularies defined by the islandora and controlled_access_terms modules.
"},{"location":"user-documentation/metadata/#configurations-for-entity-reference-field","title":"Configurations for Entity Reference field","text":"The screenshots below show how you can configure an entity reference field (in this case the Subject field on the Repository Item content type).
Tip
Note that once the type of entity to reference has been defined, and data has been created, it cannot be changed.
Storage settings for entity reference field where you set whether the field will reference content nodes or taxonomy terms:
Reference type settings for entity reference field where you select which vocabularies the autocomplete utility should query when editors are entering data:
Data Consistency
Selecting which vocabularies can be referenced from an entity reference field only affects which vocabularies will be searched when a user types into the autocomplete field in the Drupal form for adding a new item. These settings do not impose constraints on the underlying database, so it is still possible to load references to other vocabularies without being stopped or warned when ingesting data through various migration methods.
"},{"location":"user-documentation/metadata/#edtf","title":"EDTF","text":"The EDTF field type is defined in the controlled_access_terms module, and designed for recording dates in Extended Date Time Format, which is a format based off of the hyphenated form of ISO 8601 (e.g. 1991-02-03 or 1991-02-03T10:00:00), but also allows expressions of different granularity and uncertainty. The Default EDTF widget has a validator that only allows strings that conform to the EDTF standard. The Default EDTF formatter allows these date strings to be displayed in a variety of human-readable ways, including big- or little-endian, and presenting months as numbers, abbreviations, or spelling month names out in full. Close review of the EDTF Specifications is recommended when configuring this field type.
Endianness
Big-endian = year, month, day. Little-endian = day, month, year. Middle-endian = month, day, year.
Known EDTF Bug
When configuring the EDTF widget for a field in a content type, you can choose to allow date intervals (aka date ranges), but doing this prevents the widget from accepting values that include times. (The EDTF standard states that date intervals cannot contain times, but the field should be able to accept either a valid EDTF range or a valid EDTF datetime, so this is a bug.)
Example of valid inputs in a multivalued EDTF Date field (including the seasonal value 2019-22 as defined in the EDTF specification):
Example of the same EDTF dates displayed using little-endian format:
EDTF field values cannot include textual representations of dates, as shown below in this example of a valid EDTF value ('1943-05') and an invalid value ('May 1943') with the corresponding error message. Use the formatter configurations detailed further below to achieve textual display of dates.
"},{"location":"user-documentation/metadata/#configuration-for-the-default-edtf-widget","title":"Configuration for the Default EDTF Widget","text":"This configuration can be set per field by clicking the gear icon next to any field defined with EDTF field type at Administration >> Structure >> Content types >> Repository Item >> Manage form display (admin/structure/types/manage/islandora_object/form-display)
Configuration options include strictness level of date validation, allowing date intervals and allowing date sets.
"},{"location":"user-documentation/metadata/#configuration-for-the-default-edtf-formatter","title":"Configuration for the Default EDTF Formatter","text":"This configuration can be set per field by clicking the gear icon next to any field defined with EDTF field type at Administration >> Structure >> Content types >> Repository Item >> Manage display (admin/structure/types/manage/islandora_object/display)
Example of how the EDTF formatter settings can change the display of an EDTF value:
"},{"location":"user-documentation/metadata/#configuration-for-indexing-and-sorting-edtf-fields-in-search-results","title":"Configuration for indexing and sorting EDTF fields in search results","text":"By default, EDTF date values are indexed in Solr as string values. The entered value (not the displayed value) is indexed.
Solr
The Solr string data type requires the full field value to match the query in order to count as a match. This means that searching for 2014 will not retrieve a record where the recorded date value is 2014-11-02.
EDTF date fields may be configured as sort fields in your search results Views. By default, this results in a simple ordering by the literal EDTF date string.
An EDTF date field with multiple or unlimited number of allowed values may be set as a sort field. In this case, the first occurrence of the field value is used as the sorting value.
"},{"location":"user-documentation/metadata/#typed-relation","title":"Typed Relation","text":"The Typed Relation field is defined in the controlled_access_terms module, is an extension of Drupal's Entity Reference field type, and allows the user to qualify the type of relation between the resource node and other entities, such as taxonomy terms. For example, it enables the inclusion of a resource's contributor's (assuming contributor names are modelled as taxonomy terms or some other Drupal entities) as well as their roles (such as \"author\", \"illustrator\", or \"architect\") in the resource node itself. Using only Drupal's Entity Reference fields, we would need individual fields for \"author\", \"illustrator\", \"architect\", and any other roles that may need to be made available. Using a Typed Relation field, we can have one Entity Reference field for \"Contributors\" and let the user pick the affiliated role from a predefined dropdown list.
Typed relation name
The parts of a field are called properties, so 'entity reference' and 'relation type' are properties of the Typed Relation field type.
"},{"location":"user-documentation/metadata/#configurations-for-the-typed-relation-field","title":"Configurations for the Typed Relation field","text":"The Islandora Starter Site includes a Typed Relation field labelled 'Contributors' as part of the Repository Item content type, and populates the available relations from the MARC relators list. This field was formerly called \"Linked Agent\".
The list of available relations for this Contributors field is configurable at '/admin/structure/types/manage/islandora_object/fields/node.islandora_object.field_linked_agent'.
Typed relation tradeoffs
Publishers
Until Mar 2024, the Islandora Starter Site included publishers in the Contributors (field_linked_agent) field. The MIG made the decision to make publisher its own text field, in order to make it easier to separate publishers from other contributors, and to prevent clutter in the linked taxonomies. Publishers are often recorded by transcribing what is on the item, rather than formatting the name per Authority rules, so variations on a single name are expected.
Relations are defined in the format key|value, and the key is used in the RDF mapping (see below).
By default, facets can be created for typed relation fields that will facet based on the linked entity alone, not separating references based on the relationship type.
"},{"location":"user-documentation/metadata/#adding-a-new-field","title":"Adding a new field","text":"You are free to add new fields to your Islandora content type(s). After you set one up, you may want to configure the following:
rest_oai_pmh module's templates/mods.html.twig. For example, that template file includes {{ elements.publisher }} so the label in views needs to be exactly publisher. Save the View.An example of this, in terms of config changes, is visible in the changeset of the Starter Site's 1.6.0 version for the Publisher field. However, pulling such a changeset is unlikely to work smoothly and it is recommended to set these configurations manually.
"},{"location":"user-documentation/metadata/#getting-metadata-into-fedora-and-a-triple-store","title":"Getting Metadata into Fedora and a Triple-store","text":"Depending on the needs at your institution, you may or may not be using Fedora with your Islandora installation. You also may or may not be hoping to publish your metadata as RDF triples that can be queried in a triplestore. Both of these functionalities are driven by the JSON-LD module (written for Islandora), which provides a JSON-LD serialization of your content nodes, media nodes, as well as your taxonomy terms. This JSON-LD is what gets ingested by Fedora, and is also what is used to add RDF triples to the blazegraph triplestore if you choose to use that service.
The JSON-LD serialization for an entity is available by appending _format=jsonld to the entity's URL. Below is an example JSON-LD document representing the RDF serialization of a Repository item node created in a standard islandora-playbook based vagrant VM:
{\n \"@graph\":[\n {\n \"@id\":\"http://localhost:8000/node/1\",\n \"@type\":[\n \"http://pcdm.org/models#Object\"\n ],\n \"http://purl.org/dc/terms/title\":[\n {\n \"@value\":\"New York, New York. A large lobster brought in by the New England fishing boat [Fulton Fish Market]\",\n \"@language\":\"en\"\n }\n ],\n \"http://schema.org/author\":[\n {\n \"@id\":\"http://localhost:8000/user/1\"\n }\n ],\n \"http://schema.org/dateCreated\":[\n {\n \"@value\":\"2019-03-14T19:05:24+00:00\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#dateTime\"\n }\n ],\n \"http://schema.org/dateModified\":[\n {\n \"@value\":\"2019-03-14T19:20:51+00:00\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#dateTime\"\n }\n ],\n \"http://purl.org/dc/terms/date\":[\n {\n \"@value\":\"1943-05\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n },\n {\n \"@value\":\"1943-05\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#gYearMonth\"\n }\n ],\n \"http://purl.org/dc/terms/extent\":[\n {\n \"@value\":\"1 negative\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/identifier\":[\n {\n \"@value\":\"D 630714\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/type\":[\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/11\"\n }\n ],\n \"http://purl.org/dc/terms/rights\":[\n {\n \"@value\":\"No known restrictions. For information, see U.S. Farm Security Administration/Office of War Information Black & White Photographs(http://www.loc.gov/rr/print/res/071_fsab.html)\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/subject\":[\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/26\"\n }\n ],\n \"http://schema.org/sameAs\":[\n {\n \"@value\":\"http://localhost:8000/node/1\"\n }\n ]\n },\n {\n \"@id\":\"http://localhost:8000/user/1\",\n \"@type\":[\n \"http://schema.org/Person\"\n ]\n },\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/11\",\n \"@type\":[\n \"http://schema.org/Thing\"\n ]\n },\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/26\",\n \"@type\":[\n \"http://schema.org/Thing\"\n ]\n }\n ]\n}\nThe RDF mapping for a content type, media type, or vocabulary defines how fields in Drupal are mapped to properties in the JSON-LD serialization. The mapping defines the RDF predicates that should be used for each field. You reference Drupal fields via their Machine Name, and the RDF predicate by using the conventional syntax namespace:predicate. In this example, the dc prefix stands for http://purl.org/dc/terms/, so when concatenated the final RDF predicate is http://purl.org/dc/terms/title.
To show a small example, the RDF mapping:
title:\n properties:\n - dc:title\nwill map the Repository item's title field to http://purl.org/dc/terms/title. As a result, the node's title value appears like this in the JSON-LD output:
\"http://purl.org/dc/terms/title\":[\n {\n \"@value\":\"New York, New York. A large lobster brought in by the New England fishing boat [Fulton Fish Market]\",\n \"@language\":\"en\"\n }\n],\nTip
To set up prefixes for namespaces and see a list of available predefined namespaces, see the \"RDF Generation\" page.
"},{"location":"user-documentation/metadata/#typed-relation-fields-in-rdf","title":"Typed Relation fields in RDF","text":"Unlike other fields, which can be assigned RDF predicates in RDF Mapping YAML files, a typed relation field uses a different predicate depending on the chosen type. These predicates are assigned using the 'keys' in the key|value configuration. The key must be formatted namespace:predicate, e.g. relators:act.
Bug
The Drupal RDF module is currently limited in the complexity of graph you can generate. All fields must be mapped directly to either a literal value, or a reference to another Content Type instance, media type instance, or taxonomy term instance. It is not currently possible to create blank nodes or skolemized nodes for nesting fields under more complex structures.
"},{"location":"user-documentation/metadata/#batch-editing-metadata-in-fields","title":"Batch editing metadata in fields","text":"If you are editing multiple resources in order for them to have the same metadata value, the Views Bulk Edit module can help. Here is a video of creating a view using Views Bulk Operations to apply a subject term to multiple resources simultaneously.
For more complex changes, or when the values need to differ for each value, an export-modify-reimport method may be needed. Use a view to export CSV or other structured data (including an identifier such as a node id), modify the values as necessary, then use migrate csv or Workbench to re-import and update the values.
"},{"location":"user-documentation/metadata/#exporting-data","title":"Exporting Data","text":"One common approach for exporting your content and/or taxonomy data out of Islandora is to use Drupal's Views Data Export module. The module has extensions that can allow you to configure exports as CSV, XML, text files, and other formats based on your local needs.
"},{"location":"user-documentation/metadata/#drupal-bundle-configurations","title":"Drupal Bundle Configurations","text":"In Islandora, content models are primarily created using content types (also known as node bundles) and media bundles. Bundles are defined by YAML configuration files. To create new content models, one would create the needed content types and media bundles via UI, then export the yml files related for those bundles using Configuration Synchronization (http://localhost:8000/admin/config/development/configuration) or Features. An understanding about the structure of a bundle and various configuration files used to define it helps in creating and updating it.
Content types and media bundles can be thought of as web forms consisting of fields. Drupal provides widgets to define the behavior of a field and field storage to define how the data is stored in the database. Drupal provides various display modes to show the forms to user when they are editing (Manage form display) or viewing (Manage display).
A content model is packaged as a module for installation. All yml files are put in config/install folder of the module. Note that not all content models would contain media bundles.
The following files define the bundles themselves. It contains some metadata about the bundle and lists its dependencies.
node.type.your_content_type.yml\nmedia_entity.bundle.your_media_bundle.yml\nThe following files define the fields attached to the bundle forms. There must be one config file for each field in your bundle, except for the default drupal fields.
field.field.node.your_content_type.field_name1.yml\nfield.field.node.your_content_type.field_name2.yml\n...\nfield.field.media.your_media_bundle.field_name1.yml\nfield.field.media.your_media_bundle.field_name2.yml\nIf the new bundle contains new fields, then field storage configurations for the newly created fields would be needed as well. Note that if you reused existing fields, storage definitions should not be defined again. Storage config contains information about the number of values allowed for that field (cardinality).
field.storage.node.field_new_name3.yml\nfield.storage.media.field__new_name3.yml\nThere is a configuration file for each combination of bundle / display mode when managing form displays. Usually, form displays will have default and inline modes.
core.entity_form_display.media.your_media_bundle.default.yml\ncore.entity_form_display.media.your_media_bundle.inline.yml\n---\ncore.entity_form_display.node.your_content_type.default.yml\ncore.entity_form_display.node.your_content_type.inline.yml\nThere is a configuration file for each combination of bundle / display mode when managing displays. Usually, displays will have default and teaser modes for content types and default and content modes for media bundles.
core.entity_view_display.media.your_media_bundle.default.yml\ncore.entity_view_display.media.your_media_bundle.content.yml\n---\ncore.entity_view_display.node.your_content_type.default.yml\ncore.entity_view_display.node.your_content_type.teaser.yml\nIn addition, Islandora needs an RDF mapping to express the content in RDF and to sync to fedora. There will be one RDF mapping per bundle.
rdf.mapping.media.your_media_bundle.yml\nrdf.mapping.node.your_content_type.yml\nIslandora enables you to build full-fledged multilingual repositories leveraging the multilingual support provided by Drupal core modules. The multilingual content gets indexed into Fedora repository as well as the Triplestore (Blazegraph), and can be queried using the SPARQL endpoint. In this guide, we will describe the steps needed to set up a basic multilingual Islandora site.
"},{"location":"user-documentation/multilingual/#drupal-concepts","title":"Drupal Concepts","text":"Drupal allows you to translate user interface text, configuration text, and content. See section 2.7 of the Drupal documentation for details.
"},{"location":"user-documentation/multilingual/#islandora-configuration","title":"Islandora Configuration","text":"Islandora enables Drupal's Language and Content Translation modules by default. Drupal provides additional modules for multilingual support, for instance for translating the built-in user interface or editable interface text.
"},{"location":"user-documentation/multilingual/#adding-languages","title":"Adding Languages","text":"From the top menu, go to Configuration >> Regional and language >> Languages. Add a language.
"},{"location":"user-documentation/multilingual/#adding-language-switcher","title":"Adding Language switcher","text":"You can place the default language selector block to switch between languages. To create the language switcher block go to Structure >> Block layout. Click Place block in a region of your choice. Search for Language switcher block and click Place block.
You can place the language switcher block in different regions of the user interface. You might have to customize the theme to style the language switcher block.
"},{"location":"user-documentation/multilingual/#adding-multilingual-menus","title":"Adding Multilingual Menus","text":"From the top menu, go to Configuration >> Regional and language >> Content language and translation. Check Custom menu link under Custom language settings. Scroll down to Custom menu link section and check all the relevant fields and Save the configurations. Clear the cache (Configuration >> Development >> Performance).
From the top menu, go to Structure >> Menu. Edit \"Main navigation\" menu. Default home menu item cannot be translated due to this issue. Disable that menu item. Click Add link to create a new menu item. Provide a menu title (i.e. Home) and input <front> for the link field. Save. Right-click on the Operations beside the new menu link and click the Translate button. Translate the menu link title for the language added above and save.
Go back to home. The language switcher will enable you to switch the language/content of the menu and content.
"},{"location":"user-documentation/multilingual/#adding-a-multilingual-repository-item","title":"Adding a Multilingual Repository Item","text":"From the top menu, go to Content >> Add content >> Repository item. Provide the required fields and save the object. Click the Translate tab of the object, provide a title in the second language and fill any translatable fields (i.e description). Add the media for the object. Media objects can be translated similar to the repository item node.
Go back to home, you should be able to view content in the language selected in the language switcher.
"},{"location":"user-documentation/multilingual/#field-label-translations","title":"Field Label Translations","text":"If you need the field labels of the repository Item displayed in a different language, additional configuration is needed. The Drupal module Configuration translation module in the core needs to be enabled. Note that this will enable the module User Interface translation as well.
Each field label needs to be translated through the Drupal GUI (Configuration >> Regional and Language: User interface translation). Alternatively, you can import existing translations, or translations generated with an external translation editor (for example Gtranslator). Go to Interface translation import (Configuration >> Regional and Language: User interface translation >> Import tab). Set the Treat imported strings as custom translations option, select the import file, the target language, and click import. Clear the cache to see the changes. An example second language display is shown below.
Islandora indexes the multilingual metadata values in Fedora as RDF literals using language tags. An example representation is shown below.
"},{"location":"user-documentation/multilingual/#triplestore-representation","title":"Triplestore Representation","text":"Islandora indexes the multilingual metadata values in Blazegraph as RDF literals using language tags. An example representation is shown below.
You can query the result and filter the results by a specific language. For example, to get all titles and filter by language, the following query can be used:
PREFIX dcterm: <http://purl.org/dc/terms/>\nselect ?s ?o {\n ?s dcterm:title ?o\nFILTER (lang(?o) = 'ta')\n}\nPaged content, such as books, periodicals, photographs with the front and back, etcetera, can use the membership structure provided by Islandora, namely, field_member_of. This involves creating a resource node for the root record (i.e. the entire book or the photograph) and child resource nodes for each sub-component (e.g. \"Page 1\", \"Page 2\", etc., or \"recto\" and \"verso\") with their corresponding media. Each \"child\" resource node contains a reference to their \"parent\" resource node using the field_member_of property.
Child v. Member
Islandora uses the \"child\" and \"member\" descriptor for resource nodes that store a reference to another resource node in the \"Member Of\" field interchangeably. Administrators will more often see the \"member\" terminology more often while front-end users will usually see \"child\" terminology.
Similar to the collection view showing members of a collection, Islandora provides taxonomy terms Paged Content and Page in the Islandora Model vocabulary, that can be used to trigger paged content behaviors. Islandora Starter Site provides a Context for Paged Content resource nodes to trigger displaying a Mirador viewer showing the child resource nodes' service files.
To enable this, use Paged Content for the Islandora Model of the parent resource node and Page for the Islandora Model of the child resource node. After the child resource nodes have \"service file\" media (either generated via the built-in derivative creation or added manually), they will be included in the Mirador paginated viewer displayed on the parent resource node's page. (See the IIIF Documentation for more details on controlling the viewer's behavior.)
However, repository managers can use any method they like, such as a Views-based list of teasers, a gallery, or a slide-show to display child resource nodes.
"},{"location":"user-documentation/paged-content/#ordering-weight","title":"Ordering (weight)","text":"By default, child resource nodes are un-ordered. To order the subcomponents of a paged content resource node, Islandora provides a Weight field to store an integer value on child resource nodes. Children resource nodes with smaller weight values will float to the top and come before child resource nodes with heavier weight values.
Weight values do not need to be sequential, just ordered from smallest to largest. For example, the first child resource node can have a value of 10 and the next could have a value of 20 and they will be ordered accordingly. Should a new child be added with the weight value of 15 it will automatically be sorted after the child with the weight value 10 and before the child with the weight value 20.
Child resource nodes can be reordered using a drag-and-drop interface by clicking on the Re-order Children button on the Children tab of the parent resource node.
Re-ordering children resource nodes on this page and clicking Save will cause each child resource node's weight value to be updated using sequential values.
Why not Drupal's book or weight modules?
Drupal provides the book module in core for creating multi-level ordered content, such as books and manuals. However, this module stores structure and pagination separately from the nodes making serializing these relationships as RDF we can provide to Fedora more difficult than simply using field_member_of with an RDF mapping. Support for the book module may be added in the future.
Drupal also has a weight module that provides a weight field and a drag-and-drop reordering user-interface. However, this module requires users to set a specified range of values which includes their negative corresponding value. E.g. a range setting of '20' will require all children to have a value between '-20' to '20'. This presumes a repository manager can predict how many pages the largest paged content item in their repository will be beforehand. Also, these weight values are serialized into RDF using the Collections Ontology 'index' predicate which assumes positive integer values which cannot be guaranteed using the weight module.
"},{"location":"user-documentation/paged-content/#adding-children","title":"Adding children","text":"Islandora provides an interface on the Children tab to either Batch upload children or Add Child. Both methods will result in new resource nodes that are member of the current node. It is also possible to create child nodes separately, then edit their Member Of field to point to the desired parent.
"},{"location":"user-documentation/paged-content/#batch-upload-children","title":"Batch upload children","text":"If you want to add a number of pages to this node, for example, the Batch Upload Children button may suit your needs. It allows you to upload multiple files, after selecting a content type for the child nodes and the media type and media use for the media that will hold the uploaded files. If selecting a node with the Model (field_model) field, it will also let you select a model from the Islandora Models vocabulary. These settings will apply to all nodes and media created for this batch.
This method does not create full metadata for the child nodes. It uses the filename as the node title. It also does not accept zip files; individual files must be uploaded. For more methods of bulk uploading content, see Islandora Workbench and Migrate Islandora CSV
"},{"location":"user-documentation/searching/","title":"Setup and Configure Search","text":"Islandora comes with the Drupal 8 Search API and SOLR modules enabled with a corresponding SOLR instance. This guide gives an overview to the setup provided by the islandora-playbook. Much more detail is available in the Search API documentation. Another helpful resource is Adam Fuch's \"Drupal 8 Custom Site Search with Search API\" (https://www.electriccitizen.com, 2018-01-10; last accessed 2019-03-08).
"},{"location":"user-documentation/searching/#indexing-islandora-with-solr","title":"Indexing Islandora with SOLR","text":"To access the search indexing settings, log in as an administrator and navigate to '/admin/config/search/search-api' or click Configuration and then Search API.
"},{"location":"user-documentation/searching/#solr-server-configuration","title":"SOLR Server Configuration","text":"On the Search API page, Use the Solr Server link to view the SOLR server's configuration and the Default Solr content index link to view or update the index settings. In most cases, where the site was built using Ansible, the Solr Server settings that were configured during installation should be left alone.
"},{"location":"user-documentation/searching/#solr-index-configuration","title":"SOLR Index Configuration","text":"The Default Solr content index user interface is divided into four tabs: View, Edit, Fields, and Processors.
"},{"location":"user-documentation/searching/#view-tab","title":"View Tab","text":"The 'View' tab gives an overview of the index and its status:
The View tab also provides links to some common actions. Start Indexing Now allows you to start an indexing job for a specified number of items (default is 'all'). You can also specify how many items should be indexed in each batch (default is '50'). The other links allow a repository manager to queue all objects for reindexing, clear the index, or rebuild tracking information.
"},{"location":"user-documentation/searching/#edit-tab","title":"Edit Tab","text":"The Edit tab allows repository managers to configure how the index works as a whole, including the Index name, the data sources \u2014 entity types \u2014 it can index (including which specific content types or taxonomies will be indexed), which server it is connected to, and other SOLR-specific options.
Content (types) is the only data source enabled by default. Selecting Taxonomy term will enable searching taxonomies which is recommended if the repository uses taxonomies for subjects or other discovery points. Once the data sources are enabled a configuration box for each of them will appear in a section just below the list of data sources. This allows repository managers to select which content types (or taxonomy vocabularies) will be included in the index. By default, all the content types, and vocabularies if the taxonomy data source is enabled, are indexed.
Defaults
The defaults assume a repository is adding content using the web interface. If a repository manager plans on bulk-loading content they should disable the Index items immediately option in the expandable Index Options box and increase the 'Cron batch size' option.
"},{"location":"user-documentation/searching/#fields-tab","title":"Fields Tab","text":"The Fields tab allows repository managers to select which fields will be indexed. The default set of fields enabled come from a standard Drupal installation and do not reflect the fields Islandora adds for 'Repository Item'. Repository managers need to add the fields necessary for their Islandora instance.
To add a field, click the + Add fields button. A shadow-box will appear with a list of the fields available for the index.
Some fields, such as the Body ('body') field provided by Drupal, have multiple properties which can be completely different values or variations on the same value. Click on the plus-sign next to the field to show the properties available to index. In most cases repository managers can ignore the properties list and click the Add button by the field to index the default property ('value'). Only select a different field property if you understand how it will impact user searching. Entity reference fields, such as Tags ('field_tags'), allow you to select fields or their properties from the referenced entity for indexing, such as a referenced taxonomy term's name field.
Once the fields are added they can be configured further on the Fields tab, although the label, machine name, and type usually don't need to be changed. The 'Type' dropdown has several Full-text processing options available, which may be of interest. Each is described in the expandable Data Types box at the bottom of the page. The Boost setting allows repository managers to increase the weight of particular fields when calculating search relevancy.
"},{"location":"user-documentation/searching/#processors-tab","title":"Processors Tab","text":"The Processors tab allows repository managers to adjust how data and search queries are processed to adjust results. The defaults are acceptable in most cases.
"},{"location":"user-documentation/searching/#indexing-edtf-dates","title":"Indexing EDTF Dates","text":"EDTF date fields can only be successfully indexed directly as strings. However, there is a way to include a custom \"edtf year\" field containing the facets-friendly year (or year ranges) from one or more EDTF fields. This must first be enabled by checking \"EDTF Year\" under the Processors tab, then can be added on the Field tab (it'll be called \"EDTF Creation Date Year (edtf_year)\"), and finally can be configured as a facet. See \"How should this be tested\" on this pull request for instructions on setting it up.
"},{"location":"user-documentation/searching/#searching-islandora","title":"Searching Islandora","text":"Searching using Search API in Drupal is done using Drupal Views. The Islandora Starter Site comes with a search page pre-configured (accessible at '/solr-search/content'). To edit the search page, navigate to '/admin/structure/views/view/solr_search_content'.
Repository managers may want to change the URL used to access the page, add it to the site navigation, or add a search box. In the 'Page Settings' box in the middle of the page, click on the existing path to open a shadow-box with an edit field. Change the URL as desired, for example, to 'search' and click Apply. Then, click the No menu link just below it to open the menu settings shadow-box. Selecting 'Normal menu entry' will allow a repository manager to add a menu link text, description, and place it within the site menu tree (the default, <Main navigation> works for most sites). A search box can be added by expanding the Advanced options and changing the Exposed form in block setting and then use the Block Layout interface (found at '/admin/structure/block') to place the block where desired. After making changes to the View's settings, click the Save button to ensure the changes are not lost.
Islandora's Repository Items are displayed in the search results as a fully rendered entity by default. Repository managers can choose which view mode should be used for each search datasource by clicking the Settings link next to the Show: setting under the Format section of the search view configuration page (shown in a red box in the screenshot below). The Teaser and Search result highlighting input are the two most likely options. Alternatively, repository managers can select specific fields to display instead by clicking the Rendered Entity link and changing it to Fields and then choosing which fields will be displayed in the Fields section underneath.
Thumbnails
thumbnails will not immediately be available using the Fields display option without more advanced configurations.
"},{"location":"user-documentation/starter-site-metadata-configuration/","title":"Islandora Starter Site Metadata Configuration","text":""},{"location":"user-documentation/starter-site-metadata-configuration/#introduction","title":"Introduction","text":"As described in Metadata In Islandora, in Islandora metadata is stored in Drupal, in fields attached to entities. Provided by the Islandora Starter Site, the \u201cRepository Item\u201d content type contains a set of default fields to describe the digital objects an Islandora repository might contain. This set of fields is based on MODS fields commonly used in Islandora 7, and is not intended to be \"the standard\" metadata profile, rather, a starting point for institutions designing their own repository.
This page presents the primary descriptive and administrative metadata fields found in the Islandora Starter site. We will define each field and give its basic configuration. Most of these configurations can be customized after installation, and new fields can be added as per the needs of an individual institution. Fields are grouped in standard MODS order.
Further information on the metadata configuration can be found in a Google Spreadsheet Islandora Starter Site Metadata Configuration (Google Sheets) which parallels the information on this page. It goes into further detail about the configurations connected to each field, provides information on taxonomies and mappings, and can be filtered or sorted in a variety of ways. The document, or the spreadsheet above, can be copied and used as a basis for planning your own configuration customizations as you work on your Islandora site.
"},{"location":"user-documentation/starter-site-metadata-configuration/#administrativesystem-fields","title":"Administrative/System Fields","text":"Title The name given to the resource. Title is a system field and as such its configurations can not be adjusted. It is the only field that is required by the system. Machine Name title Drupal Field Type Text (plain) Required yes Maximum Length 255 (character length is set by system but can be changed with a contrib module, see below) Repeatable no CSL Citation Mapping title RDF Mapping dcterms:title XPath MODS mods/titleInfo/title Transformation To create a single string out of the subelements of <titleInfo>, we suggest to use the <titleInfo> section of the LOC MODS-DC transform https://www.loc.gov/standards/mods/v3/MODS3-5_DC_XSLT1-0.xsl. In words, it says to:It is worth noting that the subtitle, partNumber, and partName elements are technically repeatable, but this transform only uses the first value encountered. If you have repeating subelements, information will be lost. Drupal Module Integration The Title Length module allows you to extend the length of the title to 500 characters (possibly more). The Views Natural Sort module allows you to sort a view by titles while skipping a configurable list of non-sorting characters (\"A\", \"The\", \"L'\", etc.). It's not as precise as nonSort, but does most of the job. Alternatives Paragraphs are a way to model a multi-part title (nonSort, title, subtitle, partName, partNumber, etc). Combine paragraphs with (automatic entity label? automatic nodetitles?) to not have to enter your title information twice. (Who to contact with experience about this?) Member of This item's parent item in Islandora. Usually this will be a collection, book (\"Paged Content\"),or compound object. Machine Name field_member_of Drupal Field Type Entity reference Required no (will default to empty if no value is entered) Repeatable yes Create Referenced Entities no (can only be connected to existing entities) Facet Member Of Alternatives You could arrange your content with Entity Reference Hierarchy, which is a very scalable way of representing large hierarchy trees. However, you'd have to re-work a number of hard-coded elements in the islandora module. Model The internal-to-Islandora category of the resource. Affects how the item is displayed/viewed.
This field is actionable by Islandora and was designed to trigger derivatives and view modes through the use of Contexts. It is a controlled list and new values should only be added when required (i.e. new behaviors are made available and these values are created to trigger them). Machine Name field_model Drupal Field Type Entity reference Required yes Repeatable no Taxonomies Islandora Models (pre-populated/controlled)
This field does not allow creating names of entities on the fly. First create a term in either the person, family, or corporate body taxonomies, then link it here.
Field name changed from \"Linked Agent\" 09/23. Machine Name field_linked_agent Drupal Field Type Typed Relation [link to https://islandora.github.io/documentation/user-documentation/metadata/#typed-relation] Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Person Create Referenced Entities no Relators Starter Site includes 269 terms from the MARC Relators list. Further terms from the list, or custom terms, can be added in the configuration for this field. Publisher has been removed from this list to encourage use of the Publisher field. CSL Citation Mapping author ; contributor ; editor Facet Creators and Contributors RDF Mapping dcterms:contributor XPath For MODS mods/name/namePart + role/roleTerm > relator
mods/name[@type=\"personal\"]/namePart > \"Person\" taxonomy
mods/name[@type=\"corporate\"]/namePart > \"Corporate Body\" taxonomy
mods/name[@type=\"family\"]/namePart > \"Family\" taxonomy Alternatives Create custom facets using the \"Typed Relation filtered by type\" Search API processor. This would allow you to separate out, for instance, creator|author|photographer from the other types of relators as a Search API field/facet."},{"location":"user-documentation/starter-site-metadata-configuration/#type-and-genre","title":"Type and Genre","text":"Type The general nature or genre of the content of the resource. To describe the digital or physical format of the resource, use Form instead. In contrast with Model, which is system-actionable, this field is designed to record the Type purely as a metadata element. Machine Name field_resource_type Drupal Field Type Entity reference [link to https://islandora.github.io/documentation/user-documentation/metadata/#entity-reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable no Taxonomies Resource Types (pre-populated with hierarchical LOC Resource Type Scheme terms) Create Referenced Entities no Facet Resource Type RDF Mapping dcterms:type XPath For MODS mods/typeOfResource, mods/typeOfResource[@collection=yes] Alternatives You may be required to use the Dublin Core types (for compatibility/harvesting); in that case you can replace the current Type vocabulary contents with the desired values. Genre A term or terms that designate a category characterizing a particular style, form, or content, such as artistic, musical, literary composition, etc. Machine Name field_genre Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Genre Create Referenced Entities yes CSL Citation Mapping genre RDF Mapping dcterms:type XPath For MODS mods/genre; mods/subject/genre"},{"location":"user-documentation/starter-site-metadata-configuration/#origin-information","title":"Origin Information","text":"Place Published Plain text field to describe the place of publication in full or transcribed from an item \\ \\ See also the entity reference field Place Published Country. Machine Name field_place_published Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes CSL Citation Mapping publisher-place RDF Mapping relators:pup XPath For MODS mods/originInfo/place/placeTerm [@type = 'text'] OR [not(@type)] Country of Publication Entity reference field to singularly describe the country (or jurisdiction) of publication. Connected to the unpopulated Country taxonomy by default, which can be used for terms or codes from MARC Code List for Countries or elsewhere. Disabled, by default, on Metadata Display and Metadata Form. Machine Name field_place_published_country Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable no Taxonomies Country Create Referenced Entities no RDF Mapping relators:pup XPath For MODS mods/originInfo/place/placeTerm[@type=\"code\"] Publisher New field added 04/2024. Previously the Contributors/Linked agent field (with a relator value of Publisher) was used for this content. Machine Name field_publisher Drupal Field Type Text (plain) Required no Maximum Length 500 Repeatable yes CSL Citation Mapping publisher RDF Mapping XPath For MODS mods/originInfo/publisher"},{"location":"user-documentation/starter-site-metadata-configuration/#date-fields","title":"Date FieldsDate IssuedDate CreatedDate (Unspecified)Copyright DateDate ValidDate CapturedDate ModifiedEdition StatementMode of IssuanceFrequency","text":"
All date fields use the Drupal EDTF field type and must follow EDTF formatting. For more information on this format and how it works with Islandora see https://islandora.github.io/documentation/user-documentation/metadata/#edtf
Date of formal issuance of the resource. This includes publication dates. Machine Name field_edtf_date_issued Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes CSL Citation Mapping issued Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:issued XPath For MODS mods/originInfo/dateIssued Date of creation of the resource Machine Name field_edtf_date_created Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:created XPath For MODS mods/originInfo/dateCreated A date without a type or relationship to the resource specified Machine Name field_edtf_date Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:date XPath For MODS mods/originInfo/dateOther Date of copyright of the resource. Machine Name field_copyright_date Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:dateCopyrighted XPath For MODS mods/originInfo/copyrightDate Date (often a range) of validity of a resource. Machine Name field_date_valid Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping dcterms:valid XPath For MODS mods/originInfo/dateValid The date on which the resource was digitized or a subsequent snapshot was taken. Machine Name field_date_captured Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping premis:creation XPath For MODS mods/originInfo/dateCaptured Date on which the original resource being represented in Islandora was changed. Typically modification dates of digital representations of the resource stored in Islandora will be recorded on the relevant Media instead of here. This field is not populated automatically by any Drupal functionality. Machine Name field_date_modified Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping dcterms:modified XPath For MODS mods/originInfo/dateModified Information identifying the version of the resource. Machine Name field_edition Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping rdau:P60329 XPath For MODS mods/originInfo/edition A term that designates how the resource is issued. MODS standards limit the values of this field to monographic, single unit, multipart monograph, continuing, serial, and integrating resource, but there are no such limitations in Islandora and the \u201cIssuance Mode\u201d vocabulary is empty on initial installation. Machine Name field_mode_of_issuance Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Issuance Mode Create Referenced Entities yes RDF Mapping rdau:P60051 XPath For MODS mods/originInfo/issuance The publication frequency, in textual form. Can be based on MARC Frequency terms. Machine Name field_frequency Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Frequency Create Referenced Entities yes RDF Mapping rdau:P60538 XPath For MODS mods/originInfo/frequency"},{"location":"user-documentation/starter-site-metadata-configuration/#language","title":"Language","text":"Language Language of the resource content. Can be based on a languages vocabulary such as ISO 639-2.MARC Code List for Languages.This field is not connected to Drupal system language fields or related to Drupal translation functionality. Machine Name field_language Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Language Create Referenced Entities yes CSL Citation Mapping language RDF Mapping dcterms:language XPath For MODS mods/language/languageTerm[@type = 'text'] or [@type = 'code'] mapped to text per iso639-2b"},{"location":"user-documentation/starter-site-metadata-configuration/#physical-description","title":"Physical Description","text":"Form The physical format of the original resource being described. If the resource is a physical object, the physical form is recorded here. If the resource is born-digital, the original digital form is recorded here. Details of the formats of resource representations stored in Islandora should be recorded on the relevant Media files. Machine Name field_physical_form Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Physical Form Create Referenced Entities yes Facet Physical Form RDF Mapping dcterms:format XPath For MODS mods/physicalDescription/form Extent The size or duration of the resource in its original form. Extent of representations of the resource stored in Islandora should be recorded on the relevant Media files. Machine Name field_extent Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:extent XPath For MODS mods/physicalDescription/extent"},{"location":"user-documentation/starter-site-metadata-configuration/#descriptions","title":"Description(s)","text":"Description Description of the resource. Description may include but is not limited to: an abstract, a table of contents (alternately its own field), a graphical representation, or a free-text account of the resource. Machine Name field_description Drupal Field Type Text (plain, long) Required no Repeatable no CSL Citation Mapping abstract RDF Mapping dcterms:description XPath For MODS mods/abstract (xpath could also map to Abstract; do not map same values to both) Abstract Supplementary to the Description field, for including descriptions that more fit the definition of Abstract as it refers to ETDs, journal articles, or other scholarly publications. Machine Name field_abstract Drupal Field Type Text (formatted, long) Required no Repeatable no RDF Mapping dcterms:abstract XPath For MODS mods/abstract (xpath could also map to Description; do not map same values to both) Table of Contents Supplemental to Description, for breaking out specific Table of Contents information separate from a summary description of the object. Allows formatting. Machine Name field_table_of_contents Drupal Field Type Text (formatted, long) Required no Repeatable no RDF Mapping dcterms:tableOfContents XPath For MODS mods/tableOfContents Note General textual information relating to a resource, not described in other fields. Machine Name field_note Drupal Field Type Text (formatted, long) Required no Repeatable yes RDF Mapping skos:note XPath For MODS mods/note (with [@displayLabel] or [@type] prepended with ': ' to the note)"},{"location":"user-documentation/starter-site-metadata-configuration/#subjects","title":"Subject(s)","text":"Subject General subjects which may be constructed from topical, geographic, temporal, and genre subdivisions. If you wish to manage these types of subjects separately, use the more specific Subject fields below.
This field can be used if you are storing composed subject strings rather than component headings/subdivisions in their own fields. Machine Name field_subject_general Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Geographic Location; Person; Subject Create Referenced Entities yes Store New Items In Subject Facet Subject RDF Mapping dcterms:subject XPath For MODS mods/subject Subject (Topical) For use when using separate fields to record topical, geographic, name, and temporal terms. Topical subject terms would be recorded here. Machine Name field_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Subject Create Referenced Entities yes RDF Mapping dcterms:subject XPath For MODS mods/subject/topic Subject (Geographic) For use when using separate fields to record topical, geographic, name, and temporal terms. Geographic subject terms would be recorded here. Machine Name field_geographic_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Geographic Location Create Referenced Entities yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/geographic Subject (Name) For use when using separate fields to record topical, geographic, name, and temporal terms. Names of individuals, organizations, or families would be recorded here. Machine Name field_subjects_name Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Person Create Referenced Entities yes Store New Items In Person Facet Subject (name) RDF Mapping dcterms:subject XPath For MODS mods/subject/name/namePart Subject (Temporal) For use when using separate fields to record topical, geographic, name, and temporal terms. Temporal subject terms would be recorded here. Machine Name field_temporal_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Temporal Subjects Create Referenced Entities yes CSL Citation Mapping Facet Temporal Subject RDF Mapping dcterms:temporal XPath For MODS mods/subject/temporal"},{"location":"user-documentation/starter-site-metadata-configuration/#coordinates","title":"Coordinates","text":"Coordinates Must be formatted either in decimal (51.47879) or sexagesimal format (51\u00b0 28' 43.644\") Machine Name field_coordinates Drupal Field Type Geolocation Required no Repeatable yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/cartographics/coordinates Coordinates (Text) A plain text field meant as an alternative to the Coordinates field, for values that do not meet the requirements of the Drupal geolocation field type. Machine Name field_coordinates_text Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/cartographics/coordinates"},{"location":"user-documentation/starter-site-metadata-configuration/#classification","title":"Classification","text":"Dewey Classification Classification based on the Dewey Decimal system. Machine Name field_dewey_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification[@authority=\"ddc\"] Library of Congress Classification Classification based on the Library of Congress Classification system. Machine Name field_lcc_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification[@authority=\"lcc\"] Classification (Other) Classification based on any system that does not have a dedicated field. Machine Name field_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification"},{"location":"user-documentation/starter-site-metadata-configuration/#identifiers","title":"Identifier(s)","text":"Identifier A unique standard number or code that distinctively identifies a resource. Machine Name field_identifier Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:identifier XPath For MODS mods/identifier[not(@type)] ISBN An International Standard Book Number (ISBN) identifier. Machine Name field_isbn Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dbpedia:isbn XPath For MODS mods/identifier[@type=\"isbn\"] OCLC Number An identifier assigned by an OCLC system, such as WorldCat. Machine Name field_oclc_number Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dbpedia:oclc XPath For MODS mods/identifier[@type=\"oclc\"] Local Identifier Identifier from any locally managed system. Machine Name field_local_identifier Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:identifier XPath For MODS mods/identifier[@type=\"local\"] PID PID of this object in Islandora 7.x (applies to migrated objects only)
Disabled, by default, in Metadata Display. Machine Name field_pid Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable no"},{"location":"user-documentation/starter-site-metadata-configuration/#access-conditions","title":"Access Conditions","text":"Rights Free text field for information about restrictions imposed on, and/or rights to, access to a resource, particularly a digital resource. It is not connected to any access control functionality within Drupal. Machine Name field_rights Drupal Field Type Text (formatted, long) Required no Repeatable yes RDF Mapping dc11:rights XPath For MODS mods/accessCondition"},{"location":"user-documentation/transcripts/","title":"Transcripts","text":"
WEBVTT transcripts, captions, or subtitles may be displayed along with audio and video media. Transcripts must be added to the media entity that will be playing, using a special field type (\"Media track\"). When the media is configured to use the \"Audio with Captions\" and \"Video with Captions\" field formatters, tracks in \"Media track\" fields will be available.
"},{"location":"user-documentation/transcripts/#media-track-field-type","title":"Media track field type","text":"The Islandora module provides a field type, Media Track, that accepts .vtt files. Once a file has been uploaded, additional options relevant to media tracks become available to configure.
"},{"location":"user-documentation/transcripts/#using-transcripts-with-the-islandora-starter-site","title":"Using Transcripts with the Islandora Starter Site","text":"In the Islandora Starter Site, Audio and Video media are pre-configured to work with tracks.
Sandbox
On the public sandbox, or other sites using the Islandora Install Profile Demo, you will first need to make the \"Track\" field visible in the media form, at Structure > Media > Audio|Video > Manage Display.
"},{"location":"user-documentation/transcripts/#video","title":"Video","text":"Selecting the correct media
Note that if you add the transcript file to the Original File, but you have configured the site to play the Service File, then you will not see the transcript.
Languages
While you can add subtitles in different languages, you may only choose from the site's installed languages.
Types
The five options: captions; subtitles; descriptions; chapters; and metadata come from the HTML standard's <track> element. As per their definitions, captions and subtitles will be displayed as optional text over the video, available through the usual [cc] icon in the viewer controls. Descriptions, chapters, and metadata will not be displayed as they are intended for programmatic use.
If you want to upload large (> 1GB) files, you'll need to tune the following settings in your php.ini:
upload_max_filesize \u2013 The maximum allowed upload file size.post_max_size \u2013 The maximum allowed POST data size.max_input_time \u2013 Maximum allowed input time.max_execution_time \u2013 The maximum allowed time the scripts are allowed to run.default_socket_timeout - Default timeout (in seconds) for socket based streams.However, large file transfer over HTTP still has a host of issues once you properly configure your server. Uploads are not resumable and subject to connectivity issues. If you really want to upload large files, you should consider some alternatives such as
If loading large (e.g. range 30-45 GB) files into Fedora, you may need to change the fcrepo.session.timeout property, which defaults to 3 minutes (180,000 ms). Documentation is on the Properties page on the Fedora wiki.
If using FITS, you may need to change the following in /var/lib/tomcat9/webapps/fits/WEB-INF/classes/fits-service.properties:
# Maximum allowable size of uploaded file\nmax.upload.file.size.MB=2000\n# Maximum size of HTTP Request object. Must be equal to or larger than the value for max.upload.file.size.MB\nmax.request.size.MB=2000\n# Maximum size of an uploaded file kept in memory. Otherwise temporarily persisted to disk.\nmax.in.memory.file.size.MB=4\nA URL alias is an alternate URL pattern that resolves to a Drupal entity, such as a node, media, taxonomy term, or user. For example, you could set \"/welcome\" as an alias for \"/node/1\". Aliases are part of Drupal Core and can be extended by contrib modules. One such module is Pathauto, which enables automatic alias generation based on patterns, and the patterns may involve \"Tokens\" (such as [node:title]).
This page will attempt to cover the Islandora Starter Site's use of aliases and what we consider to be best practices. A full description of creating and managing aliases is out of scope.
"},{"location":"user-documentation/url-aliases/#best-practices-with-url-aliases","title":"Best practices with URL aliases","text":"While every site may choose to set up their aliases differently, we cannot prescibe a universal setup.
A common \"nice-to-have\" is the presence of the slug /islandora/ in the URL which identifies the content as \"Islandora\".
A potential \"best practice\" is that if your site uses persistent identifiers such as DOIs or Handles, that those identifiers make up part of the URL alias.
"},{"location":"user-documentation/url-aliases/#use-of-url-aliases-in-islandora-starter-site","title":"Use of URL aliases in Islandora Starter Site","text":"The Islandora Starter Site includes the Pathauto module, which we consider that most sites will want to use in some way. However its default configuration should not be interpreted as prescriptive. You are encouraged to use persistent identifiers if you have them!
The default Pathauto pattern for Repository Items is /islandora/[node:title] with the pathauto configuration trimming the alias at 100 characters.
Drupal 8 supports various web analytics integrations such as Google Analytics and Matomo. For privacy reasons, Islandora integrates Matomo web analytics platforms by default.
The Matomo server is installed here: http://localhost:8000/matomo. The default configurations can be found in http://localhost:8000/admin/config/system/matomo. You can log in to the Matomo server using username admin and password islandora. The dashboard will look like this:
To see page views, login to Matomo and go to Behaviour >> Pages.
"},{"location":"user-documentation/usage-stats/#further-reading","title":"Further Reading","text":"This Using Islandora section is aimed at site administrators and repository managers who need to understand and configure their Islandora repositories. It will go in-depth on how Islandora allows you to use the various features of Drupal to construct and display repository items. For easy readability, we have divided this documentation into five major sections, organized by functionality.
Islandora, like Drupal, provides tools to create a site, but does not force you to conform to any specific site structure, organization, or navigation. There is a hope that we can provide something useful out of the box, while also allowing the full suite of Drupal configuration options. This out-of-the-box configuration is the Islandora Demo module.
As you read this documentation, it is recommended to be familiar with the basics of Drupal, including content types, fields, users, and views. The Official Drupal User Guide and the Community Guide to Drupal are a good place to start.
"},{"location":"user-documentation/user-intro/#modeling-content","title":"Modeling content","text":"In Islandora, we often want to store (and manage, and preserve) sets of metadata with binaries (digital files). Previously in Islandora Legacy, these components were referenced together as a single \"object.\" In Islandora, metadata and their binaries are now represented as multiple interconnected entities.
Metadata is stored in nodes (a.k.a. content).
Binary files are stored in are media, which are wrapper entities to help manage files.
Metadata values can be stored as taxonomy terms, which let you reuse the same value in multiple places.
For users familiar with Islandora Legacy, the relationship between Fedora and Islandora has greatly changed in the current version of Islandora.
Islandora Legacy inherited its object model from Fedora 3.x. In Legacy, Fedora stored all properties and content associated with an object - not only its owner, dc.title, status, PID, and status - but also any content files such as OBJ, DC, MODS, and RELS-EXT. In Islandora Legacy, Fedora acted as the authoritative, primary source for all aspects of an object. Fedora 3.x was not an optional component of an Islandora Legacy repository, instead it served as the primary datastore.
In Islandora, using Fedora is optional. That's right, optional. Drupal, and not Fedora, is the primary source of all aspects of an Islandora object, and, with some variations, Drupal, not Fedora, is the primary datastore in an Islandora repository. If Fedora is present in an Islandora repository, content in it is a tightly synchronized copy of object properties and files managed by Drupal.
Even though Fedora is optional in Islandora, most repositories will use it since it provides its own set of services that are worth taking advantage of, such as:
In Islandora repositories that use Fedora, all properties about Drupal nodes are mirrored in Fedora as RDF properties. But, even if an Islandora instance does not use Fedora, Drupal can provide an object's properties as RDF (again, Drupal is the primary source of data in Islandora). In addition, the Drupal media associated with Islandora objects are persisted to Fedora, although exactly which media is configurable within the Islandora admin interface. Just as Drupal out of the box has a public and private filesystem, Islandora adds a third filesystem to Drupal called, not surprisingly, \"fedora\", and it is to this filesystem that media are persisted. We will provide more information about Fedora's role in an Islandora repository in the metadata and media sections.
"},{"location":"user-documentation/user-intro/#architecture","title":"Architecture","text":""},{"location":"user-documentation/user-intro/#conceptual-diagram","title":"Conceptual diagram","text":"Many users of Islandora may be familiar with the metaphorical diagram of Islandora Legacy as a cheeseburger, which provides a memorable approximation of how the different parts of the software stack interact in a vertically-integrated, relatively customizable fashion (ie, Drupal, Solr, Islandora, and Fedora are stable layers, and the \"toppings\" stand in for Solution Packs and other utilities that can be added or removed to customize Islandora):
For a similar conceptual approach to Islandora, we present it as a bento box: a very modular platform, in which each piece may be removed and replaced with something different, without disrupting other parts of the stack:
For a true diagram of how the various parts of Islandora interact, please see the full Architecture Diagram.
"},{"location":"user-documentation/users/","title":"Managing Users","text":""},{"location":"user-documentation/users/#overview","title":"Overview","text":"Anyone who visits your Drupal site is a user. There are three different default users in Drupal:
Islandora adds an additional default role:
Additional user roles can be created and assigned customized permissions, as described below.
"},{"location":"user-documentation/users/#before-you-start","title":"Before you start","text":"Warning
If you are writing to Fedora, your username must not contain spaces.
"},{"location":"user-documentation/users/#how-to-add-a-user","title":"How to Add a User","text":"To review/edit the permission for each role, in the People page click the Permissions tab in the set of tabs above the Add user button.
"},{"location":"user-documentation/users/#how-to-create-a-new-user-role","title":"How to Create a New User Role","text":"Islandora Quick Lessons
Learn more with this video on how to Add a User.
"},{"location":"user-documentation/users/#video-walk-through-users-roles-and-permissions","title":"Video Walk-through: Users, Roles, and Permissions","text":"Click the image below to open the Users, Roles, and Permissions video tutorial on the Islandora YouTube channel.
See more videos from the Drupal 101 series here.
"},{"location":"user-documentation/users/#further-reading","title":"Further Reading","text":"For more information on managing users in Drupal visit the section Managing User Accounts of Drupal.org.
"},{"location":"user-documentation/versioning/","title":"Versioning","text":"As a user of an Islandora repository, you may be wondering - Is this content being versioned? Could I restore from a previous version if I needed to? Can I see a list of versions for an object? The answer to these questions is two-fold, and largely yes. The architecture of Islandora provides users with a Drupal implementation and a Fedora implementation which are connected in Islandora.
Islandora Software Versioning
Looking for information about versions of Islandora itself? The latest Islandora follows semantic versioning. Previously, Islandora's versions were tied to the version of Drupal and numbered in order of release, such as Islandora 6.x-13.1 or Islandora 7.x-1.13. More information.
"},{"location":"user-documentation/versioning/#drupal-revisioning","title":"Drupal Revisioning","text":"Drupal provides a concept of revisions which allows you to track the differences between multiple versions of your content and revert to older ones. The list of revisions for a node are available at http://localhost:8000/node/1/revisions. There are Drupal docs on revisioning. Media objects are also versioned in Drupal but there is not a UI component for this - see related issue.
"},{"location":"user-documentation/versioning/#fedora-and-memento","title":"Fedora and Memento","text":"Fedora implements the Memento specification for versioning resources, which is a time-based HTTP framework. Fedora provides documentation as well as an API implementation.
"},{"location":"user-documentation/versioning/#basic-data-flow","title":"Basic Data Flow","text":"Islandora Quick Lessons are a series of short videos demonstrating how to do common tasks in Islandora.
New videos are added to the playlist regularly.
"},{"location":"user-documentation/video-docs/#general-information","title":"General Information","text":"The following recipe details how to connect Islandora with Alexa, using custom Alexa skills and the Drupal Alexa module. The potential applications are broad: - Send Citations, metadata, whatever we want to the user if they have set up their email - Creating a collaborative exhibit - Play audio and video objects and read transcripts - Respond to user search queries (like how many objects match the subject in the repositories) - Answer specific questions about the object (\u201cInvocation Name, when was this recorded?\u201d) - Interact with other applications or modules (got a print ordering system? Want to add event calendar items to your exhibit?) - Be accessed via web page, Alexa device, or phone app
Note
This recipe has not been extensively tested.
"},{"location":"user-documentation/recipes/alexa-search/#ingredients","title":"Ingredients","text":"composer require \"drupal/alexa\"Islandora is an open-source framework that provides the necessary tools to use a Drupal website as a fully-functional Digital Assets Management System.
Islandora:
Can handle messages at scale - Islandora created Alpaca, an integration middleware based on Apache Camel, to handle messaging and queueing at an enterprise scale. To the user, this means large batch uploads can be processed gracefully.
Offers digital preservation features - Using a robust storage layer for preservation (Fedora), and generating technical metadata with FITS, are digital preservation tools provided by our automated installation methods. Community members have created additional features for doing digital preservation using Islandora, which are not yet part of our automated setup.
Try Islandora without installing anything at sandbox.islandora.ca.
Login credentials for the sandbox can be found here.
Anyone can log in to this sandbox as an administrator and explore the interface! However, this site is refreshed every night so your changes will not be permanent. This site uses the Islandora Starter Site. This is not the only way that Islandora can be made to work! This sandbox includes some sample content and configuration (such as views and blocks) to increase its usefulness as a sandbox. Learn more about the Islandora Starter Site.
"},{"location":"#ansible-playbook","title":"Ansible Playbook","text":"Islandora can be installed via an an Ansible Playbook, a machine-operable set of instructions which provisions the full Islandora stack. It can be used to create a locally installed Islandora (requiring your computer to have Vagrant and Virtualbox) or can be used to provision a remote Linux server. The provisioning process involves many steps where software is downloaded and installed, so it can take a while. There is an option to get a basic (\"standard\") site, or to install a suite of demo configurations known as the Demo Install Profile. See Installation - Ansible Playbook for more details.
"},{"location":"#docker","title":"Docker","text":"Islandora sites can also be created using Docker. This can be done using the ISLE-DC project, which launches the Islandora Docker images created by Isle Buildkit. Like in the Ansible Playbook, there is an option to use a pre-built demo site, or build it completely from scratch. (There is also an option to build the demo site on command, which takes a bit more time, or to use ISLE-DC to build up the environment to support a Drupal site that you have already exported.) See Installation - Docker ISLE for more details
"},{"location":"#join-the-community","title":"Join the Community","text":"The Islandora community is an active group of users, managers, librarians, documenters, and developers from galleries, libraries, archives, museums, and other institutions worldwide. We welcome discussion and contribution through various mailing lists, channels, interest groups, and calls. The Islandora community operates under the Islandora Code Of Conduct. See our Contributing Guidelines for more information.
Documentation for previous versions
Documentation for Islandora 6 and 7 is on the Lyrasis documentation wiki.
"},{"location":"placeholder/","title":"Placeholder page","text":"This page serves as a placeholder for functionality that still needs documentation.
Interested in contributing to the Islandora documentation? Join the Community!
The Islandora community is an active group of users, managers, librarians, documenters, and developers from GLAM (and beyond!) institutions worldwide. We welcome discussion and contribution through various mailing lists, channels, interest groups, and calls. The Islandora community operates under the Islandora Code Of Conduct. See our Contributing Guidelines for more information, or drop by a meeting of the Documentation Interest Group for a helping hand.
Documentation for previous versions
Documentation for Islandora 6 and 7 is on the Lyrasis documentation wiki.
"},{"location":"alpaca/alpaca-technical-stack/","title":"Alpaca Technical Stack","text":"As of version 2.0.0, Alpaca contains several tools bundled into a single runnable jar file. The different tools can be enabled/disabled depending on the configuration you define.
"},{"location":"alpaca/alpaca-technical-stack/#gradle","title":"Gradle","text":"Gradle is used by Alpaca as a build and package management tool. It is similar to Maven.
"},{"location":"alpaca/alpaca-technical-stack/#apache-camel","title":"Apache Camel","text":"Apache Camel is an integration framework that aids in implementing integration patterns.
"},{"location":"alpaca/alpaca-technical-stack/#apache-activemq","title":"Apache ActiveMQ","text":"Apache ActiveMQ is a JMS compliant Messaging Queue. Messaging client can make use of JMS to send messages.
"},{"location":"alpaca/alpaca-technical-stack/#installing-activemq","title":"Installing ActiveMQ","text":"Installing ActiveMQ is relatively easy. Download the latest stable release here. Go to the activemq_install_dir/bin. Start the ActiveMQ by using the activemq script or batch file and start command.
$ cd activemq_install_dir/bin\n$ ./activemq start\nWhen ActiveMQ gets started, go to http://localhost:8161/admin/. You can login using admin:admin.
Note that ActiveMQ in Islandora playbook does not have a UI.
"},{"location":"alpaca/alpaca-technical-stack/#references","title":"References","text":"Collections are groups of related content that can be viewed or managed as a unit. Islandora-specific use cases include:
Islandora provides:
field_member_of). This mechanism is also used by Paged Content and Compound Objects. Islandora on its own does not prescribe any particular Content Type, so this field can be configured for any node bundle intended to represent Islandora resources.field_model) which can take various values including \"Collection\".Islandora Starter Site is an optional set of presets for Islandora, intended to provide a more user-friendly out-of-the-box experience and starting point for more specific customization.
Islandora Starter Site provides:
field_member_of field, so that users may add nodes of this type to a collection (or paged content, or compound resource),For more details, see the tutorial on How to create and add to a collection
"},{"location":"concepts/collection/#bulk-management-of-members-of-a-collection","title":"Bulk management of members of a collection","text":"Bulk management of items can be done from a Views Bulk Operations-compatible View, including the main Drupal Content View (at admin/content) and the Manage Members View (at node/[node]/members). This is possible due to the Drupal contrib modules Views Bulk Operations (VBO) and Views Bulk Edit. VBO allows you to perform Drupal Actions on objects. These Actions include making Islandora Derivatives such as \"Audio - Generate a Service File from an Original File\" and also to perform Drupal core Actions such as publishing/unpublishing content. The Views Bulk Edit module extends VBO and allows you to edit field values (such as the Member Of field, which would change which collection a group of nodes are in).
For instance, if you want to move a number of nodes from one collection into another, then you can
For more information see the video tutorial on Batch Editing.
"},{"location":"concepts/collection/#deleting-a-collection","title":"Deleting A Collection","text":"Deleting Collections creates orphans.
When you delete a Collection object, it will be deleted from the \"Member of\" field of all its Children. This means that it will not be possible to track down the children after the Collection has been deleted. If you intend to move the children, see above. If you intend to delete the children, this must be done before-hand. This is a known issue.
Deleting content creates orphan media.
Deleting content (Collections or otherwise) will NOT delete any attached Media. This is a known issue.
"},{"location":"concepts/collection/#permissions-on-a-collection","title":"Permissions on a Collection","text":"No reliable access control methods currently allow you to assign people to administer and manage only specific collections.
"},{"location":"concepts/derivatives/","title":"Derivatives","text":"Derivatives are files, usually generated automatically from a source file, which may be useful to the repository. Examples of derivatives include:
There are two schemes you can use for derivatives:
Derivative configuration is stored using Drupal's Actions. This means that all derivative configuration, such as parameters dictating the derivative size and quality can be edited by a repository administrator in the Drupal GUI under Manage > System > Actions.
As Actions, they can be executed on nodes manually using Views Bulk Operations. They can also be configured to run automatically on media save thanks to Islandora's additions to the Drupal [Contexts] module.
Derivative actions will replace existing files
If a derivative action runs but the target derivative (as identified by its taxonomy term) already exists, the new file will replace the old file (leaving the rest of the Media intact).
"},{"location":"concepts/derivatives/#derivatives-can-run-automatically-with-contexts","title":"Derivatives can run automatically with Contexts","text":"With the Contexts module, you can configure derivatives to run under specific conditions. These are set in the \"Conditions\" section. Islandora provides a number of context conditions including:
You can set up as many of these as you like, with \"and\" or \"or\" logic between them.
In the \"Reactions\" section is where you can set up derivatives. For standard model derivatives, choose the \"Derivatives\" reaction. This lists all actions, including derivative actions. Note that multifile derivatives won't work here.
Multi-file media
The multi-file media derivatives can NOT be selected from within the \"Derivatives\" reactions. From the \"Reactions\" pop-up window, you must choose \"Derive file for Existing Media\". This panel lists only Multi-file media-type derivatives.
"},{"location":"concepts/derivatives/#derivatives-have-types","title":"Derivatives have Types","text":"When creating a new Derivative Action, there are a number of flavours of derivative \"types\" available. All derivative Actions fall into one of these types.
Derivative type name machine name Supplying module Expected Microservice (software) Generate a Technical metadata derivativegenerate_fits_derivative Islandora FITS (roblib) Crayfits (FITS) Generate a audio derivative generate_audio_derivative Islandora Audio Homarus (FFmpeg) Generate a video derivative generate_video_derivative Islandora Video Homarus (FFmpeg) Generate an image derivative generate_image_derivative Islandora Image Houdini (ImageMagick) Get OCR from image generate_ocr_derivative Islandora Text Extraction Hypercube (tesseract/pdftotext) Multi-file media
The derivatives types available for multi-file media are the ones marked as \"for Media Attachment\" e.g. \"Generate an Image Derivative for Media Attachment\".
"},{"location":"concepts/derivatives/#derivatives-are-created-by-microservices-crayfish","title":"Derivatives are created by microservices (\"Crayfish\")","text":"In Islandora, we generate derivatives using microservices, which are small web applications that do a single task. Each microservice takes the name of a file as well as some parameters. It runs an executable and returns a transformed file, which can be loaded back into the repository. The microservices in Islandora stack are:
Repository Microservice name executable Crayfish Homarus FFmpeg Crayfish Houdini ImageMagick Crayfish Hypercube tesseract/pdftotext Crayfits Crayfits FITS"},{"location":"concepts/derivatives/#derivatives-are-created-using-an-external-queue","title":"Derivatives are created using an external queue","text":"To send orders to the microservices, Islandora sends messages to an external queue, from which the microservices process the jobs as they are able. This is a robust system that can \"operate at scale\", i.e. can large handle batches of uploads without slowing down the repository.
The queue system used by Islandora is ActiveMQ, and the listeners are part of \"Alpaca\"
"},{"location":"concepts/derivatives/#derivative-swimlane-diagram","title":"Derivative Swimlane Diagram","text":"The following diagram shows the flow of derivative generation from start to finish. A user saves a Media in Drupal, which may trigger Drupal to emit a derivative event to a queue, which is read by Alpaca and sent to a microservice. The microservice gets the original file and makes a transformation, returning the derivative file to Alpaca, which sends it back to Drupal to become a Drupal Media.
"},{"location":"concepts/node-concepts/","title":"Node Concepts","text":"This page describes the core fields and features that Islandora uses to manage content as nodes in an Islandora repository.
"},{"location":"concepts/node-concepts/#nodes-hold-metadata","title":"Nodes hold metadata","text":"In Islandora, Drupal nodes are created to hold descriptive metadata about content in the repository. This metadata is held in the usual way for nodes, which is by populating Drupal fields. Fields are configured on a Drupal content types, which serve as metadata profiles.
"},{"location":"concepts/node-concepts/#nodes-can-have-memberschildren","title":"Nodes can have members/children","text":"To build the capacity for creating hierarchical structures like collections, books and their pages, and complex objects, Islandora introduces a special field, \"Member of\" (field_member_of) which must be present on all Islandora content types. It enables a \"Children\" tab to display on Islandora nodes which lists the children of that node, and allows a repository manager to perform bulk operations on them. It also enables a repository manager to create children of nodes directly, individually or in bulk.
Within a single content type (i.e. metadata profile), Islandora provides the ability to designate some objects as different \"types\" than others. Key behaviours, such as what derivatives are created or what viewer is used, can be configured (see Contexts) based on this value. The available values are taxonomy terms in the Islandora Models vocabulary, and they are attached to nodes via the special mandatory field, \"Model\" (field_model), which must be present on all Islandora content types. These values are installed through a Drupal Migration after the Islandora module is installed. All installation methods perform this migration, so out of the box, the following values should be available in the Islandora Models vocabulary:
With Islandora alone, choosing a value from this list will have zero effects. The contingent behaviour must be configured during repository implementation. Islandora Starter Site provides an example of what behaviours are possible for these types.
The External URI field
This vocabulary, like many others in Islandora, includes an External URI field. This is intended to be used when transforming Islandora content into RDF, but also serves to make it easier to share configuration. Islandora provides code so that context conditions and derivative configs can be created without referencing the taxonomy term by ID, rather, using the taxonomy term's External URI. Since terms IDs are likely to change across sites, this makes our configs more portable.
"},{"location":"concepts/node-concepts/#nodes-are-attached-to-media","title":"Nodes are attached to Media","text":"In an Islandora repository, the files in the repository are uploaded as Media, which are linked to the node providing the descriptive metadata. Media belonging to a specific node can be found in the Islandora-provided \"Media\" tab on that node. For more, see the [Media in Islandora] section.
"},{"location":"concepts/node-media/","title":"Media in Islandora","text":"In Islandora, Media are created to hold the various files that pertain to the descriptive metadata provided by a Node. Islandora provides some fields and features to make this possible.
"},{"location":"concepts/node-media/#media-are-wrappers-for-files","title":"Media are wrappers for files","text":"Drupal Media are said to be wrappers around files, that allow that file to have fields attached. These fields may contain technical metadata about that file, such as its mimetype or size. This is how Islandora uses Media (though our approach to [technical metadata] differs).
"},{"location":"concepts/node-media/#media-may-store-files-in-different-locations","title":"Media may store files in different locations.","text":"Configuration on Media types determines where uploaded files will be stored - for example, the Drupal public or private filesystem, or through a tool called Flysystem to another data store such as Fedora. Islandora does not dictate where you put your files. Islandora Starter Site sets all media types to store their files in Fedora. This can be overridden when creating media programmatically, such as for derivatives.
"},{"location":"concepts/node-media/#media-belong-to-nodes","title":"Media belong to nodes","text":"While in Drupal, media are said to be \"reusable files\" that can be referred to by any number of content items, in Islandora there is a special relationship, the \"Media Of\" (media_of) field, that links media to a single node that describes that file and how it fits into the repository. The field \"Media Of\" must be on all Islandora media types for media of those types to behave as Islandora media.
Then, the media \"of\" an Islandora node (if any) will appear in that node's \"Media\" tab. From this tab, there are also links to add new Media, individually or in batch. When media are created this way, their \"Media of\" field is automatically populated with the current node.
"},{"location":"concepts/node-media/#media-have-different-uses","title":"Media have different uses","text":"In a repository, a node may have several media that belong to it, which represent the originally uploaded file, a smaller service file, a thumbnail, and perhaps a file transformed into a specific format for preservation. All of these media can be distinguished using the Islandora-provided field, \"Media Use\" (field_media_use). This is a taxonomy reference field that points to the vocabulary, Islandora Media Use.
The Islandora module provides, through a migration, the following values:
Media Use term External URI Extracted Text http://pcdm.org/use#ExtractedText Intermediate File http://pcdm.org/use#IntermediateFile Original File http://pcdm.org/use#OriginalFile Preservation Master File http://pcdm.org/use#PreservationMasterFile Service File http://pcdm.org/use#ServiceFile Thumbnail Image http://pcdm.org/use#ThumbnailImage Transcript http://pcdm.org/use#TranscriptThis migration is performed by all installation methods, so these values should be available \"out of the box\". These values were provided by the PCDM data model (see RDF in Islandora), and not all values are associated with behaviours you might expect - for instance, Islandora is not configured out-of-the box to display a transcript if you add a Media (file) and tag it as \"Transcript\". See Audio and Video for setting up transcripts.
Standard vs Multi-file media model
This describes the standard file-media relationship in Islandora. There is an alternative method of arranging files and their derivatives which we call the \"Multi-file media\" method.
"},{"location":"concepts/rdf/","title":"RDF in an Islandora Repository","text":"Islandora was build on the idea that a repository can be represented in Drupal in a way that can be mapped to RDF and present the repository as Linked Data.
"},{"location":"concepts/rdf/#portland-common-data-model","title":"Portland Common Data Model","text":"Islandora uses the Portland Common Data Model (PCDM) to arrange elements of the repository. Nodes correspond to pcdm:Objects, and Media to pcdm:Files. The \"Member of\" field on nodes allows us to create pcdm:memberOf relationships (the opposite of pcdm:hasMember), and the \"Media of\" field represents pcdm:fileOf relationships (the opposite of pcdm:hasFile).
PCDM, plus additional metadata mappings, is the organizing model which allows us to create an RDF version of the repository in Fedora and/or in an RDF triplestore like Blazegraph.
"},{"location":"concepts/rdf/#mapping-to-rdf-rdf-module","title":"Mapping to RDF (RDF Module)","text":"In Islandora, the RDF module provides a way to map metadata from fields into RDF. This provides a mechanism of using configuration entities (YAML files, as there is no robust UI) to map fields on nodes, media, and taxonomy terms to RDF predicates.
Namepspaces in RDF must be registered before they can be used. The Islandora module registers a number of namespaces and more can be added using hook_rdf_namespaces(). See RDF Mappings for more details.
Nodes, Media, and taxonomy terms can have their RDF (per their mappings) exposed to the world as RDF formatted in the JSON-LD syntax thanks to the Islandora-built JSON-LD Drupal module. The JSON-LD module converts the RDF metadata, with some alterations, to a JSON-LD format that can be consumed by RDF consumers such as Fedora and Blazegraph.
"},{"location":"concepts/rdf/#syncing-to-fedora-and-blazegraph","title":"Syncing to Fedora and Blazegraph","text":"Islandora provides the pathways for objects and media in the repository to be synced to Fedora and Blazegraph.
Objects are sent to Fedora and Blazegraph through an \"Indexing\" Drupal Action, which, after being put on a queue, is read by an indexer which pushes the JSON-LD information to the appropriate target.
Files can be stored in Fedora directly, using the Flysystem module. Whether or not a file is in Fedora, information about that file can be synced (from Drupal Media) into Fedora.
Neither Fedora nor Blazegraph are read as part of the standard Islandora configuration.
"},{"location":"concepts/starter-site/","title":"Islandora Starter Site","text":"The Islandora Starter Site is an out-of-the-box deployment of Islandora. It is a complete exported Drupal site, that makes use of the Islandora modules and configures them in a way that is illustrative and useful.
The Islandora Starter Site contains no code, only references to other modules and lots of Drupal configuration. A very motivated person could re-create the Starter Site just by installing and configuring modules. This means there's nothing tying you to using the Starter Site. There's also nothing tying you to doing things in a particular way. Also, it means that you won't be getting any \"updates\" - there's no code to update.
To experience the full Islandora Starter Site, it requires access to external services such as Solr, Fedora, Alpaca, and Matomo. It is therefore suggested to deploy the Starter Site using one of our two deployment platforms: ISLE-DC (using the make starter or make starter_dev commands), or the Islandora Playbook (using the starter (default) or starter_dev option in the Vagrantfile).
If you are reading this document then you are interested in contributing to Islandora. You do not need to be a programmer to speak up! By joining the Islandora community, you agree to abide by the Islandora Code of Conduct.
"},{"location":"contributing/CONTRIBUTING/#join-the-community","title":"Join the community!","text":"Onboarding Form: Submit your information through the Onboarding Form, and we can invite you to all the listed channels below.
Slack: Use this invite link to join our Slack workspace. If you're not sure what channels you're looking for, get started by asking your questions in the #general channel!
Google Group: Subscribe to the mailing list by joining the Google Group.
Github: Although you can participate without a Github account, having one ensures you can be made a member of the Islandora General Members team to better enable your access to the Islandora project. Additionally, it will allow you to be tagged on relevant issues or pull requests that are of interest to you! To become a part of the Islandora General Members Github team, email community@islandora.ca or use the Onboarding form.
Attend our weekly Zoom meetings on Wednesdays at 1:00 PM EST! You can access these meetings through our Agendas or our Community Calendar. We discuss the past week's Github activity in Issues and Pull Requests, and any other topics that are raised. Anybody is welcome to join the call, and add items to the agenda.
"},{"location":"contributing/CONTRIBUTING/#github-issues","title":"Github Issues","text":"Anyone with a Github account can make issues. Please check the applicable places first to see if there's already a ticket.
Issues should be added in the individual module or component's repository where applicable. For issues that are broader in scope than one module, or about the official Islandora documentation, open an issue in the Documentation repository. This is also the repository for hundreds of \"legacy\" (but still valid) tickets from before we opened issue queues on individual repositories.
Issue templates are provided for the following:
Issues are reviewed during the Weekly Open Tech Call every Wednesday at 1:00 p.m. Eastern.
"},{"location":"contributing/CONTRIBUTING/#github-pull-requests","title":"Github Pull Requests","text":"If you have code to address an issue, or can offer documentation, please create a pull request (PR) in the appropriate repository. Before we can merge your pull request, you must have a signed CLA or CCLA (see below). Code PRs must conform to Drupal Coding Standards, and Documentation PRs must conform to our Documentation Style Guide. There are no guides specific to READMEs or other documentation (help text, etc) that lives in Drupal, but please try to write clearly and follow existing practices. See more in our Contributing Workflow.
"},{"location":"contributing/CONTRIBUTING/#new-modules","title":"New Modules","text":"If you have code that doesn't fit within the scope of Islandora's existing repositories, but that the Islandora community is in a better position to maintain than you are, please consider using the LSAP Process to contribute your code to the ownership of the Islandora Foundation.
"},{"location":"contributing/CONTRIBUTING/#contributor-license-agreements","title":"Contributor License Agreements","text":"Before you set out to contribute code you will need to have completed a Contributor License Agreement or be covered by a Corporate Contributor License Agreement. This license is for your protection as a contributor as well as the protection of the Foundation and its users; it does not change your rights to use your own contributions for any other purpose.
"},{"location":"contributing/committers/","title":"Islandora Committers","text":"Islandora is open source and released under open-source (usually MIT and GPLv2) licenses. The software, associated documentation, and community is developed collectively by a community of contributors and committers. All interested community members are encouraged to contribute to the project. Contributors who demonstrate sustained engagement with the project through quality participation in meetings, mailing lists, documentation and code updates can be self-nominated, or nominated by existing committers, to become a committer. Committers need not be limited to software developers! Contributors with skills in documentation and testing, or who are very active in the community for events and support, can also be committers.
Committers are supported in their specific work by having permissions to do things like merge code as well as manage Github, Slack, and the Website. The current list of committers is managed as a group in Github, and can be viewed in the Github repository by existing members. A separate list of committers is publicly available at the bottom of this page. If a committer does not make a contribution in a year (without known mitigating circumstances) they are usually moved out of the committers GitHub group and listed as an Emeritus committer. There are subgroups with specific enhanced access depending on their skills and levels of commitment to the codebase. These are more fully explained in this spreadsheet of keys and permissions. If you have questions about committers, you can send them to the Chair of the TAG committee, or to community@islandora.ca
"},{"location":"contributing/committers/#rights","title":"Rights","text":"Committers share the following rights:
The following is an alphabetized list of the current Islandora committers:
Name Organization Github username Bryan Brown Florida State University bryjbrown Joe Corall Lehigh University joecorall Jordan Dukart discoverygarden jordandukart Willow Gillingham Born-Digital wgilling Jonathan Hunt Catalyst.Net kayakr Mark Jordan Simon Fraser University mjordan Danny Lamb Born-Digital dannylamb Natkeeran Ledchumykanthan University of Toronto Scarborough natkeeran Rosie Le Faive University of Prince Edward Island rosiel Gavin Morris Born-Digital g7morris Annie Oelschlager Northern Illinois University aOelschlager Alexander O'Neill Born-Digital alxp Don Richards Johns Hopkins University DonRichards Bethany Seeger Johns Hopkins University bseeger Seth Shaw University of Nevada, Las Vegas seth-shaw-unlv Alan Stanley Agile Humanities ajstanley Yamil Suarez Berklee College of Music ysuarez Adam Vessey discoverygarden adam-vessey Jared Whiklo University of Manitoba whikloj"},{"location":"contributing/committers/#emeritus-committers","title":"Emeritus Committers","text":"The following is an alphabetized list of the prior Islandora committers:
Name Organization Daniel Aitken discoverygarden Melissa Anez LYRASIS Aaron Coburn Amherst College Jonathan Green LYRASIS Debbie Flitner Arizona State University Diego Pino METRO Nick Ruest York University Eli Zoller Arizona State University"},{"location":"contributing/committers/#how-are-you-evaluated-as-a-potential-committer","title":"How are you evaluated as a potential committer?","text":"When you are nominated, the other committers review these categories before recommending you as a committer. Note that appointment to committers/TAG is governed by the TAG Terms of reference, and appointment to TAG comes with additional rights and responsibilities. If you become a committer and a member of committers/code, you are eligible to serve in TAG.
"},{"location":"contributing/committers/#you-are-committed-to-the-islandora-community","title":"You are committed to the Islandora Community","text":"You\u2019ve been making strong and consistent community contributions, you can stick it out through tough issues and have demonstrated a willingness to lend a hand.
"},{"location":"contributing/committers/#you-have-skills-and-abilities-suited-to-the-role","title":"You have skills and abilities suited to the role","text":"You demonstrate that you understand the project and community and have skills suited to being a committer. Perhaps you have participated in multiple event planning sessions, you write a lot of documentation, or you chair an effective interest group. If you are a software developer, you\u2019ve written good code. If your patches are easy to apply, this could be you!
"},{"location":"contributing/committers/#you-are-a-great-colleague","title":"You are a great colleague","text":"You respond to and deliver criticism well, and do what you say you will do. You participate actively in decision-making processes, keeping the larger good of the Islandora codebase and community in your purview. You have a helpful attitude and respect everybody\u2019s ideas.
"},{"location":"contributing/committers/#you-mentor-the-less-experienced","title":"You mentor the less experienced","text":"You\u2019ve demonstrated a willingness to reach out to less experienced community members, and link them to resources they need (maybe you made some of these resources - that\u2019s great!)
"},{"location":"contributing/committers/#committers-acting-as-maintainers","title":"Committers acting as Maintainers","text":"If you are maintaining a specific Islandora repository, you commit to the following responsibilities for as long as you act as maintainer:
A great maintainer also communicates important changes within the component to the community.
As maintainer, you are empowered to:
If these responsibilities are not being performed, and/or you choose to step down as maintainer, the TAG will attempt to find another maintainer. If no maintainer can be found, and the TAG/Committers do not elect to jointly maintain the component, it may be deprecated.
"},{"location":"contributing/committers/#process-for-inducting-new-committers","title":"Process for inducting new Committers","text":"Contributors can be self-nominated or be nominated by others. These nominations can go to any committer willing to shepherd the process, or be sent to community@islandora.ca to be forwarded by IF staff. Whoever initiates the process is the sponsor and responsible for either completing the following steps, or delegating them to IF staff.
Upon receiving a nomination:
Checklist for a new committer (can be completed by sponsor or designate):
For additional information about committers, visit this Islandora community wiki page.
"},{"location":"contributing/contributing-workflow/","title":"Contributing Workflow","text":""},{"location":"contributing/contributing-workflow/#terms","title":"Terms","text":"Choosing how we work together is an important privilege the Islandora community offers to its contributors. Our workflow is built around engagement and consensus to encourage high quality code that is expected to meet defined standards. All contributors are expected to follow the defined workflow. Any contributor can propose changes to the workflow at any time by initiating a conversation on the mailing list. Any proposed changes must be confirmed by the committers by way of an Apache rules vote in order to take effect.
"},{"location":"contributing/contributing-workflow/#social-dynamics","title":"Social Dynamics","text":"We operate under the Islandora Community Code of Conduct. Some additional general observations to keep in mind when interacting with others in this workflow:
The Islandora community uses GitHub issues to track bug reports, requests for improvements, requests for new features, and use cases. Issues are welcome from anyone who works with Islandora.
The Islandora issue queue is maintained in the Islandora Documentation repository on GitHub. Issues posted to the queue are reviewed weekly on the Islandora Tech Call. Members of the Islandora community can then respond to posted issues by replying with comments, taking on the assignment to do the work described in the issue, or making pull requests relating to the issue.
Video version available
The material in this tutorial is presented in our video, Create a GitHub Issue.
"},{"location":"contributing/create-issues/#before-you-start","title":"Before you start","text":"Fill in the information for your issue:
Submit documentation formatted in Markdown format.
#)# and ##) to the content sections.Use the \"bold/emphasis\" style in Markdown by enclosing text in double asterisks or underscores, **bold text** or __bold text__, for UI elements that users will interact with. For example, a button label for a button that must be pressed should be made bold in Markdown.
*italic text* or _italic text_, for UI elements that have a label or title if you need to reference them in the documentation. For example, a title of a screen or page that will visit should be made italic in Markdown.>> and **bold text** to indicate clicking through nested menu items, and also include the direct path. Example: **Administration** >> **Structure** >> **Views** (/admin/structure/views)\n- instead of * for bulleted lists. Indent four (4) spaces for nested lists (Github renders nesting in markdown with 2 spaces, but mkdocs needs 4). Example: - I am a list item\n - And I am a sub-item.\nExample:
!!! note \"Helpful Tip\"\n I am a helpful tip!\nResult:
Helpful Tip
I am a helpful tip!
islandora type within the Admonition syntax to call attention to areas where Islandora configuration differs from standard Drupal configuration:Example:
!!! islandora \"Lobster trap\"\n This setting is specific to Islandora and is not standard to Drupal.\nResult:
Lobster trap
This setting is specific to Islandora and is not standard to Drupal.
"},{"location":"contributing/docs-style-guide/#donts","title":"Don'ts","text":"These documentation pages are written and maintained by volunteers from the Islandora community, and new contributions are always welcomed. The documentation that you see here is formatted in the easy-to-use Markdown markup language, which can add formatting such as links, headings, and bolded text. Then the documentation HTML site is built using mkdocs from the Markdown files hosted in the Islandora documentation GitHub repo. This documentation is rebuilt automatically upon changes being committed to GitHub, so you will see changes reflected here within a few minutes of making an edit.
The Islandora Documentation Interest Group holds regular meetings on the second Wednesday of each month between 2:00 p.m. - 3:00 p.m. EST to discuss how to improve and expand Islandora's documentation. If you are interested in becoming involved, all are welcome to join! You can view DIG meetings on the Islandora community calendar and add them to your own calendar by clicking the event and selecting \"copy to my calendar\". If you just want to make a contribution to the existing documentation, that process is outlined here.
"},{"location":"contributing/editing-docs/#before-you-start","title":"Before you start","text":"To edit documentation in Islandora, you must:
\u2192 If you are not a member of the Islandora GitHub organization, GitHub will create a new fork of the documentation repository in your GitHub account. From there you can make a pull request to have your changes sent back to the main Islandora documentation repo to be reviewed and accepted.
\u2192 If you are a member of the Islandora GitHub organization, you will be able to create a new temporary branch within the main Islandora documentation repo, and then create a pull request by selecting Create a new branch for this commit and start a pull request. Give your branch a brief name starting with your username, with no spaces, indicating what it is for. Then click Commit changes.
@ symbol. If you don't have anyone specific in mind, you may tag @Islandora/committers to alert all Islandora 8 Committers that there is a new pull request for their review.When you have completed the template, submit your changes for review by clicking Create pull request.
Once your pull request has been made, it will go into the pull request queue and anyone tagged in the request will receive a notification. Any member of the Islandora GitHub organization can review and merge your pull request, committing your changes to the documentation, with a few exceptions:
Once your changes are committed, these documentation pages will be rebuilt to reflect your changes within a few minutes.
For very minor changes
For members of the Islandora GitHub organization (i.e., those with write access to the documentation repo): If the change you made was very small, such as a formatting or spelling correction that does not change the semantics of the document, you may push your changes into the documentation immediately by selecting commit directly to the main branch and clicking Commit changes.
Warning
These steps are for more advanced users that are familiar with using the Git source control software, which is different from just using the GitHub website with a web browser.
#  Repository Name\n\n[](link)\n[](link)\n[](./CONTRIBUTING.md)\n[](./LICENSE)\n[](link)\n\n## Introduction\n\nA brief introduction and summary of the module.\n\n## Requirements\n\nThis module requires the following modules/libraries:\n\n* [Name](Link)\n* [Name](Link)\n* Any\n* Requirements\n\n## Installation\n\nInstallations instructions.\n\n## Configuration\n\nDescribe path to configuration.\n\nInclude a screenshot of configuration page. When using your choice of screenshot software, resize your browser\nfirst to avoid wide screenshots. Here are a few browser extension examples to take screenshots.\n\n - [Fireshots](https://chrome.google.com/webstore/detail/take-webpage-screenshots/mcbpblocgmgfnpjjppndjkmgjaogfceg)\n - [Nimbus](https://chrome.google.com/webstore/detail/nimbus-screenshot-screen/bpconcjcammlapcogcnnelfmaeghhagj)\n\nTo upload the image drag the image into the comment section of an existing Pull Request.\n\nThis will generate the image URL link for you\n \n\nVideo example on [How to attach an Image in README.md file with Github](https://youtu.be/wVHJtL-y7P0)\n\n## Other Sections As Needed\n\nSections specific to this repo, but not found in all repos, should go here.\n\n## Documentation\n\nFurther documentation for this module is available on the [Islandora documentation site](https://islandora.github.io/documentation/).\n\n## Troubleshooting/Issues\n\nHaving problems or solved a problem? Check out the Islandora google groups for a solution.\n\n- [Islandora Group](https://groups.google.com/forum/?hl=en&fromgroups#!forum/islandora)\n- [Islandora Dev Group](https://groups.google.com/forum/?hl=en&fromgroups#!forum/islandora-dev)\n\n## FAQ\n\nQ. Is this normal?\n\nA. Yes. This is normal. Why ...\n\n## Maintainers/Sponsors\n\nCurrent maintainers:\n\n- [Maintainer Name](https://github.com/maintainer_github)\n- [Another Maintainer](https://github.com/maintainer_github)\n\nThis project has been sponsored by:\n\n- Some really awesome sponsor\n\n## Development\n\nIf you would like to contribute, please get involved by attending our weekly [Tech Call](https://github.com/Islandora/documentation/wiki). We love to hear from you!\n\nIf you would like to contribute code to the project, you need to be covered by an Islandora Foundation [Contributor License Agreement](https://drive.google.com/file/d/1k6eCM5EV-w4I4ErkiGj4NJwvLnXkejyk/view?usp=sharing) or [Corporate Contributor License Agreement](https://drive.google.com/file/d/1-SQYuHWRxvltQYgkFWpYv7nGbvJp1u8h/view?usp=sharing). Please see the [Contributors](https://github.com/Islandora/islandora-community/wiki/Onboarding-Checklist) pages on Islandora.ca for more information.\n\nWe recommend using the [islandora-playbook](https://github.com/Islandora-Devops/islandora-playbook) to get started. If you want to pull down the submodules for development, don't forget to run git submodule update --init --recursive after cloning.\n\nAlso include any Travis gotcha's here.\n\n## License\n\n[Name](link). GPLv2 for Drupal modules. MIT for other modules.\nIslandora is an ecosystem of repositories and follows a Semantic Versioning Approach. This allows the community to remain aligned with Drupal's approach and support more a more modular approach and more frequent releases, as well as better upgrade paths for those using components of the system. In semantic versioning, a version has three elements 'MAJOR.MINOR.PATCH'. This looks something like 2.1.1, or you may see major versions labelled as 2.x.x. To guide repository maintainers, we recommend you increment the:
Learn more about Islandora's Versioning Policy
"},{"location":"contributing/releasing-islandora/#how-to-release-java-code","title":"How to Release Java Code","text":"You will need Java 8 on your system to release java code. The rest of the work is handled handled by Gradle, which is included in the Git repositories. If you cannot get Java 8, you can still release Syn using Docker and the openjdk:8-jdk image. For Alpaca, because of our use of keys, Java 8 is required.
To release Syn
-SNAPSHOT from projectVersion in build.gradle$ ./gradlew build shadowJar$ docker run --rm -v /path/to/Syn:/opt/Syn openjdk:8-jdk bash -lc 'cd /opt/Syn && ./gradlew build shadowJar'v to the release tag (i.e. use vX.X.X instead of just X.X.X)/path/to/Syn/build/libs. You want both islandora-syn-X.X.X.jar and islandora-syn-X.X.X-all.jar.projectVersion in build.gradle and add -SNAPSHOT to the end again.To make sure the release goes smoothly, you should ensure that:
~/.gradle/gradle.properties) exists and includes the following: ossrhUsername = jiraname\nossrhPassword = jirapass\nsigning.keyId = pubkeyid\nsigning.password = keypassphrase\nsigning.secretKeyRingFile = /your/home/.gnupg/secring.gpg\ngit is configured (locally or globally) to cache github credentials for https or use sshNote: As of GPG 2.1 secring.gpg has been removed so you need to export secret keys to create the secret key ring.
gpg --export-secret-keys -o secring.gpg\nThe following assumes you are using ssh (e.g. git@github.com for authentication).
It will also work for https if you properly cache your github credentials. The credentials must be cached and valid because Gradle will not prompt you for them.
"},{"location":"contributing/releasing-islandora/#release-artifacts-to-sonatype-and-github","title":"Release artifacts to Sonatype and Github","text":"You need to merge the latest code into the master branch and use Gradle to release. You can do this by running
git clone git@github.com:Islandora-CLAW/Alpaca.gitcd Alpacagit checkout mastergit pull origin dev./gradlew releaseThe script will start but then will ask you to confirm the version to release as.
??> This release version: [0.3.1] (WAITING FOR INPUT BELOW)\n<-------------> 0% EXECUTING [13s]\n> :release\n> :alpaca-release:confirmReleaseVersion\n0.4.0) or nothing to use the suggested version (0.3.1 in this example) and hit ENTER. The gradle release task will take care of dropping -SNAPSHOT from the version, uploading artifacts to Maven central for staging, tagging and pushing a release to github, and bumping master of the Alpaca repository up by a point release for the next development iteration.
The release process will also generate documentation at ./docs/<new-version>. This will need to be added to the git repository and pushed to the master branch:
git add docs/<new-version>git commit -m \"Add documentation for <new-version>\"git push origin\u26a0\ufe0f The following steps, once completed are final. They cannot be undone, revoked or altered. Only proceed if you've completed all the above steps and are absolutely certain the release is ready for publication.
Release the jsonld module by creating a new release for it in Github.
Release the openseadragon module by creating a new release for it in Github.
Release the islandora_mirador module by creating a new release for it in Github.
Release the migrate_islandora_csv module by creating a new release for it in Github.
Release chullo by creating a new release for it in Github.
Crayfish commons depends on the chullo library, and must have its dependencies updated before release.
islandora/chullo from dev-dev to the release you just made in the previous step.composer update -Wcomposer.json and composer.lock files to Github.crayfish-commons by creating a new release in Githubislandora/chullo back to dev-devcomposer update -W againcomposer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Crayfish depends on the crayfish-commons library, and must have its dependencies updated before release.
islandora/crayfish-commons in each composer.json for each microservice except for Houdini. Houdini needs the dev-symfony-flex branch of Crayfish Commons, so just leave that dependency alone for now.composer update -W on each microservice. I did this with a little bash-fu: for D in */; do (cd $D; composer update -W) donecomposer.json and composer.lock files to Github.islandora/crayfish-commons back to dev-dev except for Houdini.composer update -W on each microservice again. for D in */; do (cd $D; composer update -W) done makes this easy.composer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Release controlled_access_terms by slicing a new release for it in Github.
"},{"location":"contributing/releasing-islandora/#release-islandora","title":"Release Islandora","text":"The islandora module depends on the crayfish-commons library, and must have its dependencies updated before release.
islandora/crayfish-commons in composer.json.composer update -Wcomposer.json and composer.lock files to Github.islandora/crayfish-commons back to dev-devcomposer update -W again.composer.json and composer.lock files to Github with a commit message of \"Preparing for next development iteration\".Note to module maintainers specifying composer requirements: please familiarize yourself with the Next Significant Release Modifiers so that module dependencies are as permissive as possible. Most often dependency versions should be tagged using the caret (^) so that bug-fixes and minor releases can be pulled in without updating their module\u2019s composer file.
"},{"location":"contributing/releasing-islandora/#undoing-a-release","title":"Undoing a Release","text":"Dependencies mean that if you are going to release all of the Islandora ecosystem, order is very important. At any point, releases can be deleted, updated, and redone in Github. You can reach out to the community if you have questions. Note that if you want to 'redo' a release, you can follow these steps:
git tag -d TAG_NAME; git push --delete origin TAG_NAMEYou cannot follow these steps when publishing to Sonatype with Alpaca, but this should rarely be an issue. Instead, increment the version number and tag a new release.
Learn more about Islandora's Versioning Policy
"},{"location":"contributing/sandbox/","title":"Updating the Sandbox","text":"Note
This page is about the online sandbox, https://sandbox.islandora.ca. If you are looking to set up your own site, see the Installation section.
The sandbox at https://sandbox.islandora.ca is built, configured, and deployed, by the https://github.com/Islandora-Devops/sandbox repository. This build process happens nightly from the Sandbox's latest release.
"},{"location":"contributing/sandbox/#maintaining-the-front-end-site","title":"Maintaining the front-end site","text":"The Sandbox makes use of the following components, which are version-locked in the drupal/Dockerfile file:
In order to update them, you need to update the desired XXX_COMMIT hash; compute and add a sha-256 checksum; and then cut a release of the Sandbox repository. It will be deployed to the production site overnight. See full instructions at the Sandbox README.
Note that some of these components will need to be updated together, for example a new column in the Demo Objects may require a new field in the Starter Site.
"},{"location":"contributing/sandbox/#maintaining-the-back-end-containers","title":"Maintaining the back-end containers","text":"The containers used in the Sandbox are specified by the ISLANDORA_TAG value in the .env file.
A functioning Islandora Stack is made up of dozens of components working in synchronization with each other to store information in your repository, manage that information, and disseminate it intelligently to users. Whether running an installation using the provided Ansible playbook or installing the stack manually, it may be helpful to have a brief overview of all the components we're going to need, in the order we're going to install them, as well as a brief introduction to each component's installation and configuration process.
This list includes four different kinds of components:
Combined together, Apache, PHP, and MySQL/PostgreSQL comprise a LAMP or LAPP server used to provide end-user-facing components - namely, the website.
Apache is the webserver that will serve up webpages to the public. It will also manage some internal functionality provided by Crayfish, and will expose Cantaloupe to the public. We\u2019ll be making changes to the VirtualHost entry, enabling some modules, and modifying the ports configuration. The VirtualHost entry will eventually be modified when we need to expose other services like Cantaloupe to the public.
PHP is the runtime interpreter for all the code Drupal and Crayfish need to be processed. By default, installing PHP 7.2 will give us a command-line interpreter, as well as an interpreter for Apache. We\u2019re going to install several PHP modules required and/or useful for the components that make use of PHP.
MySQL and PostgreSQL are database management systems that we will use to store information for many different components like Drupal and Fedora. By default, the Ansible playbook installs MySQL, though this can be switched to PostgreSQL. The manual installation guide recommends and walks through installing and using PostgreSQL.
"},{"location":"installation/component-overview/#the-front-facing-cdm-composer-drush-and-drupal","title":"The Front-Facing CDM - Composer, Drush, and Drupal","text":"Composer will be used to install both Drupal and Drush simultaneously using Islandora's fork of the drupal-project repository.
Composer is an installer and dependency manager for PHP projects. We're going to need it to install components for any PHP code we need to make use of, including Drupal and Crayfish.
Drush and Drupal are installed simultaneously using drupal-project. Drupal will serve up webpages and manage Islandora content, and Drush will help us get some things done from the command-line.
"},{"location":"installation/component-overview/#the-web-application-server-tomcat-and-cantaloupe","title":"The Web Application Server - Tomcat and Cantaloupe","text":"Several applets will be deployed via their .war files into Tomcat, including Fedora and Cantaloupe.
Tomcat serves up webpages and other kinds of content much like Apache, but is specifically designed to deploy Java applications as opposed to running PHP code.
Cantaloupe is an image tileserver that Islandora will connect to and use to serve up extremely large images in a way that doesn't have an adverse effect on the overall system.
"},{"location":"installation/component-overview/#the-back-end-file-management-repository-fedora-syn-and-blazegraph","title":"The Back-End File Management Repository - Fedora, Syn, and Blazegraph","text":"Fedora will be installed in its own section, rather than as part of the Tomcat installation, as the installation process is rather involved and requires some authorization pieces to be set up in order to connect them back to Drupal and other components.
Fedora is the default backend repository that Islandora content will be synchronized with and stored in. A great deal of configuration will be required to get it up and running, including ensuring a database is created and accessible.
Syn is the authorization piece that allows Fedora to connect to other components.
Blazegraph will store representative graph data about the repository that can be queried using SPARQL. Some configuration will also be required to link it back to Fedora, as well as to ensure it is being properly indexed.
"},{"location":"installation/component-overview/#the-search-engine-solr-and-search_api_solr","title":"The Search Engine - Solr and search_api_solr","text":"The installation of Solr itself is rather straightforward, but a configuration will have to be generated and applied from the Drupal side.
Solr will be installed as a standalone application. Nothing of particular importance needs to happen here; the configuration will be applied when search_api_solr is installed.
search_api_solr is the Drupal module that implements the Solr API for Drupal-side searches. After installing and configuring the module, the drush solr-gsc command will be used to generate Solr configs, and these configs will be moved to the Solr configuration location.
Crayfish is a series of microservices that perform different asynchronous tasks kicked off by Islandora. It contains a series of submodules that will be installed via Composer. Later, these configured components will be connected to Alpaca.
"},{"location":"installation/component-overview/#the-broker-connecting-everything-karaf-and-alpaca","title":"The Broker Connecting Everything - Karaf and Alpaca","text":"Karaf\u2019s job is similar to Tomcat, except where Tomcat is a web-accessible endpoint for Java applets, Karaf is simply meant to be a container for system-level applets to communicate via its OSGI. Alpaca is one such applet; it will broker messages between Fedora and Drupal, and between Drupal and various derivative generation applications.
Alpaca contains Karaf services to manage moving information between Islandora, Fedora, and Blazegraph as well as kicking off derivative services in Crayfish. These will be configured to broker between Drupal and Fedora using an ActiveMQ queue.
"},{"location":"installation/component-overview/#finalized-drupal-configurations","title":"Finalized Drupal Configurations","text":"Drupal configuration exists as a series of .yaml files that can either be created in a feature, or exported from Drupal using the content_sync module. It can also be manually entered in via the UI. We're going to place configuration in a few different ways; Some content will be synchronized onto the site, and some core configurations from the main Islandora module will need to be run in order to facilitate ingest.
There's a lot of useful commands available to you from within the isle-dc folder.
make -B docker-compose.yml will rebuild your docker-compose.yml file based on configuration in your .env file. make demo and make local will automatically do this for you, but if you change configuration at a later point in time, you'll need to run this command to see your changes reflected.
make config-export will export your site's configuration to your site's config sync directory (usually config/sync inside your Drupal root folder).
make config-import will import site's configuration from your site's config sync directory (usually config/sync inside your Drupal root folder).
make drupal-database-dump DEST=/your/path/dump.sql will dump your Drupal database and place the file at DEST.
make drupal-database-import SRC=/your/path/dump.sql will import your Drupal database from the file at SRC.
make drupal-public-files-dump DEST=/your/path/public_files.tgz will export your public filesystem and place it as a single zipped tarball at DEST.
make drupal-public-files-import SRC=/your/path/public_files.tgz will import your public filesystem from a single zipped tarball at SRC.
make fcrepo-export DEST=/your/path/fcrepo-export.tgz will export your Fedora repository and place it as a single zipped tarball at DEST
make fcrepo-import SRC=/your/path/fcrepo-export.tgz will import your Fedora repository from a single zipped tarball at SRC
make reindex-fcrepo-metadata will reindex RDF metadata from Drupal into Fedora. Requires the Views Bulk Operations Module.
make reindex-solr will rebuild rebuild Solr search index for your repository.
make reindex-triplestore will reindex RDF metadata from Drupal into Blazegraph. Requires the Views Bulk Operations Module.
ISLE Docker Compose has a single configuration file, .env. Within it, here is all the values you can set and their effects. This list is subjectively sorted in order of importance and usefulness.
Setting the ENVIRONMENT changes how your docker-compose.yml file gets constructed. When you switch from a demo to development to production environment, this is the variable to change.
Available values for this setting are
demo - For demo environments where you do not need access to the codebaselocal - For development environments where you need edit the codebasecustom - For production environments where your codebase gets baked into a custom containerBy default, this is set to demo.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
What domain your Islandora site will be available at. If you are deploying anywhere other than your personal machine, you'll probably want to change this.
By default, this is set to islandora.traefik.me.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are indexing RDF metadata in Fedora and/or a triplestore, please be aware of the implications of changing this once you have installed. By changing your domain, all of the subjects of your triples will be incorrect and you will have to reindex. There are make commands to streamline this process, but if you have lots of data it can potentially be costly in terms of time and effort.
Repository to use for pulling isle-buildkit images.
By default, this is set to islandora. Change to local to use images you have built locally with isle-buildkit, or use your custom docker registry if you have set up one.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
The version of the isle-buildkit images to use. Non isle-buildkit images have their versions specified explicitly in their respective docker-compose files.
By default, this is set to the latest stable release of isle-buildkit.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not you want to use secrets. For demo and development environments, secrets are not required. They are essential if you are running a production environment, though.
Secrets are contained in the secrets folder within your isle-dc installation. Each file represents an individual secret, and its contents are the value you're trying to protect.
Available values for this setting are:
truefalseIf you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Which install profile to use when making an initial installation. Valid values for this setting are the machine names of any Drupal profile.
By default, this is set to standard.
Set this to true if you want to install an existing Drupal site whose configuration was exported with drush config:export
Available values for this setting are - true - false
By default, this is set to false.
If you set this to true, be sure to set DRUPAL_INSTALL_PROFILE to minimal.
Whether or not to include Watchtower as a service. When developing isle-buildkit, this is extremely useful and will auto-deploy new containers as you make changes. You should not use watchtower on production environments, though.
Available values for this setting are - true - false
By default, this is set to true.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not to include etcd as a service.
Available values for this setting are - true - false
By default, this is set to false. If you don't know what etcd is, then leave this be.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Whether or not to include the coder IDE as a service. If you're developing on Islandora, this can be pretty useful when developing. You should not deploy this service on production environments.
Available values for this setting are - true - false
By default, this is set to false.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Which database engine to use for Drupal.
Available values are - mariadb - postgres
By default, this value is set to mariadb.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are changing from mariadb to postgres or vice versa, you MUST migrate your data yourself. ISLE will not convert your database from one to the other, and it's generally not advised to change this once you've installed.
"},{"location":"installation/docker-available-configuration/#fcrepo_database_service","title":"FCREPO_DATABASE_SERVICE","text":"Which database engine to use for Fedora.
Available values are - mariadb - postgres
By default, this value is set to mariadb.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Changing this after installation has consequences
If you are changing from mariadb to postgres or vice versa, you MUST migrate your data yourself. ISLE will not convert your database from one to the other, and it's generally not advised to change this once you've installed.
"},{"location":"installation/docker-available-configuration/#compose_http_timeout","title":"COMPOSE_HTTP_TIMEOUT","text":"Sometimes when bringing up containers, you can encounter timeouts with an error like this:
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.\nIf you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: XXX).\nBy default, this value is set to 480, but if you have slow network conditions and encounter this error, consider bumping it higher.
"},{"location":"installation/docker-available-configuration/#compose_project_name","title":"COMPOSE_PROJECT_NAME","text":"Used for naming services in traefik as well as defining network alias and urls. For example the drupal service will be found at ${COMPOSE_PROJECT_NAME}_drupal_1. This is useful to change if you are running multiple ISLE instances on one machine.
By default, this value is set to isle-dc.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Allows building your custom image when setting ENVIRONMENT=custom
By default, this is set to 1.
You most likely will never have a need to change this.
"},{"location":"installation/docker-available-configuration/#docker_buildkit","title":"DOCKER_BUILDKIT","text":"Instructs Docker to use buildkit when building your custom image.
By default, this is set to 1.
You most likely will never have a need to change this. Trust us, you want buildkit.
"},{"location":"installation/docker-available-configuration/#project_drupal_dockerfile","title":"PROJECT_DRUPAL_DOCKERFILE","text":"The name of the Dockerfile to use after setting ENVIRONMENT=custom.
By default, this is set to Dockerfile.
You most likely will never have a need to change this.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
Set this to true to disable JWT authentication for Fedora. This is necessary when performing a Fedora import using their import/export tools.
By default, this is set to false.
Whether or not to use Fedora 6. If you set this to false, you will be given Fedora 5. In general, unless you already have an existing site on Fedora 5, you'll want Fedora 6.
By default, this is set to true.
If you change this setting, you need to make -B docker-compose.yml to generate a new docker-compose.yml file that contains the changes you've made.
After you've finished installing Islandora using ISLE, here's some useful information to keep close at hand about running your site.
"},{"location":"installation/docker-basic-usage/#important-files","title":"Important Files","text":"The make commands that you used to install Islandora will leave you with two very important files.
.env A configuration file that is yours to customize. This file controls how the docker-compose.yml file gets generated to meet your use case.It also allows you to set variables that make their way into the final docker-compose.yml file, such as your site's domain. docker-compose.yml A ready to run docker-compose.yml file based on your .env file. This file is considered disposable. When you change your .env file, you will generate a new one."},{"location":"installation/docker-basic-usage/#available-services","title":"Available Services","text":"Here's a list of all the available services. Note that there are some services over http and not https. Those aren't meant to be exposed to the public, but internally people from your organization will want to access them. In practice, you can restrict access to these services using firewall rules to just those who you trust.
If you want to stop Islandora, you can bring down all the containers with
docker-compose down\nIf you want to start Islandora back up after stopping it, use
docker-compose up -d\nIf you want to stop Islandora and delete all of its content, use
docker-compose down -v\nIf you make changes to configuration in the .env file, you may need to regenerate your docker-compose.yml file so that those changes take effect.
make -B docker-compose.yml\nOnce you have a new docker-compose.yml file, you'll need to restart your containers that have had configuration change. You can do this easily with
docker-compose up -d\nEven if the site is up and running, that command will only restart the containers it needs to.
"},{"location":"installation/docker-basic-usage/#listing-services","title":"Listing services","text":"You can see a list of all the containers that you have running and their statuses by running
docker ps -a\nYou can tail logs using
docker-compose logs service_name
For example, to tail nginx logs for Drupal, use docker-compose logs drupal.
If you don't know what you're looking for exactly, you can turn on the fire hose and look through all logs by dropping the service name and simply using
docker-compose logs
If you are deploying Islandora on a server that is publicly accessible, there are extra precautions you should take for performance and security reasons.
For local development we bind mount our codebase (Drupal) folder so that we can do development work locally, outside of our Docker container. For a production site we don\u2019t want to do this. Instead, we make changes to our development site, then build a Drupal image with our changes that we can use on our production site instead of the default Islandora Drupal image.
"},{"location":"installation/docker-custom/#creating-your-image","title":"Creating your Image","text":"In order to generate that custom Drupal image we need to set up a development environment. You will do this on your development computer, rather than your production server.
Once your development site is set up you will need somewhere to store your custom Drupal image. You should create a private repository in your container registry of choice (Dockerhub, GitHub, GitLab, etc.)
Once you have a place to store it, you can create your custom Drupal image by editing your .env to set CUSTOM_IMAGE_NAMESPACE to your dockerhub username or the URL and username for your container repository, and CUSTOM_IMAGE_NAME to the name of the repository you just created.
Once this is done you can run make build to create the image from a Dockerfile. If you don't have a custom Dockerfile, it will create one from sample.Dockerfile. This will create a new image from the Islandora Drupal image, with your codebase folder copied into the image during the build process.
You should now have a custom Drupal image on your local machine. You will need to docker login to your container registry to push your image, so make sure to do that if you haven't yet (you should only need to do this once). You can then push this image to your container registry by running make push-image.
Now that you have a development site set up with your own codebase folder, you should create a new git repository for your site. This way you can easily spin up a new site based on your modules and configuration, instead of the Islandora Starter Site. This will also allow you to sync changes between your production, staging, and development sites.
You will likely want to include the Isle-dc directory as well as your codebase folder in your git repository. This will allow you to make modifications to your Makefile, Dockerfile, docker-compose.yml, etc. and keep those modifications alongside your Drupal files in the codebase folder. To store these files in your own private repository you should change the git remote repository URL from the Isle-DC URL to the URL for your private repository.
Note that Isle-DC has the codebase folder in its .gitignore, so you will want to remove codebase* from your .gitignore. You don't need the web or vendor directories to be stored in your repo, since they will be added when you do a composer install during the site setup, so you can add those into your .gitignore as codebase/vendor/* and codebase/web/*.
At this point you should have a custom git repository and a custom Drupal image, and both should be accessible by your production server. On your production server you will need to clone your custom copy of Isle-dc, copy the sample.env file, and name it .env.
In that .env you should set the following variables:
ENVIRONMENT=custom\nCOMPOSE_PROJECT_NAME=(same as dev site)\nCUSTOM_IMAGE_NAMESPACE=(same as dev site)\nCUSTOM_IMAGE_NAME=(same as dev site)\nDOMAIN=your-domain.com\nOnce this is set up, run make production to install your drupal site. You can generate new secrets for your production passwords, or copy them from your dev server if you would like them to be the same.
At this point, your site is ready, but you won't be able to access it at your URL until you update the SSL certificates (see TLS section below).
Codebase Folder
Because your codebase folder is in your git repository it will be cloned to your development server, but unlike in your development environment, it is not bound to the Drupal container. This means that any change you make to those files will not be represented in your Drupal site.
You can also use this folder to build your Drupal image on the production server instead of on your development server if you like.
"},{"location":"installation/docker-custom/#secrets","title":"Secrets","text":"Sensitive information, such as passwords, should never be built into a container. It also shouldn't ever be bind-mounted in like we do with our codebase folder. If you use secrets, it's like bind-mounting in a file, except that file is provided from the host machine to the container using an encrypted channel.
Secrets are on by default. They can be toggled in your .env file, but you should never turn them off for production sites.
USE_SECRETS=true\nThe secrets themselves are stored in the secrets/live folder of isle-dc. If you navigate to that directory, you'll see several small files, where each file represents a different password. When you run make production you can choose whether to generate new random secrets or create them yourself.
All public facing sites need to use HTTPS, and it's definitely a stumbling block for the uninitiated. Fortunately, isle-dc is set up to use HTTPS by default. Even when running make demo, your site runs over HTTPS at https://islandora.traefik.me.
The default certificates are stored in the certs folder of isle-dc, and you can simply overwrite them with certificates from your certificate authority. As long as the certificates match the DOMAIN variable in your .env file, that is. Changing the files in the certs folder requires restarting the Traefik container, which you can do by running docker-compose restart traefik.
To use Let's Encrypt to acquire your SSL Certificate, set the following in your .env file and run make -B docker-compose.yml && make up.
USE_ACME=true\nACME_EMAIL=your-email@example.org\nBe sure to replace your-email@example.org with the email address you've associated with Let's Encrypt.
The way this is setup, is it performs an HTTP Challenge to verify you are in control of the domain. So your system will need to be accessible at http://DOMAIN/.
??? warning \"Let's Encrypt Rate Limit\" If you aren't careful, you can hit Let's Encrypt's rate limit, and you'll be locked out for up to a week! If you want to use their staging server instead while testing things out, add the following to your .env file
```\nACME_SERVER=https://acme-staging-v02.api.letsencrypt.org/directory\n```\nYou'll still get security exceptions when it's working, but you should be able to check the certificate from the browser and confirm you are getting it from the staging server.
\nIf you are still getting security exceptions, check what certificate is being used through your browser. Setting TRAEFIK_LOG_LEVEL=DEBUG in your .env file will help out greatly when debugging Traefik. You can tail the logs with docker-compose logs -tf traefik.
The *.traefik.me certificate that covers islandora.traefik.me will need to be redownloaded occasionally, due to the certificate expiring or possibly being revoked. You can download the updated certificates by performing the following commands:
rm certs/cert.pem\nrm certs/privkey.pem\nmake download-default-certs\ndocker-compose restart traefik\ntraefik.me Certificate Note
Please note that sometimes the upstream provider of the traefik.me certificate takes a couple of days to update the certificate after it expires or is accidentally revoked.
"},{"location":"installation/docker-custom/#requesting-certificates-through-acme-and-external-account-binding","title":"Requesting Certificates through ACME and External Account Binding","text":"To request certificates through another Certificate Authority (CA) that supports External Accounting Binding through ACME such as InCommon or ZeroSSL you will need to add the following to your .env file:
USE_ACME=true\nACME_EMAIL=your-email@example.org\nACME_SERVER=\nACME_EAB_KID=\nACME_EAB_HMAC=\nWhere ACME_SERVER is the CA server to use, ACME_EAB_KID is the key identifier from the External CA, and ACME_EAB_HMAC is the HMAC key from the External CA.
Once you have added these commands you will need to run the following commands:
make -B docker-compose.yml\nmake up\nISLE, short for ISLandora Enterprise, is a community initiative to ease the installation and maintenance of Islandora by using Docker. It was originally started by the Islandora Collaboration Group for Islandora Legacy. When transitioning to Islandora, the project came under the purview of the Islandora Foundation. All documentation on this site assumes you're trying to install Islandora. See here if you are looking for ISLE for Islandora Legacy.
"},{"location":"installation/docker-introduction/#why-use-isle","title":"Why use ISLE?","text":"ISLE's architecture using Docker separates out the \"state\" of your site (i.e. all the content, files, and configurations that you've entered) from the underlying software that runs it (e.g. webserver, database, microservices, etc). This allows for easier upgrades, faster development, and more flexible deployment. It is hands down the easiest way to install, run, and maintain and Islandora instance.
"},{"location":"installation/docker-introduction/#where-is-isle","title":"Where is ISLE?","text":"ISLE is a suite of Docker containers that run the various components of Islandora: drupal, fedora, solr, alpaca, crayfish, matomo, etc. The individual containers are created (and automatically pushed to Docker Hub) by ISLE BuildKit.
In order to deploy the containers, however, you need to use a container orchestration tool. The ISLE project provides tools for running and maintaining the containers using docker-compose with ISLE Docker Compose.
"},{"location":"installation/docker-local/","title":"Installing a Development or Demo Server","text":"When developing locally, your Drupal site resides in your isle-dc/codebase folder and is bind-mounted into your Drupal container. This lets you update code using the IDE of your choice on your host machine, and the changes are automatically reflected on the Drupal container.
If you don't already have a Drupal site, you'll be given a basic setup using Drupal 9 and the Islandora Starter Site.
If you do already have a Drupal site, use git to clone it into place as the codebase folder.
cd /path/to/isle-dc\ngit clone https://github.com/your_org/your_repo codebase\nNow you'll need to tell isle-dc to look for it by setting the ENVIRONMENT variable in your .env file. If you don't have one, copy over sample.env and name it .env. Then set
ENVIRONMENT=starter\nYou should also change the COMPOSE_PROJECT_NAME variable. This determines the name of the Docker containers and volumes that are created when you run make starter. If you leave this as the default you will need to be careful not to overwrite the containers with another install of isle-dc later.
COMPOSE_PROJECT_NAME=isle-dc\nIf your site includes exported configuration from drush config:export, then you'll also need to set
INSTALL_EXISTING_CONFIG=true\nDRUPAL_INSTALL_PROFILE=minimal\nOnce you are ready, run
make starter\nto install the Drupal site in your codebase folder and spin up all the other containers with it.
Enjoy your Islandora instance! Check out the basic usage documentation to see all the endpoints that are available and how to do things like start and stop Islandora. Your passwords, including the Drupal admin password, can be found in the secrets/live directory after you run make starter.
To populate your site with some demo content, you can run make demo_content. This will import some sample objects into your Islandora site.
Drupal has a dedicated security team, and regularly produces updates to address security issues that are discovered. You should always keep your Drupal site up to date so that you are protected against known vulnerabilities. Updating Drupal can be intimidating, but we have step-by-step instructions to help you do it safely.
"},{"location":"installation/docker-maintain-drupal/#note-on-production-sites","title":"Note on Production Sites","text":"Please note that maintaining Drupal should be done on your development site. For production sites, you should build an image from the codebase folder on your development machine, and run that image in production. For more information on this, please see the production install instructions.
Once you have a codebase folder, how do you maintain it and keep it up to date? The quick answer is \"the same way you maintain any Drupal site with Composer and Drush\"... with one small caveat. You most likely do not have PHP or Composer on your machine, and even if you do, you want to make sure you're using the exact same version that your Docker container is using. So to ensure all the versions of things line up, we use Docker to execute Drush and Composer from the Drupal container. The general template for running a command in your Drupal container looks like this:
docker compose exec -T drupal with-contenv bash -lc 'YOUR COMMAND'\nYou can also just shell into the Drupal container and run commands as well, just be aware that if you shut down your container for any reason, you'll lose your bash history. If you want to shell in to run commands, drop the -T and -lc 'YOUR COMMAND' bits.
docker compose exec drupal with-contenv bash\nUse Composer to update your site's modules and their dependencies. The working directory of the Drupal container is the Drupal root (a.k.a. codebase), so you don't need to cd into any other directory before running the command. The following command will update all modules and their dependencies that are not pinned to specific versions.
docker compose exec -T drupal with-contenv bash -lc \"su nginx -s /bin/bash -c 'composer update -W'\"\nWhen running Composer commands you may come across the following error
[ErrorException]\nfile_put_contents(/var/www/drupal/web/sites/default/settings.php): failed to open stream: Operation not permitted\nAfter getting the newest code, you'll want to use Drush to update the Drupal database and run any other update hooks that have been introduced. However, YOU SHOULD BACK UP YOUR DATABASE BEFORE GOING ANY FURTHER. You never know when something will go wrong and you don't want to be stuck with an unusable database and no plan B.
make drupal-database-dump DEST=/path/to/dump.sql\nNow you can safely update the Drupal database with Drush via
docker compose exec -T drupal with-contenv bash -lc 'drush updb'\nIf for any reason, something goes wrong, you can Restore the Drupal database at any time by running
make drupal-database-import SRC=/path/to/dump.sql\nYou will regularly be updating your Drupal site as security patches and module updates are released. Less often, you will also need to update the rest of your Islandora installation. ISLE makes this easy. In fact, it was specifically designed to streamline this process.
Since Islandora is not a single piece of software, but instead many pieces of software working together in concert, maintaining all of it is a daunting task. There's nginx, tomcat, karaf, etc... Then there's everything needed for the authentication layer and JWT keys. Plus there's all the microservices. You can see that all this adds up to a significant maintenance burden.
Now imagine if all that time and effort spent on security updates and getting the newest versions could be boiled down to a handful of simple commands. That's exactly what ISLE does!
"},{"location":"installation/docker-maintain-isle/#updating-isle","title":"Updating ISLE","text":"Updating
Before we update ISLE we should pull the most recent version of the isle-dc repository. This ensures that our docker-compose.yml file has any new additions that the new version of ISLE will need.
You may also want to check your .env file against the sample.env found in the repository to see if there are new variables you wish to make use of.
Updating ISLE is easy. When a new release is made available, you update the TAG variable in your .env file to the latest version. Suppose you are on ISLE 1.0.0, and ISLE 1.1.0 has been released. Then we would set
TAG=1.1.0\nBefore we update our docker-compose.yml file we want to stop the running containers with
make down\nWe'll then generate a new docker-compose.yml file that includes the new tag with
make -B docker-compose.yml\nAfter that, we pull the new containers with
make pull\nAnd finally we deploy the updated containers by running
make up\nYou can check that everything is running at the version you've specified with
docker ps -a\nThe version that's running can be confirmed by looking at the IMAGE column in the output.
Major version updates may require a bit more work after the new containers are up and running, if the new images are running higher major versions of software. For example, if there has been an update to PHP or Solr.
"},{"location":"installation/docker-maintain-isle/#solr","title":"Solr","text":"When upgrading to a new major version of Solr, you need to regenerate a new set of config files. You will need to do this on both your development site, and your production site.
First, remove the old solr data. This will remove everything that has been indexed, so we will have to reindex our site in a later step.
docker compose exec -T solr with-contenv bash -lc 'rm -r server/solr/ISLANDORA'\nNext, recreate our solr core with a new set of config files.
make solr-cores\nFinally, we reindex our site.
make reindex-solr\nDrupal updates will be handled by your composer.json file, but you may require a certain version of ISLE in order to have the correct version of PHP. Once you have the correct PHP version installed, you can update Drupal to a new major version in the same way you normally update modules. It may be helpful to reference the Starter Site's composer.json file at https://github.com/Islandora-Devops/islandora-starter-site/blob/main/composer.json
Once you have done this on your development site, the process for your production site is the same as any other Drupal updates.
"},{"location":"installation/docker-maintain-isle/#mariadb","title":"Mariadb","text":"After updating Mariadb, you may need to run mariadb-upgrade inside your mariadb container, to update your system tables. This should be safe to run any time, but it is a good idea to back up your database first, just in case.
You can run this from your isle directory with
docker compose exec mariadb mariadb-upgrade\nUpgrading ISLE from 1.x to the next major version requires the TAG be set to 2.0.5 or higher. Once you create your containers you will need to follow the above instructions for Mariadb, Solr, and Drupal.
ISLE 2.0 bumps PHP up to version 8.1, which allows you to upgrade to Drupal 10.
"},{"location":"installation/docker-prereq/","title":"Prerequisites","text":""},{"location":"installation/docker-prereq/#software-involved-in-isle","title":"Software involved in ISLE","text":"ISLE uses the following software in order to help you set up and run your site. If you are new to any of the following software, that's OK, but learning more about each piece will make it much easier to modify ISLE to suit your particular needs, and maintain your Islandora site.
The following pieces of software are not unique to Islandora, and there are many great resources online to learn more about them. Below is a brief introduction to each piece and how they fit into your Islandora site. There are also many great videos on the Islandora Foundation's YouTube page if you would like to learn more about how they are used within the context of Islandora.
"},{"location":"installation/docker-prereq/#git","title":"Git","text":"Git is a version control system that is used to store the code for Islandora. When you first download ISLE you will be downloading it using Git. Git is also useful for managing your site's Drupal codebase, once you have a site up and running.
More about using Git for Drupal sites can be found in the Drupal documentation.
"},{"location":"installation/docker-prereq/#docker","title":"Docker","text":"Docker allows us to run our Islandora sites in virtualized environments, called containers. Each container runs one part of the Islandora software, and it takes several containers to run a complete Islandora site.
The three main components of Docker are Images, Containers, and Volumes.
"},{"location":"installation/docker-prereq/#images","title":"Images","text":"Images are what our containers are built from. Images are read-only, and allow us to quickly create containers based on them.
The Islandora images are provided by Islandora Buildkit.
"},{"location":"installation/docker-prereq/#containers","title":"Containers","text":"Containers are where the Islandora software actually runs. ISLE gives us tools to easily download the Islandora images and start our containers from them.
"},{"location":"installation/docker-prereq/#volumes","title":"Volumes","text":"Volumes are where our persistent storage lives. Because containers are often rebuilt from images, any data stored in them is easily lost. Volumes allow us to store specific parts of our containers that we don't want to lose. For example, our Drupal database will live in a volume so that it is not lost when a container is shut down.
"},{"location":"installation/docker-prereq/#docker-compose","title":"Docker Compose","text":"Docker Compose is a tool to simplify the process of running multiple Docker containers for a single project. It uses a file called docker-compose.yml to store the settings for your project's containers. ISLE gives us tools to create this docker-compose.yml file, so we can use it to manage the containers for our site.
"},{"location":"installation/docker-prereq/#gnu-make","title":"GNU Make","text":"Make allows us to define commands that simplify installing and maintaining our Islandora site. For a complete list of available commands see the Makefile included with ISLE.
"},{"location":"installation/docker-prereq/#composer","title":"Composer","text":"Composer is a dependency manager for PHP, and is the recommended way to install and update Drupal modules. Composer is installed in the Islandora Drupal container and should be run within the container to manage your Drupal site. More information on using Composer with Drupal is available in the Drupal documentation.
"},{"location":"installation/docker-prereq/#drupal","title":"Drupal","text":"Drupal is a Content Management System used to create websites. During the setup of your Islandora site, a Drupal site is created for you in your Drupal container, by the commands in the Makefile.
"},{"location":"installation/docker-prereq/#drush","title":"Drush","text":"Drush is a command line tool for managing your Drupal site. It comes installed in your Drupal container and allows you to perform Drupal actions from the command line inside your container.
"},{"location":"installation/docker-prereq/#requirements-for-using-isle-with-docker-compose","title":"Requirements for using ISLE with Docker Compose","text":"sudo apt update and sudo apt install make to installWhat are we missing?
Are you, or your computer, new to spinning up development sandboxes? Do you have wisdom about installing make on Windows? We suspect these requirements may be incomplete and your experience would be appreciated at Issue #1640.
"},{"location":"installation/docker-prereq/#installing-docker","title":"Installing Docker","text":"To see if you have Docker installed, type docker --version in a terminal.
If you need to install Docker, we recommend using the application Docker Desktop. It provides a GUI for managing Docker container in Windows and MacOS, along with the Docker engine and suite of command-line tools. Linux users don't get a desktop client, but can install the Engine and command-line tools the instructions here.
Memory, Processors, and Swap Requirements
To run ISLE on Docker Desktop, you must increase the resources allocated to the software. See Docker docs on setting resources on Windows (see note on how to allocate/restrict memory when using WSL 2) or setting resources on Mac.
CPUs (processors): The CPUs allowed to Docker Desktop are still shared with the host machine, so increasing this to the maximum value should allow both the Docker containers and your host machine to run simultaneously.
Memory (RAM): This memory is completely dedicated to Docker while Docker Desktop is running, so do not allocate more than you can spare and still run your host machine. Windows users may not require as much memory for Docker as Mac users. Current suggestions for memory allocated to Docker Desktop are below, but please edit this document if you have new information.
Swap: Swap space is space borrowed from your hard disk drive to serve as makeshift RAM as needed. If you cannot provide as much RAM as you would like, increase this as is reasonable given your free disk space.
"},{"location":"installation/docker-prereq/#installing-isle-docker-compose","title":"Installing ISLE Docker Compose","text":"Use Git to install the ISLE Docker Compose tool:
git clone https://github.com/islandora-devops/isle-dc
Tagged versions are available here.
"},{"location":"installation/docker-troubleshooting/","title":"Troubleshooting","text":""},{"location":"installation/docker-troubleshooting/#make-command-not-found","title":"Make command not found","text":"If you get the error bash: make: command not found then you need to install GNU make. Run sudo apt update and sudo apt install make to install.
If you get an error such as: ERROR: Version in \"./docker-compose.activemq.yml\" is unsupported., then you need to upgrade Docker. Enter the command make clean before re-attempting to make demo.
make demo","text":"If make fails for any reason, enter make clean before attempting to make again. If not, you may see an error such as: ERROR: Top level object in './docker-compose.yml' needs to be an object not '<class 'NoneType'>'.
If you notice some Docker containers drop (exited(0)), and (in Docker Desktop) the isle-dc app icon is yellow instead of green, try increasing the resources allocated to Docker (see note above).
"},{"location":"installation/docker-troubleshooting/#connection-timed-out-mac","title":"Connection timed out (Mac).","text":"If you are using Docker Desktop for Mac, and get timeout errors when spinning up the containers (during docker-compose up -d or during make local) such as this:
ERROR: for isle-dc_mariadb_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=480)\nyou can try quitting Docker completely (make sure there is no whale icon in your top toolbar - may need to select \"Quit\" from the whale icon itself) and then restart Docker.
"},{"location":"installation/docker-troubleshooting/#504-bad-gateway","title":"504 Bad Gateway","text":"If you get a white screen with a 504 Bad Gateway error, this means your containers haven't finished initializing themselves yet. If you've waiting an appropriate amount of time (2-5 minutes), then there is most likely an error in a container's startup script. Use docker ps -a to see which services have Exited status, and then tail their logs with docker-compose logs service_name.
To test out Islandora, there is a Starter-Site based Portainer demo available (Download the .zip file). This demo is loaded with sample content to demonstrate features of Islandora and provide a starting point for exploration. Follow the instructions in the README.md.html file provided in the download.
It is recommended that you have a minimum of 4GB of hard drive space available on your machine before getting started. It will take some time to build the image, do not be surprised if it takes a long time. Thanks, and happy building!
"},{"location":"installation/install-a-demo/#installation-video-tutorial","title":"Installation Video Tutorial","text":"The README.md.html is the most up to date version of the installation documentation, but you can also view this video: How to Install Docker Desktop and Set up Starter Site (video tutorial).
"},{"location":"installation/install-a-demo/#troubleshooting","title":"Troubleshooting","text":"These troubleshooting issues and solutions for Docker Desktop have arised from and been used with Windows 10.
"},{"location":"installation/install-a-demo/#docker-failed-to-start-error-message","title":"\"Docker Failed to Start\" Error Message","text":"For Windows 10, you may receive the \"Docker Failed to Start\" Error message. To resolve this:
If no content is showing up in your local installation when on islandora.io in a browser, you may need to wait longer for the deployment process to complete. 1. Go to \u2018containers\u2019, click the \u2018logs\u2019 for \u2018sandbox-drupal-1\u2019 to see the deployment process. When completed you will see:
#####################\n# Install Completed #\n#####################\nIf Docker Desktop and Virtual Machine (Vmmem in Task Manager) are taking up too much CPU and RAM, you can create a .wslconfig file to limit how much of your computers resources are allocated to these processes: 1. Open Notepad++ and type the following:
[wsl2]\nmemory=4GB\nprocessors=2\nThe Islandora Playbook (Islandora-Devops/islandora-playbook) is a tool for installing the Islandora stack on a single virtual machine. It can be used both as a Vagrant project to create a local development environment, or as an Ansible playbook to provision a local or remote server. It can set up a Drupal based either on Islandora Starter Site, or on the Install Profile Demo.
"},{"location":"installation/playbook/#basic-usage-vagrant","title":"Basic Usage (Vagrant):","text":"Install requirements (below), then:
$ git clone -b dev https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\n$ ISLANDORA_BUILD_BASE=true vagrant up\n$ vagrant package --output islandora_base\n$ vagrant destroy\n$ vagrant up\nWhen used this way, you can trash your working site and get a fresh Islandora relatively quickly, with vagrant destroy (you will be asked to confirm, as this will delete all your changes and your content), and then vagrant up.
Full instructions below.
"},{"location":"installation/playbook/#requirements-vagrant","title":"Requirements (Vagrant)","text":"To create a local VM, download and install the following.
OpenSSL is already pre-installed on MacOS. Git can be installed using XCode's command line tools (see below). Python and Pip can either be installed via the downloaded installer direct from the site or via Homebrew (not shown below). Ansible is best installed using Homebrew (see below).
# Use xcode-select to install command line components, including git\n$ xcode-select --install\n# Install homebrew\n$ /usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n# Use homebrew to install ansible\n$ brew install ansible\nYou will use vagrant up twice, first to create an Islandora base box, and then again to provision that base box into a full Islandora. This method uses Vagrant, VirtualBox, and Ansible.
Warn
Make sure that no required ports are currently in use.
"},{"location":"installation/playbook/#clone-the-playbook","title":"Clone the playbook","text":"$ git clone -b dev https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\nThe Playbook will create a \"base box\" that includes core software (PHP, Apache, etc). Since these do not need to be updated too often, this can be packaged as a Vagrant base box that will be used to quickly provision the Islandora part.
Notes: - If building a CentOS box, you also need to install the vbguest additions with vagrant plugin install vagrant-vbguest. - If this is not your first time spinning up the Islandora Playbook from this directory, you may want to clear cached ansible roles in roles/external\" rm -rf roles/external
First, create the islandora base box by overriding the Vagrant variable ISLANDORA_BUILD_BASE (which defaults to false in the Vagrantfile):
$ ISLANDORA_BUILD_BASE=true vagrant up\nOnce it is complete (without errors), store it as a Vagrant base box. It will appear as islandora_base in the current directory.
$ vagrant package --output islandora_base\nThen, get rid of the existing VM since we've saved what we need:
$ vagrant destroy\n default: Are you sure you want to destroy the 'default' VM? [y/N] y\n==> default: Forcing shutdown of VM...\n==> default: Destroying VM and associated drives...\nThen, to install Islandora (including Drupal and its configuration, Crayfish, Alpaca, etc) on top of the new base box:
vagrant up\nAccess your site at http://localhost:8000.
You can quickly create a new \"out of the box\" Islandora VM by just re-doing the last step (provisioning Islandora) while using your existing base box. In doing this, you will destroy your existing VM, and all of its configuration and data. This will not upgrade PHP, apache, or other back-end services (see below, \"Refreshing your base box.\")
vagrant destroy\nvagrant up\nIf you want to use the Playbook to rebuild the base box (e.g. if this playbook now includes updated versions of PHP or Apache), then you will need to take some extra steps to ensure that Vagrant sees your new base box.
vagrant destroy # Delete your existing VM\nvagrant box remove islandora_base # clear Vagrant's cached version of islandora_base\nISLANDORA_BUILD_BASE=true vagrant up # and proceed to package the box, as above.\nThis is because once you use the base box once, it is stored in the vagrant box list under your home directory. While subsequent builds of ISLANDORA_BUILD_BASE=true vagrant up will build a new box and you can package it, but that file does not update the cached box and consequently you still get the old base box when you try build a VM. See vagrant box documentation for more useful commands for managing base boxes.
The ansible-playbook command using playbook.yml can provision a remote environment. First, you will need to create a new folder in inventory/ with the details of your remote deployment. This includes:
Before beginning, you may want to:
sudo) as this user. See Ansible Docs on Understanding privilege escalation: becomeWe're going to build up this new remote environment configuration from the default provided Vagrant configuration. To start, take the inventory for the vagrant development environment and make a copy of it. Be sure to give it an appropriate name. Here we're using production.
$ git clone https://github.com/Islandora-Devops/islandora-playbook\n$ cd islandora-playbook\n$ cp -r inventory/vagrant inventory/production\nThen you can update the following entries in the following files using your own information. If an entry does not exist in a file, just add it. Ansible will then use the value you provide instead of relying on its defaults.
We're using changeme to represent passwords and assume the server will be available at example.org, but you'll want to provide your own values.
# Drupal\ndrupal_db_password: changeme\ndrupal_account_pass: changeme\n\n# MySQL/Postgres\nislandora_db_root_password: changeme\n\n# Tomcat\nislandora_tomcat_password: changeme\n\n# Syn\nislandora_syn_token: changeme\n\n# Cantaloupe\ncantaloupe_admin_password: changeme\n\n# Fedora\nfcrepo_db_password: changeme\nThis is where we specify that the webserver is listening on the default port 80, instead of the development machine port 8000.
apache_listen_port: 80\nYou will have to add the matomo line.
matomo_site_url: http://example.org\nIf you have a domain name, change the default to your domain name.
- servername: \"myactualddomain.com\"\nIf you have a domain name, change the default to your domain name. And set the Drupal site name to whatever you need it to be.
drupal_domain: \"myactualdomain.com\"\ndrupal_site_name: \"Example Sitename\"\ndrupal_trusted_hosts: drupal_trusted_hosts:\n - ^localhost$\n - \"{{ hostvars[groups['webserver'][0]].ansible_host }}\"\n - '^myactualdomain\\.com$'\nNote the backslash which escapes the period (which would otherwise match any character). Because of this escape character, the string needs to be surrounded by single quotes.
"},{"location":"installation/playbook/#hosts","title":"hosts","text":"You'll need to put particulars for logging into your server in the inventory/production/hosts file . This example is set up to login as the ansible user (to avoid trying to run Composer as root) and uses an SSH key. You'll need to get the details for logging into your remote server from your hosting provider (AWS, Digital Ocean, etc...) or your systems administrator if you're running the server in-house. See this page for more details about what you can put into a host file
default ansible_host=example.org ansible_port=22 ansible_user=ansible ansible_ssh_private_key_file='/home/username/.ssh/id_rsa'\nFirst, you'll want to install the ansible roles that are needed for the version of Islandora you are trying to install. This can be done with
$ ansible-galaxy install -r requirements.yml\nThen, if you are on Ubuntu 22.04, run the following commands:
$ ansible-playbook -i inventory/production playbook.yml -e \"islandora_distro=ubuntu/jammy64\" -e \"islandora_profile=starter\" -e \"islandora_build_base_box=true\"\n$ ansible-playbook -i inventory/production playbook.yml -e \"islandora_distro=ubuntu/jammy64\" -e \"islandora_profile=starter\" -e \"islandora_build_base_box=false\"\nAnsible caches the code used to provision the environment, so if you've already installed once you may not be getting the latest version of things even if you've git pull'd the latest playbook. The code is stored in roles/external, so if you want to clear it out you can remove these before attempting to provision an environment
$ rm -rf roles/external\nWhen provisioning using a local environment, you should be aware of any ports that are already in use by your computer that are also going to be used by Vagrant, as these may clash and cause problems during and after provisioning. These include:
If there are port clashes for any of these, you will need to either find and replace them in the configuration .yml files under inventory/vagrant/group_vars, or provide new values for the different playbooks that support changing the ports (for example, postgresql_databases supports adding a port property which is currently simply unused). You will also need to replace the port forwarding values in Vagrantfile.
Additionally, Ansible attempts to use port 2200 for SSH. If this port is already in use, your local environment cannot be provisioned. To change this, set a new value for ansible_port in inventory/vagrant/hosts.
If you run into any issues installing the environment, do not hesitate to email the mailing list to ask for help. If you think you've stumbled across a bug in the installer, please create an issue in the Islandora issue queue and give it an ansible tag.
The ISLE Site Template is a system for installing Islandora on Docker. As with ISLE-DC, it uses Docker Compose to orchestrate the installation of all the different services (Docker containers) that make up Islandora. Unlike ISLE-DC, in ISLE Site Template you use Docker Compose commands directly, helping you to get familiar with the kinds of commands that will be a key part of running and maintaining Islandora.
"},{"location":"installation/site-template/#usage","title":"Usage","text":"Do not clone the Isle Site Template!
curl and installed either manually or automatically.Instead, follow the instructions in the ISLE Site Template's README.md and README.template.md files.
dev and prod environments, with different services available on each.During installation, you will install a copy of the Islandora Starter Site.
Customizing your site can be persisted to your own repo.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
After all of the above pieces are in place, installed, configured, started, and otherwise prepared, the last thing we need to do is to finally configure the front-end Drupal instance to wire all the installed components together.
"},{"location":"installation/manual/configuring-drupal/#drupal-pre-configuration","title":"Drupal Pre-Configuration","text":""},{"location":"installation/manual/configuring-drupal/#settingsphp","title":"settings.php","text":"Notice
By default, settings.php is read-only for all users. It should be made writable while this pre-configuration is being done, then set back to 444 afterwards.
Some additional settings will need to be established in your default settings.php before Drupal-side configuration can occur.
The below configuration will establish localhost as a trusted host pattern, but on production sites this will need to be expanded to include the actual host patterns used by the site.
/opt/drupal/web/sites/default/settings.php
Before (at around line 789):
'driver' => 'pgsql',\n);\nAfter:
'driver' => 'pgsql',\n);\n\n$settings['trusted_host_patterns'] = [\n 'localhost',\n];\n\n$settings['flysystem'] = [\n 'fedora' => [\n 'driver' => 'fedora',\n 'config' => [\n 'root' => 'http://localhost:8080/fcrepo/rest/',\n ],\n ],\n];\nOnce this is done, refresh the cache to take hold of the new settings.
cd /opt/drupal\ndrush -y cr\nSkip this by using the Islandora Starter Site
The Islandora Starter Site, which was presented as an option in the \"Installing Composer, Drush, and Drupal\" step, installs Islandora and other modules and configures them, allowing you to skip this section. You may want to use this manual method in the case where you want to pick and choose which Islandora modules you use.
"},{"location":"installation/manual/configuring-drupal/#downloading-islandora","title":"Downloading Islandora","text":"The Islandora Drupal module contains the core code to create a repository ecosystem in a Drupal environment. It also includes several submodules; of importance to us is islandora_core_feature, which contains the key configurations that allow you to use Islandora features.
Take note of some of the other comments in the below bash script for an idea of what the other components are expected, and which may be considered optional.
cd /opt/drupal\n# Since islandora_defaults is near the bottom of the dependency chain, requiring\n# it will get most of the modules and libraries we need to deploy a standard\n# Islandora site.\nsudo -u www-data composer require \"drupal/flysystem:^2.0@alpha\"\nsudo -u www-data composer require \"islandora/islandora:^2.4\"\nsudo -u www-data composer require \"islandora/controlled_access_terms:^2\"\nsudo -u www-data composer require \"islandora/openseadragon:^2\"\n# These can be considered important or required depending on your site's\n# requirements; some of them represent dependencies of Islandora submodules.\nsudo -u www-data composer require \"drupal/pdf:1.1\"\nsudo -u www-data composer require \"drupal/rest_oai_pmh:^2.0@beta\"\nsudo -u www-data composer require \"drupal/search_api_solr:^4.2\"\nsudo -u www-data composer require \"drupal/facets:^2\"\nsudo -u www-data composer require \"drupal/content_browser:^1.0@alpha\" ## TODO do we need this?\nsudo -u www-data composer require \"drupal/field_permissions:^1\"\nsudo -u www-data composer require \"drupal/transliterate_filenames:^2.0\"\n# These tend to be good to enable for a development environment, or just for a\n# higher quality of life when managing Islandora. That being said, devel should\n# NEVER be enabled on a production environment, as it intentionally gives the\n# user tools that compromise the security of a site.\nsudo -u www-data composer require drupal/restui:^1.21\nsudo -u www-data composer require drupal/console:~1.0\nsudo -u www-data composer require drupal/devel:^2.0\nsudo -u www-data composer require drupal/admin_toolbar:^2.0\nComponents we've now downloaded using composer require can be enabled simultaneously via drush, which will ensure they are installed in the correct dependent order. Enabling islandora_defaults will also ensure all content types and configurations are set up in Islandora. The installation process for all of these modules will likely take some time.
Notice
This list of modules assumes that all of the above components were downloaded using composer require; if this is not the case, you may need to pare down this list manually. It also includes devel, which again, should not be enabled on production sites.
cd /opt/drupal\ndrush -y en rdf responsive_image devel syslog serialization basic_auth rest restui search_api_solr facets content_browser pdf admin_toolbar controlled_access_terms_defaults islandora_breadcrumbs islandora_iiif islandora_oaipmh\n# After all of this, rebuild the cache.\ndrush -y cr\nTo allow our installation to talk to other services via Syn, we need to establish a Drupal-side JWT configuration using the keys we generated at that time.
Log onto your site as an administrator at /user, then navigate to /admin/config/system/keys/add. Some of the settings here are unimportant, but pay close attention to the Key type, which should match the key we created earlier (an RSA key), and the File location, which should be the ultimate location of the key we created for Syn on the filesystem, /opt/keys/syn_private.key.
Click Save to create the key.
Once this key is created, navigate to /admin/config/system/jwt to select the key you just created from the list. Note that before the key will show up in the Private Key list, you need to select that key's type in the Algorithm section, namely RSASSA-PKCS1-v1_5 using SHA-256 (RS256).
Click Save configuration to establish this as the JWT key configuration.
"},{"location":"installation/manual/configuring-drupal/#configuring-islandora","title":"Configuring Islandora","text":"Navigate to the Islandora core configuration page at /admin/config/islandora/core to set up the core configuration to connect to Gemini. Of note here, the Gemini URL will need to be established to facilitate the connection to Fedora, and the appropriate Bundles with Gemini URI pseudo field types will need to be checked off.
Notice
Any other Drupal content types you wish to synchronize with Fedora should also be checked off here.
"},{"location":"installation/manual/configuring-drupal/#configuring-islandora-iiif","title":"Configuring Islandora IIIF","text":"Navigate to /admin/config/islandora/iiif to ensure that Islandora IIIF is pointing to our Cantaloupe server.
Next, configure OpenSeadragon by navigating to /admin/config/media/openseadragon and ensuring everything is set up properly.
Navigate to /admin/config/media/file-system to set the Default download method to the one we created in our settings.php.
fedoraAdmin Role","text":"In order for data to be pushed back to Fedora, the site administrative user needs the fedoraAdmin role.
cd /opt/drupal\nsudo -u www-data drush -y urol \"fedoraadmin\" islandora\nFinally, to get everything up and running, run the Islandora Core Features and Islandora Defaults migrations.
cd /opt/drupal\nsudo -u www-data drush -y -l localhost --userid=1 mim --group=islandora\nSome views provided by Islandora are not enabled by default.
cd /opt/drupal\ndrush -y views:enable display_media\nNeeds Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Composer provides PHP code that we can use to install it. After downloading and running the installer, we\u2019re going to move the generated executable to a place in $PATH, removing its extension:
curl \"https://getcomposer.org/installer\" > composer-install.php\nchmod +x composer-install.php\nphp composer-install.php\nsudo mv composer.phar /usr/local/bin/composer\nAt this point, you have the option of using the Islandora Starter Site, with its pre-selected modules and configurations which function \"out of the box,\" or build a clean stock Drupal via the Drupal Recommended Project and install Islandora modules as you desire.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#option-1-create-a-project-using-the-islandora-starter-site","title":"Option 1: Create a project using the Islandora Starter Site","text":"Navigate to the folder where you want to put your Islandora project (in our case /var/www), and create the Islandora Starter Site:
cd /var/www\ncomposer create-project islandora/islandora-starter-site\nThis will install all PHP dependencies, including Drush, and scaffold the site.
Drush is not accessible via $PATH, but is available using the command composer exec -- drush
Navigate to the folder where you want to put your Drupal project (in our case /var/www), and create the Drupal Recommended Project:
cd /var/www\ncomposer create-project drupal/recommended-project my-project\nBefore we can proceed with the actual site installation, we\u2019re going to need to make our new Drupal installation the default web-accessible location Apache serves up. This will include an appropriate ports.conf file, and replacing the default enabled site.
Notice
Out of the box, these files will contain support for SSL, which we will not be setting up in this guide (and therefore removing with these overwritten configurations), but which are absolutely indispensable to a production site. This guide does not recommend any particular SSL certificate authority or installation method, but you may find DigitalOcean's tutorial helpful.
/etc/apache2/ports.conf | root:root/644
Listen 80\nRemove everything but the \"Listen 80\" line. You can leave the comments in if you want.
/etc/apache2/sites-enabled/000-default.conf | root:root/777
<VirtualHost *:80>\nServerName SERVER_NAME\n DocumentRoot \"/opt/drupal/web\"\n <Directory \"/opt/drupal/web\">\nOptions Indexes FollowSymLinks MultiViews\n AllowOverride all\n Require all granted\n </Directory>\n# Ensure some logging is in place.\n ErrorLog \"/var/log/apache2/localhost_error.log\"\n CustomLog \"/var/log/apache2/localhost_access.log\" combined\n</VirtualHost>\nSERVER_NAME: localhost - For a development environment hosted on your own machine or a VM, localhost should suffice. Realistically, this should be the domain or IP address the server will be accessed at. Restart the Apache 2 service to apply these changes:
sudo systemctl restart apache2\nPostgreSQL roles are directly tied to users. We\u2019re going to ensure a user is in place, create a role for them in PostgreSQL, and create a database for them that we can use to install Drupal.
# Run psql as the postgres user, the only user currently with any PostgreSQL\n# access.\nsudo -u postgres psql\n# Then, run these commands within psql itself:\ncreate database DRUPAL_DB encoding 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8' TEMPLATE template0;\ncreate user DRUPAL_DB_USER with encrypted password 'DRUPAL_DB_PASSWORD';\ngrant all privileges on database DRUPAL_DB to DRUPAL_DB_USER;\n# Then, quit psql.\n\\q\nDRUPAL_DB: drupal9 - This will be used as the core database that Drupal is installed into - DRUPAL_DB_USER: drupal - Specifically, this is the user that will connect to the PostgreSQL database being created, not the user that will be logging into Drupal - DRUPAL_DB_PASSWORD: drupal - This should be a secure password; it\u2019s recommended to use a password generator to create this such as the one provided by random.org"},{"location":"installation/manual/installing-composer-drush-and-drupal/#install-drupal-using-drush","title":"Install Drupal using Drush","text":"The Drupal installation process can be done through the GUI in a series of form steps, or can be done quickly using Drush's site-install command. It can be invoked with the full list of parameters (such as --db-url and --site-name), but if parameters are missing, they will be asked of you interactively.
Follow the instructions in the README of the Islandora Starter Site. The steps are not reproduced here to remove redundancy. When this installation is done, you'll have a starter site ready-to-go. Once you set up the external services in the next sections, you'll need to configure Drupal to know where they are.
"},{"location":"installation/manual/installing-composer-drush-and-drupal/#option-2-site-install-the-basic-drupal-recommended-project","title":"Option 2: Site install the basic Drupal Recommended Project","text":"cd /var/www/drupal\ndrush -y site-install standard --db-url=\"pgsql://DRUPAL_DB_USER:DRUPAL_DB_PASSWORD@127.0.0.1:5432/DRUPAL_DB\" --site-name=\"SITE_NAME\" --account-name=DRUPAL_LOGIN --account-pass=DRUPAL_PASS\nSITE_NAME: Islandora 2.0DRUPAL_LOGIN: islandoraDRUPAL_PASS: islandoraCongratulations, you have a Drupal site! It currently isn\u2019t really configured to do anything, but we\u2019ll get those portions set up in the coming sections.
"},{"location":"installation/manual/installing-crayfish/","title":"Installing Crayfish","text":"Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-crayfish/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Some packages need to be installed before we can proceed with installing Crayfish; these packages are used by the microservices within Crayfish. These include:
apt-get, and a list of available packs can be procured with sudo apt-cache search tesseract-ocrsudo add-apt-repository -y ppa:lyrasis/imagemagick-jp2\nsudo apt-get update\nsudo apt-get -y install imagemagick tesseract-ocr ffmpeg poppler-utils\nNOTICE: If you get the sudo: apt-add-repository: command not found, run sudo apt-get install software-properties-common in order to make the command available.
We\u2019re going to clone Crayfish to /opt, and individually run composer install against each of the microservice subdirectories.
cd /opt\nsudo git clone https://github.com/Islandora/Crayfish.git crayfish\nsudo chown -R www-data:www-data crayfish\nsudo -u www-data composer install -d crayfish/Homarus\nsudo -u www-data composer install -d crayfish/Houdini\nsudo -u www-data composer install -d crayfish/Hypercube\nsudo -u www-data composer install -d crayfish/Milliner\nsudo -u www-data composer install -d crayfish/Recast\nNot much needs to happen here; Crayfish opts for a simple logging approach, with one .log file for each component. We\u2019ll create a folder where each logfile can live.
sudo mkdir /var/log/islandora\nsudo chown www-data:www-data /var/log/islandora\nEach Crayfish component requires one or more .yaml file(s) to ensure everything is wired up correctly.
NOTICE
The following configuration files represent somewhat sensible defaults; you should take consideration of the logging levels in use, as this can vary in desirability from installation to installation. Also note that in all cases, http URLs are being used, as this guide does not deal with setting up https support. In a production installation, this should not be the case. These files also assume a connection to a PostgreSQL database; use a pdo_mysql driver and the appropriate 3306 port if using MySQL.
/opt/crayfish/Homarus/cfg/config.yaml | www-data:www-data/644
---\nhomarus:\nexecutable: ffmpeg\nmime_types:\nvalid:\n- video/mp4\n- video/x-msvideo\n- video/ogg\n- audio/x-wav\n- audio/mpeg\n- audio/aac\n- image/jpeg\n- image/png\ndefault: video/mp4\nmime_to_format:\nvalid:\n- video/mp4_mp4\n- video/x-msvideo_avi\n- video/ogg_ogg\n- audio/x-wav_wav\n- audio/mpeg_mp3\n- audio/aac_m4a\n- image/jpeg_image2pipe\n- image/png_image2pipe\ndefault: mp4\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/homarus.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\nCurrently the Houdini microservice uses a different system (Symfony) than the other microservices, this requires different configuration.
/opt/crayfish/Houdini/config/services.yaml | www-data:www-data/644
# This file is the entry point to configure your own services.\n# Files in the packages/ subdirectory configure your dependencies.\n# Put parameters here that don't need to change on each machine where the app is deployed\n# https://symfony.com/doc/current/best_practices/configuration.html#application-related-configuration\nparameters:\napp.executable: /usr/local/bin/convert\napp.formats.valid:\n- image/jpeg\n- image/png\n- image/tiff\n- image/jp2\napp.formats.default: image/jpeg\nservices:\n# default configuration for services in *this* file\n_defaults:\nautowire: true # Automatically injects dependencies in your services.\nautoconfigure: true # Automatically registers your services as commands, event subscribers, etc.\n# makes classes in src/ available to be used as services\n# this creates a service per class whose id is the fully-qualified class name\nApp\\Islandora\\Houdini\\:\nresource: '../src/*'\nexclude: '../src/{DependencyInjection,Entity,Migrations,Tests,Kernel.php}'\n# controllers are imported separately to make sure services can be injected\n# as action arguments even if you don't extend any base controller class\nApp\\Islandora\\Houdini\\Controller\\HoudiniController:\npublic: false\nbind:\n$formats: '%app.formats.valid%'\n$default_format: '%app.formats.default%'\n$executable: '%app.executable%'\ntags: ['controller.service_arguments']\n# add more service definitions when explicit configuration is needed\n# please note that last definitions always *replace* previous ones\n/opt/crayfish/Houdini/config/packages/crayfish_commons.yml | www-data:www-data/644
crayfish_commons:\nfedora_base_uri: 'http://localhost:8080/fcrepo/rest'\nsyn_config: '/opt/fcrepo/config/syn-settings.xml'\n/opt/crayfish/Houdini/config/packages/monolog.yml | www-data:www-data/644
monolog:\nhandlers:\nhoudini:\ntype: rotating_file\npath: /var/log/islandora/Houdini.log\nlevel: DEBUG\nmax_files: 1\nThe below files are two versions of the same file to enable or disable JWT token authentication.
/opt/crayfish/Houdini/config/packages/security.yml | www-data:www-data/644
Enabled JWT token authentication:
security:\n# https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers\nproviders:\njwt_user_provider:\nid: Islandora\\Crayfish\\Commons\\Syn\\JwtUserProvider\nfirewalls:\ndev:\npattern: ^/(_(profiler|wdt)|css|images|js)/\nsecurity: false\nmain:\nanonymous: false\n# Need stateless or it reloads the User based on a token.\nstateless: true\nprovider: jwt_user_provider\nguard:\nauthenticators:\n- Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator\n# activate different ways to authenticate\n# https://symfony.com/doc/current/security.html#firewalls-authentication\n# https://symfony.com/doc/current/security/impersonating_user.html\n# switch_user: true\n# Easy way to control access for large sections of your site\n# Note: Only the *first* access control that matches will be used\naccess_control:\n# - { path: ^/admin, roles: ROLE_ADMIN }\n# - { path: ^/profile, roles: ROLE_USER }\nDisabled JWT token authentication:
security:\n# https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers\nproviders:\njwt_user_provider:\nid: Islandora\\Crayfish\\Commons\\Syn\\JwtUserProvider\nfirewalls:\ndev:\npattern: ^/(_(profiler|wdt)|css|images|js)/\nsecurity: false\nmain:\nanonymous: true\n# Need stateless or it reloads the User based on a token.\nstateless: true\n/opt/crayfish/Hypercube/cfg/config.yaml | www-data:www-data/644
---\nhypercube:\ntesseract_executable: tesseract\npdftotext_executable: pdftotext\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/hypercube.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\n/opt/crayfish/Milliner/cfg/config.yaml | www-data:www-data/644
---\nfedora_base_url: http://localhost:8080/fcrepo/rest\ndrupal_base_url: http://localhost\nmodified_date_predicate: http://schema.org/dateModified\nstrip_format_jsonld: true\ndebug: false\ndb.options:\ndriver: pdo_pgsql\nhost: 127.0.0.1\nport: 5432\ndbname: CRAYFISH_DB\nuser: CRAYFISH_DB_USER\npassword: CRAYFISH_DB_PASSWORD\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/milliner.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\n/opt/crayfish/Recast/cfg/config.yaml | www-data:www-data/644
---\nfedora_resource:\nbase_url: http://localhost:8080/fcrepo/rest\ndrupal_base_url: http://localhost\ndebug: false\nlog:\nlevel: NOTICE\nfile: /var/log/islandora/recast.log\nsyn:\nenable: true\nconfig: /opt/fcrepo/config/syn-settings.xml\nnamespaces:\n-\nacl: \"http://www.w3.org/ns/auth/acl#\"\nfedora: \"http://fedora.info/definitions/v4/repository#\"\nldp: \"http://www.w3.org/ns/ldp#\"\nmemento: \"http://mementoweb.org/ns#\"\npcdm: \"http://pcdm.org/models#\"\npcdmuse: \"http://pcdm.org/use#\"\nwebac: \"http://fedora.info/definitions/v4/webac#\"\nvcard: \"http://www.w3.org/2006/vcard/ns#\"\nFinally, we need appropriate Apache configurations for Crayfish; these will allow other services to connect to Crayfish components via their HTTP endpoints.
Each endpoint we need to be able to connect to will get its own .conf file, which we will then enable.
NOTICE
These configurations would potentially have collisions with Drupal routes, if any are created in Drupal with the same name. If this is a concern, it would likely be better to reserve a subdomain or another port specifically for Crayfish. For the purposes of this installation guide, these endpoints will suffice.
/etc/apache2/conf-available/Homarus.conf | root:root/644
Alias \"/homarus\" \"/opt/crayfish/Homarus/src\"\n<Directory \"/opt/crayfish/Homarus/src\">\n FallbackResource /homarus/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Houdini.conf | root:root/644
Alias \"/houdini\" \"/opt/crayfish/Houdini/public\"\n<Directory \"/opt/crayfish/Houdini/public\">\n FallbackResource /houdini/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Hypercube.conf | root:root/644
Alias \"/hypercube\" \"/opt/crayfish/Hypercube/src\"\n<Directory \"/opt/crayfish/Hypercube/src\">\n FallbackResource /hypercube/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Milliner.conf | root:root/644
Alias \"/milliner\" \"/opt/crayfish/Milliner/src\"\n<Directory \"/opt/crayfish/Milliner/src\">\n FallbackResource /milliner/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\n/etc/apache2/conf-available/Recast.conf | root:root/644
Alias \"/recast\" \"/opt/crayfish/Recast/src\"\n<Directory \"/opt/crayfish/Recast/src\">\n FallbackResource /recast/index.php\n Require all granted\n DirectoryIndex index.php\n SetEnvIf Authorization \"(.*)\" HTTP_AUTHORIZATION=$1\n</Directory>\nEnabling each of these configurations involves creating a symlink to them in the conf-enabled directory; the standardized method of doing this in Apache is with a2enconf.
sudo a2enconf Homarus Houdini Hypercube Milliner Recast\nFinally, to get these new endpoints up and running, we need to restart the Apache service.
sudo systemctl restart apache2\nWe're going to stop the Tomcat service while working on setting up Fedora to prevent any autodeploy misconfigurations.
sudo systemctl stop tomcat\nFedora\u2019s configuration and data won\u2019t live with Tomcat itself; rather, we\u2019re going to prepare a space for them to make them easier to manage.
sudo mkdir -p /opt/fcrepo/data/objects\nsudo mkdir /opt/fcrepo/config\nsudo chown -R tomcat:tomcat /opt/fcrepo\nThe method for creating the database here will closely mimic the method we used to create our database for Drupal.
sudo -u postgres psql\ncreate database FEDORA_DB encoding 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8' TEMPLATE template0;\ncreate user FEDORA_DB_USER with encrypted password 'FEDORA_DB_PASSWORD';\ngrant all privileges on database FEDORA_DB to FEDORA_DB_USER;\n\\q\nFEDORA_DB: fcrepoFEDORA_DB_USER: fedoraFEDORA_DB_PASSWORD: fedorafedora is not recommended.The Fedora configuration is going to come in a few different chunks that need to be in place before Fedora will be functional. We\u2019re going to place several files outright, with mildly modified parameters according to our configuration.
The basics of these configuration files have been pulled largely from the templates in Islandora-Devops/islandora-playbook internal Fedora role; you may consider referencing the playbook\u2019s templates directory for more details.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#namespace-prefixes","title":"Namespace prefixes","text":"i8_namespaces.yml is a list of namespaces used by Islandora that may not necessarily be present in Fedora; we add them here to ensure we can use them in queries.
/opt/fcrepo/config/i8_namespaces.yml | tomcat:tomcat/644
# Islandora 8/Fedora namespaces\n#\n# This file contains ALL the prefix mappings, if a URI\n# does not appear in this file it will be displayed as \n# the full URI in Fedora. \nacl: http://www.w3.org/ns/auth/acl#\nbf: http://id.loc.gov/ontologies/bibframe/\ncc: http://creativecommons.org/ns#\ndc: http://purl.org/dc/elements/1.1/\ndcterms: http://purl.org/dc/terms/\ndwc: http://rs.tdwg.org/dwc/terms/\nebucore: http://www.ebu.ch/metadata/ontologies/ebucore/ebucore#\nexif: http://www.w3.org/2003/12/exif/ns#\nfedoraconfig: http://fedora.info/definitions/v4/config#\nfedoramodel: info:fedora/fedora-system:def/model#\nfoaf: http://xmlns.com/foaf/0.1/\ngeo: http://www.w3.org/2003/01/geo/wgs84_pos#\ngn: http://www.geonames.org/ontology#\niana: http://www.iana.org/assignments/relation/\nislandorarelsext: http://islandora.ca/ontology/relsext#\nislandorarelsint: http://islandora.ca/ontology/relsint#\nldp: http://www.w3.org/ns/ldp#\nmemento: http://mementoweb.org/ns#\nnfo: http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#\nore: http://www.openarchives.org/ore/terms/\nowl: http://www.w3.org/2002/07/owl#\npremis: http://www.loc.gov/premis/rdf/v1#\nprov: http://www.w3.org/ns/prov#\nrdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#\nrdfs: http://www.w3.org/2000/01/rdf-schema#\nrel: http://id.loc.gov/vocabulary/relators/\nschema: http://schema.org/\nskos: http://www.w3.org/2004/02/skos/core#\ntest: info:fedora/test/\nvcard: http://www.w3.org/2006/vcard/ns#\nwebac: http://fedora.info/definitions/v4/webac#\nxml: http://www.w3.org/XML/1998/namespace\nxmlns: http://www.w3.org/2000/xmlns/\nxs: http://www.w3.org/2001/XMLSchema\nxsi: http://www.w3.org/2001/XMLSchema-instance\nWe have Fedora provide metadata for some resources that are contained in Drupal. Fedora needs to know to allow access to these External Content hosts.
We create a file /opt/fcrepo/config/allowed_external_hosts.txt | tomcat:tomcat/644
http://localhost:8000/\nNote: the trailing backslash is important here. For more information on Fedora's External Content and configuring it, see the Fedora Wiki pages
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#fedora-configuration-properties-file","title":"Fedora configuration properties file","text":"Fedora 6 now allows you to put all your configuration properties into a single file. We use 0640 permissions as you will want to put your database credentials in here.
/opt/fcrepo/config/fcrepo.properties | tomcat:tomcat/640
fcrepo.home=FCREPO_HOME\n# External content using path defined above.\nfcrepo.external.content.allowed=/opt/fcrepo/config/allowed_external_hosts.txt\n# Namespace registry using path defined above.\nfcrepo.namespace.registry=/opt/fcrepo/config/i8_namespaces.yml\nfcrepo.auth.principal.header.enabled=true\n# The principal header is the syn-setting.xml \"config\" element's \"header\" attribute\nfcrepo.auth.principal.header.name=X-Islandora\n# false to use manual versioning, true to create a version on each change\nfcrepo.autoversioning.enabled=true\nfcrepo.db.url=FCREPO_DB_URL\nfcrepo.db.user=FCREPO_DB_USERNAME\nfcrepo.db.password=FCREPO_DB_PASSWORD\nfcrepo.ocfl.root=FCREPO_OCFL_ROOT\nfcrepo.ocfl.temp=FCREPO_TEMP_ROOT\nfcrepo.ocfl.staging=FCREPO_STAGING_ROOT\n# Can be sha512 or sha256\nfcrepo.persistence.defaultDigestAlgorithm=sha512\n# Jms moved from 61616 to allow external ActiveMQ to use that port\nfcrepo.dynamic.jms.port=61626\n# Same as above\nfcrepo.dynamic.stomp.port=61623\nfcrepo.velocity.runtime.log=FCREPO_VELOCITY_LOG\nfcrepo.jms.baseUrl=FCREPO_JMS_BASE\nFCREPO_HOME - The home directory for all Fedora generated output and state. Unless otherwise specified, all logs, metadata, binaries, and internally generated indexes, etc. It would default to the Tomcat starting directory. A good default would be /opt/fcrepoFCREPO_DB_URL - This parameter allows you to set the database connection url. In general the format is as follows:
jdbc:<database_type>://<database_host>:<database_port>/<database_name>
Fedora currently supports H2, PostgresQL 12.3, MariaDB 10.5.3, and MySQL 8.0
So using the default ports for the supported databases here are the values we typically use:
jdbc:postgresql://localhost:5432/fcrepojdbc:mariadb://localhost:3306/fcrepojdbc:mysql://localhost:3306/fcrepoFCREPO_DB_USERNAME - The database username
FCREPO_DB_PASSWORD - The database passwordFCREPO_OCFL_ROOT - Sets the root directory of the OCFL. Defaults to FCREPO_HOME/data/ocfl-root if not set.FCREPO_TEMP_ROOT - Sets the temp directory used by OCFL. Defaults to FCREPO_HOME/data/temp if not set.FCREPO_STAGING_ROOT - Sets the staging directory used by OCFL. Defaults to FCREPO_HOME/data/staging if not set.FCREPO_VELOCITY_LOG - The Fedora HTML template code uses Apache Velocity, which generates a runtime log called velocity.log. Defaults to FCREPO_HOME/logs/velocity. A good choice might be /opt/tomcat/logs/velocity.logFCREPO_JMS_BASE - This specifies the baseUrl to use when generating JMS messages. You can specify the hostname with or without port and with or without path. If your system is behind a NAT firewall you may need this to avoid your message consumers trying to access the system on an invalid port. If this system property is not set, the host, port and context from the user's request will be used in the emitted JMS messages. If your Alpaca is on the same machine as your Fedora and you use the islandora-indexing-fcrepo, you could use http://localhost:8080/fcrepo/rest. Check the Lyrasis Wiki to find all of Fedora's properties
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#adding-the-fedora-variables-to-java_opts","title":"Adding the Fedora Variables toJAVA_OPTS","text":"We need our Tomcat JAVA_OPTS to include references to our repository configuration.
/opt/tomcat/bin/setenv.sh
Before:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -server -Xmx1500m -Xms1000m\"
After:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -server -Xmx1500m -Xms1000m\"
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#ensuring-tomcat-users-are-in-place","title":"Ensuring Tomcat Users Are In Place","text":"While not strictly necessary, we can use the tomcat-users.xml file to give us direct access to the Fedora endpoint. Fedora defines, out of the box, a fedoraAdmin and fedoraUser role that can be reflected in the users list for access. The following file will also include the base tomcat user. As always, these default passwords should likely not stay as the defaults.
/opt/tomcat/conf/tomcat-users.xml | tomcat:tomcat/600
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<tomcat-users xmlns=\"http://tomcat.apache.org/xml\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxsi:schemaLocation=\"http://tomcat.apache.org/xml tomcat-users.xsd\"\nversion=\"1.0\">\n<role rolename=\"tomcat\"/>\n<role rolename=\"fedoraAdmin\"/>\n<role rolename=\"fedoraUser\"/>\n<user username=\"tomcat\" password=\"TOMCAT_PASSWORD\" roles=\"tomcat\"/>\n<user username=\"fedoraAdmin\" password=\"FEDORA_ADMIN_PASSWORD\" roles=\"fedoraAdmin\"/>\n<user username=\"fedoraUser\" password=\"FEDORA_USER_PASSWORD\" roles=\"fedoraUser\"/>\n</tomcat-users>\nTOMCAT_PASSWORD: tomcatFEDORA_ADMIN_PASSWORD: islandoraFEDORA_USER_PASSWORD: islandoraFedora .war files are packaged up as releases on the official GitHub repository. You should download the most recent stable release.
sudo wget -O fcrepo.war FCREPO_WAR_URL\nsudo mv fcrepo.war /opt/tomcat/webapps\nsudo chown tomcat:tomcat /opt/tomcat/webapps/fcrepo.war\nFCREPO_WAR_URL: This can be found at the fcrepo downloads page; the file you're looking for is:As before, start the Tomcat service to get Fedora up and running.
sudo systemctl start tomcat\nNote: sometimes it takes a while for Fedora and Tomcat to start up, usually it shouldn't take longer than 5 minutes.
Once it starts up, Fedora REST API should be available at http://localhost:8080/fcrepo/rest. The username is fedoraAdmin and we defined the password before as FEDORA_ADMIN_PASSWORD (default: \"islandora\").
A compiled JAR of Syn can be found on the Syn releases page. We\u2019re going to add this to the list libraries accessible to Tomcat.
sudo wget -P /opt/tomcat/lib SYN_JAR_URL\n# Ensure the library has the correct permissions.\nsudo chown -R tomcat:tomcat /opt/tomcat/lib\nsudo chmod -R 640 /opt/tomcat/lib\nSYN_JAR_URL: The latest stable release of the Syn JAR from the releases page. Specifically, the JAR compiled as -all.jar is required.For Islandora and Fedora to talk to each other, an SSL key needs to be generated for use with Syn. We\u2019re going to make a spot where such keys can live, and generate one.
sudo mkdir /opt/keys\nsudo openssl genrsa -out \"/opt/keys/syn_private.key\" 2048\nsudo openssl rsa -pubout -in \"/opt/keys/syn_private.key\" -out \"/opt/keys/syn_public.key\"\nsudo chown www-data:www-data /opt/keys/syn*\nSyn sites and tokens belong in a settings file that we\u2019re going to reference in Tomcat.
/opt/fcrepo/config/syn-settings.xml | tomcat:tomcat/600
<config version='1' header='X-Islandora'>\n<site algorithm='RS256' encoding='PEM' anonymous='true' default='true' path='/opt/keys/syn_public.key'/>\n<token user='islandora' roles='fedoraAdmin'>ISLANDORA_SYN_TOKEN</token>\n</config>\nISLANDORA_SYN_TOKEN: islandoraReferencing the valve we\u2019ve created in our syn-settings.xml involves creating a <Valve> entry in Tomcat\u2019s context.xml:
There are two options here:
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#1-enable-the-syn-valve-for-all-of-tomcat","title":"1. Enable the Syn Valve for all of Tomcat.","text":"/opt/tomcat/conf/context.xml
Before:
29 | -->
30 | </Context>
After:
29 | -->
30 | <Valve className=\"ca.islandora.syn.valve.SynValve\" pathname=\"/opt/fcrepo/config/syn-settings.xml\"/>
31 | </Context>
Create a new file at
/opt/tomcat/conf/Catalina/localhost/fcrepo.xml
<Context>\n<Valve className=\"ca.islandora.syn.valve.SynValve\" pathname=\"/opt/fcrepo/config/syn-settings.xml\"/>\n</Context>\nYour Fedora web application needs to be deployed in Tomcat with the name fcrepo.war. Otherwise, change the name of the above XML file to match the deployed web application's name.
Finally, restart tomcat to apply the new configurations.
sudo systemctl restart tomcat\nNote: sometimes it takes a while for Fedora and Tomcat to start up, usually it shouldn't take longer than 5 minutes.
Note: after installing the Syn valve, you'll no longer be able to manually create/edit or delete objects via Fedora Web UI. All communication with Fedora will now be handled from the Islandora module in Drupal.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#redhat-logging","title":"Redhat logging","text":"Redhat systems have stopped generating an all inclusive catalina.out, the catalina.<date>.log does not include web application's log statements. To get Fedora log statements flowing, you can create your own LogBack configuration file and point to it.
/opt/fcrepo/config/fcrepo-logback.xml | tomcat:tomcat/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE configuration>\n<configuration>\n<appender name=\"STDOUT\" class=\"ch.qos.logback.core.ConsoleAppender\">\n<encoder>\n<pattern>%p %d{HH:mm:ss.SSS} [%thread] \\(%c{0}\\) %m%n</pattern>\n</encoder>\n</appender>\n<appender name=\"FILE\" class=\"ch.qos.logback.core.rolling.RollingFileAppender\">\n<file>${catalina.base}/logs/fcrepo.log</file>\n<append>true</append>\n<rollingPolicy class=\"ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy\">\n<fileNamePattern>${catalina.base}/logs/fcrepo.%d{yyyy-MM-dd}.log.%i</fileNamePattern>\n<maxFileSize>10MB</maxFileSize>\n<maxHistory>30</maxHistory>\n<totalSizeCap>2GB</totalSizeCap>\n</rollingPolicy>\n<encoder>\n<pattern>%p %d{HH:mm:ss.SSS} [%thread] \\(%c{0}\\) %m%n</pattern>\n</encoder>\n</appender>\n<logger name=\"org.fcrepo.auth\" additivity=\"false\" level=\"${fcrepo.log.auth:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.config\" additivity=\"false\" level=\"${fcrepo.log.config:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.event\" additivity=\"false\" level=\"${fcrepo.log.event:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.http.api\" additivity=\"false\" level=\"${fcrepo.log.http.api:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.http.commons\" additivity=\"false\" level=\"${fcrepo.log.http.commons:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.jms\" additivity=\"false\" level=\"${fcrepo.log.jms:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.kernel\" additivity=\"false\" level=\"${fcrepo.log.kernel:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.persistence\" additivity=\"false\" level=\"${fcrepo.log.persistence:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.search\" additivity=\"false\" level=\"${fcrepo.log.search:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo.storage\" additivity=\"false\" level=\"${fcrepo.log.storage:-null}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<logger name=\"org.fcrepo\" additivity=\"false\" level=\"${fcrepo.log:-INFO}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</logger>\n<root level=\"${fcrepo.log.root:-WARN}\">\n<appender-ref ref=\"STDOUT\"/>\n<appender-ref ref=\"FILE\"/>\n</root>\n</configuration>\nThen alter your $JAVA_OPTS like above to include
-Dlogback.configurationFile=/opt/fcrepo/config/fcrepo-logback.xml\nThis will generate a log file at ${catalina.base}/logs/fcrepo.log and will rotate each day or if the logs reaches 10MB. It will maintain 30 days of old logs, or 2GB whichever comes first.
Blazegraph needs a space for configurations and data; we\u2019re going to create this space in /opt.
sudo mkdir -p /opt/blazegraph/data\nsudo mkdir /opt/blazegraph/conf\nsudo chown -R tomcat:tomcat /opt/blazegraph\nThe Blazegraph .war file can be found in a few different places, but to ensure we\u2019re able to easily wget it, we\u2019re going to use the maven.org repository link to grab it.
cd /opt\nsudo wget -O blazegraph.war BLAZEGRAPH_WARFILE_LINK\nsudo mv blazegraph.war /opt/tomcat/webapps\nsudo chown tomcat:tomcat /opt/tomcat/webapps/blazegraph.war\n.war file within that version folder.Once this is downloaded, give it a moment to expand before moving on to the next step.
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#configuring-logging","title":"Configuring Logging","text":"We would like to have an appropriate logging configuration for Blazegraph, which can be useful for looking at incoming traffic and determining if anything has gone wrong with Blazegraph. Our logger isn\u2019t going to be much different than the default logger; it can be made more or less verbose by changing the default WARN levels. There are several other loggers that can be enabled, like a SPARQL query trace or summary query evaluation log; if these are desired they should be added in. Consult the Blazegraph documentation for more details.
/opt/blazegraph/conf/log4j.properties | tomcat:tomcat/644
log4j.rootCategory=WARN, dest1\n\n# Loggers.\nlog4j.logger.com.bigdata=WARN\nlog4j.logger.com.bigdata.btree=WARN\n\n# Normal data loader (single threaded).\n#log4j.logger.com.bigdata.rdf.store.DataLoader=INFO\n\n# dest1\nlog4j.appender.dest1=org.apache.log4j.ConsoleAppender\nlog4j.appender.dest1.layout=org.apache.log4j.PatternLayout\nlog4j.appender.dest1.layout.ConversionPattern=%-5p: %F:%L: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-5p: %r %l: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-5p: %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n\n#log4j.appender.dest1.layout.ConversionPattern=%-4r(%d) [%t] %-5p %c(%l:%M) %x - %m%n\n\n# Rule execution log. This is a formatted log file (comma delimited).\nlog4j.logger.com.bigdata.relation.rule.eval.RuleLog=INFO,ruleLog\nlog4j.additivity.com.bigdata.relation.rule.eval.RuleLog=false\nlog4j.appender.ruleLog=org.apache.log4j.FileAppender\nlog4j.appender.ruleLog.Threshold=ALL\nlog4j.appender.ruleLog.File=/var/log/blazegraph/rules.log\nlog4j.appender.ruleLog.Append=true\nlog4j.appender.ruleLog.BufferedIO=false\nlog4j.appender.ruleLog.layout=org.apache.log4j.PatternLayout\nlog4j.appender.ruleLog.layout.ConversionPattern=%m\nOur configuration will be built from a few different files that we will eventually reference in JAVA_OPTS and directly apply to Blazegraph; these include most of the functional pieces Blazegraph requires, as well as a generalized configuration for the islandora namespace it will use. As with most large configurations like this, these should likely be tuned to your preferences, and the following files only represent sensible defaults.
/opt/blazegraph/conf/RWStore.properties | tomcat:tomcat/644
com.bigdata.journal.AbstractJournal.file=/opt/blazegraph/data/blazegraph.jnl\ncom.bigdata.journal.AbstractJournal.bufferMode=DiskRW\ncom.bigdata.service.AbstractTransactionService.minReleaseAge=1\ncom.bigdata.journal.Journal.groupCommit=false\ncom.bigdata.btree.writeRetentionQueue.capacity=4000\ncom.bigdata.btree.BTree.branchingFactor=128\ncom.bigdata.journal.AbstractJournal.initialExtent=209715200\ncom.bigdata.journal.AbstractJournal.maximumExtent=209715200\ncom.bigdata.rdf.sail.truthMaintenance=false\ncom.bigdata.rdf.store.AbstractTripleStore.quads=true\ncom.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false\ncom.bigdata.rdf.store.AbstractTripleStore.textIndex=false\ncom.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms\ncom.bigdata.namespace.kb.lex.com.bigdata.btree.BTree.branchingFactor=400\ncom.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=1024\ncom.bigdata.journal.Journal.collectPlatformStatistics=false\n/opt/blazegraph/conf/blazegraph.properties | tomcat:tomcat/644
com.bigdata.rdf.store.AbstractTripleStore.textIndex=false\ncom.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms\ncom.bigdata.rdf.sail.isolatableIndices=false\ncom.bigdata.rdf.store.AbstractTripleStore.justify=true\ncom.bigdata.rdf.sail.truthMaintenance=true\ncom.bigdata.rdf.sail.namespace=islandora\ncom.bigdata.rdf.store.AbstractTripleStore.quads=false\ncom.bigdata.namespace.islandora.lex.com.bigdata.btree.BTree.branchingFactor=400\ncom.bigdata.journal.Journal.groupCommit=false\ncom.bigdata.namespace.islandora.spo.com.bigdata.btree.BTree.branchingFactor=1024\ncom.bigdata.rdf.store.AbstractTripleStore.geoSpatial=false\ncom.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false\n/opt/blazegraph/conf/inference.nt | tomcat:tomcat/644
<http://pcdm.org/models#memberOf> <http://www.w3.org/2002/07/owl#inverseOf> <http://pcdm.org/models#hasMember> .\n<http://pcdm.org/models#fileOf> <http://www.w3.org/2002/07/owl#inverseOf> <http://pcdm.org/models#hasFile> .\nRWStore.properties in JAVA_OPTS","text":"In order to enable our configuration when Tomcat starts, we need to reference the location of RWStore.properties in the JAVA_OPTS environment variable that Tomcat uses.
/opt/tomcat/bin/setenv.sh
Before:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -server -Xmx1500m -Xms1000m\"
After:
3 | export JAVA_OPTS=\"-Djava.awt.headless=true -Dfcrepo.config.file=/opt/fcrepo/config/fcrepo.properties -DconnectionTimeout=-1 -Dcom.bigdata.rdf.sail.webapp.ConfigParams.propertyFile=/opt/blazegraph/conf/RWStore.properties -Dlog4j.configuration=file:/opt/blazegraph/conf/log4j.properties -server -Xmx1500m -Xms1000m\"
"},{"location":"installation/manual/installing-fedora-syn-and-blazegraph/#restarting-tomcat_1","title":"Restarting Tomcat","text":"Finally, restart Tomcat to pick up the changes we\u2019ve made.
sudo systemctl restart tomcat\nThe two other files we created, blazegraph.properties and inference.nt, contain information that Blazegraph requires in order to establish and correctly use the datasets Islandora will send to it. First, we need to create a dataset - contained in blazegraph.properties - and then we need to inform that dataset of the inference set we have contained in inference.nt.
curl -X POST -H \"Content-Type: text/plain\" --data-binary @/opt/blazegraph/conf/blazegraph.properties http://localhost:8080/blazegraph/namespace\ncurl -X POST -H \"Content-Type: text/plain\" --data-binary @/opt/blazegraph/conf/inference.nt http://localhost:8080/blazegraph/namespace/islandora/sparql\nNeeds Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#in-this-section-we-will-install","title":"In this section, we will install:","text":"In our case, the default installation method for ActiveMQ via apt-get will suffice.
sudo apt-get -y install activemq\nThis will give us:
/var/lib/activemq/conf/var/lib/activemq/data/usr/share/activemqactivemq service that will be run on bootactivemq, who will be in charge of the ActiveMQ serviceTake note of the version of ActiveMQ we're going to be installing. It needs to match a Karaf blueprint we'll create later. Check the version with
sudo apt-cache policy activemq\nWrite down the version listed under Installed:.
Karaf, as well as its processes and service, will be owned by a user in charge of ensuring this portion of the stack is segregated and that the service is running.
sudo addgroup karaf\nsudo adduser karaf --ingroup karaf --home /opt/karaf --shell /usr/bin\nAs always, you will be prompted for a password, which you should create at this time. All other options can be left blank.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#downloading-and-placing-karaf","title":"Downloading and Placing Karaf","text":"Since there\u2019s no apt-get installer for Karaf, we\u2019re going to manually download and install it directly from its binary installer.
cd /opt\nsudo wget -O karaf.tar.gz KARAF_TARBALL_LINK\nsudo tar -xzvf karaf.tar.gz\nsudo chown -R karaf:karaf KARAF_DIRECTORY\nsudo mv KARAF_DIRECTORY/* /opt/karaf\nKARAF_TARBALL_LINK: It\u2019s recommended to get the most recent version of Karaf 4.2.x. This will depend on the current version of Karaf, which can be found on the Karaf downloads page under \u201cKaraf Runtime\u201d. Like Solr, you can\u2019t directly wget these links, but clicking on the .tar.gz link for the binary distribution will bring you to a list of mirrors, as well as provide you with a recommended mirror you can use here. - KARAF_DIRECTORY: This will depend on the exact version being used, but will likely be /opt/apache-karaf-VERSION, where VERSION is the current Karaf version number."},{"location":"installation/manual/installing-karaf-and-alpaca/#configuring-karaf-logging","title":"Configuring Karaf Logging","text":"We\u2019re going to apply some basic logging to our Karaf installation that should suffice for an example. In a production installation, you may want to play around with some of these values for more personally useful logging.
sudo mkdir /var/log/karaf\nsudo chown karaf:karaf /var/log/karaf\n/opt/karaf/etc/org.pos4j.pax.logging.cfg | karaf:karaf/644
# Root logger\nlog4j.rootLogger=INFO, out, osgi:*\nlog4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer\n\n# File appender\nlog4j.appender.out=org.apache.log4j.RollingFileAppender\nlog4j.appender.out.layout=org.apache.log4j.PatternLayout\nlog4j.appender.out.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name } - %X{bundle.version} | %m%n\nlog4j.appender.out.file=/var/log/karaf/karaf.log\nlog4j.appender.out.append=true\nlog4j.appender.out.maxFileSize=1MB\nlog4j.appender.out.maxBackupIndex=10\n\n# Camel Logger\nlog4j.appender.camel=org.apache.log4j.RollingFileAppender\nlog4j.appender.camel.layout=org.apache.log4j.PatternLayout\nlog4j.appender.camel.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.na me} - %X{bundle.version} | %m%n\nlog4j.appender.camel.file=/var/log/karaf/camel.log\nlog4j.appender.camel.append=false\nlog4j.appender.camel.maxFileSize=1MB\nlog4j.appender.camel.maxBackupIndex=10\n\nlog4j.logger.org.apache.camel=INFO, camel\n\n# Islandora Logger\nlog4j.appender.islandora=org.apache.log4j.RollingFileAppender\nlog4j.appender.islandora.layout=org.apache.log4j.PatternLayout\nlog4j.appender.islandora.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundl e.name} - %X{bundle.version} | %m%n\nlog4j.appender.islandora.file=/var/log/karaf/islandora.log\nlog4j.appender.islandora.append=false\nlog4j.appender.islandora.maxFileSize=1MB\nlog4j.appender.islandora.maxBackupIndex=10\n\nlog4j.logger.ca.islandora.camel=INFO, islandora\nsetenv.sh Script for Karaf","text":"Similar to Tomcat, our Karaf service is going to rely on a setenv shell script to determine environment variables Karaf needs in place when running. For now, this will simply be the path to JAVA_HOME, but this also accepts many other parameters you can find in the default setenv script.
/opt/karaf/bin/setenv | karaf:karaf/755
#!/bin/sh\nexport JAVA_HOME=\"PATH_TO_JAVA_HOME\"\nPATH_TO_JAVA_HOME: This will be the same JAVA_HOME we used when installing Tomcat , and can be found using the same method (i.e., still /usr/lib/jvm/java-11-openjdk-amd64 if that's what it was before)."},{"location":"installation/manual/installing-karaf-and-alpaca/#initializing-karaf","title":"Initializing Karaf","text":"We\u2019re going to start Karaf, then run the installer to put our configurations in place and generate a Karaf service. Once these are installed, we\u2019re going to stop Karaf, as from there on out its start/stop management should be handled via that service.
First we need to enable the default Karaf user in /opt/karaf/etc/users.properties:
Before:
32 | # karaf = karaf,_g_:admingroup
33 | # _g_\\:admingroup = group,admin,manager,viewer,systembundles,ssh
After:
32 | karaf = karaf,_g_:admingroup
33 | _g_\\:admingroup = group,admin,manager,viewer,systembundles,ssh
Save the file and close it, then:
sudo -u karaf /opt/karaf/bin/start\n# You may want to wait a bit for Karaf to start.\n# If you're not sure whether or not it's running, you can always run:\n# ps aux | grep karaf\n# to see if the server is up and running.\n/opt/karaf/bin/client feature:install wrapper\n/opt/karaf/bin/client wrapper:install\n/opt/karaf/bin/stop\nInstalling the Karaf wrapper generates several service files that can be used on different types of systems. For Debian and Ubuntu installation we want to enable the karaf.service service so that Karaf is properly started on boot.
sudo systemctl enable /opt/karaf/bin/karaf.service\nsudo systemctl start karaf\nWe can check if the service started correctly with:
sudo systemctl status karaf\nPress Q to close the status.
"},{"location":"installation/manual/installing-karaf-and-alpaca/#alpaca-10x","title":"Alpaca 1.0.x","text":""},{"location":"installation/manual/installing-karaf-and-alpaca/#adding-the-required-karaf-repositories","title":"Adding the Required Karaf Repositories","text":"Karaf features can be installed from several different types of sources, but the fastest and easiest way to do so is from existing repository URLs that we can just plug into Karaf to provide us feature lists prepared and ready for installation. Like most interactions with Karaf, we can add these repositories using its built-in client.
Notice
These repositories are updated consistently, and their updates include revised dependency lists. Commonly, when repositories are out of date or otherwise mismatched, feature installation can result in an Unable to resolve root: missing requirement error; for this reason, this guide recommends using recently-updated versions of these repositories. That being said, if such errors occur despite installing the latest versions of these features, the maintainer of the features repository should be informed.
For the Karaf features we\u2019re going to install, we need a few different repositories to be added to the list:
/opt/karaf/bin/client repo-add mvn:org.apache.activemq/activemq-karaf/ACTIVEMQ_KARAF_VERSION/xml/features\n/opt/karaf/bin/client repo-add mvn:org.apache.camel.karaf/apache-camel/APACHE_CAMEL_VERSION/xml/features\n/opt/karaf/bin/client repo-add mvn:ca.islandora.alpaca/islandora-karaf/ISLANDORA_KARAF_VERSION/xml/features\n# XXX: This shouldn't be strictly necessary, but appears to be a missing\n# upstream dependency for some fcrepo features.\n/opt/karaf/bin/client repo-add mvn:org.apache.jena/jena-osgi-features/JENA_OSGI_VERSION/xml/features\nACTIVEMQ_KARAF_VERSION: The version of ActiveMQ we wrote down at the beginning of this chapter when installing ActiveMQ via apt-get - APACHE_CAMEL_VERSION: The latest version of Apache Camel 2.x.x; you can find this listed at the apache-camel repository page (e.g., 2.25.4 at the time of writing) - ISLANDORA_KARAF_VERSION: The latest version of Islandora Karaf 1.x; you can find this listed at the islandora-karaf repository page (e.g., 1.0.5 at the time of writing) - JENA_OSGI_VERSION: The latest version of the Apache Jena 3.x OSGi features; you can find this listed at the jena-osgi-features repository page (e.g., 3.17.0 at the time of writing)"},{"location":"installation/manual/installing-karaf-and-alpaca/#configuring-karaf-features","title":"Configuring Karaf Features","text":"Our installed Karaf features require configuration files to know exactly where to route things coming and going from them.
/opt/karaf/etc/ca.islandora.alpaca.http.client.cfg | karaf:karaf/644
token.value=ISLANDORA_SYN_TOKEN\nISLANDORA_SYN_TOKEN: This should be the same token that was established during the installation of Syn in your syn-settings.xml file /opt/karaf/etc/org.fcrepo.camel.indexing.triplestore.cfg | karaf:karaf/644
input.stream=activemq:topic:fedora\ntriplestore.reindex.stream=activemq:queue:triplestore.reindex\ntriplestore.baseUrl=http://localhost:8080/blazegraph/namespace/islandora/sparql\n/opt/karaf/etc/ca.islandora.alpaca.indexing.triplestore.cfg | karaf:karaf/644
error.maxRedeliveries=10\nindex.stream=activemq:queue:islandora-indexing-triplestore-index\ndelete.stream=activemq:queue:islandora-indexing-triplestore-delete\ntriplestore.baseUrl=http://localhost:8080/blazegraph/namespace/islandora/sparql\n/opt/karaf/etc/ca.islandora.alpaca.indexing.fcrepo.cfg | karaf:karaf/644
error.maxRedeliveries=5\nnode.stream=activemq:queue:islandora-indexing-fcrepo-content\nnode.delete.stream=activemq:queue:islandora-indexing-fcrepo-delete\nmedia.stream=activemq:queue:islandora-indexing-fcrepo-media\nfile.stream=activemq:queue:islandora-indexing-fcrepo-file\nfile.delete.stream=activemq:queue:islandora-indexing-fcrepo-file-delete\nmilliner.baseUrl=http://localhost/milliner\nFor those services in Crayfish we have set up to provide derivatives to Islandora resources, we need connector blueprints to tell the derivative connector how to route incoming requests, run conversions, and return outgoing derivatives.
Our blueprints are going to look largely similar between services, with only a few properties changing between them. Largely, these mainly just need to match the ActiveMQ queues we established in the previous configuration, and route to the correct Crayfish service.
/opt/karaf/deploy/ca.islandora.alpaca.connector.ocr.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.ocr\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-ocr\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/hypercube\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorOCR\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.houdini.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.houdini\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-houdini\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/houdini/convert\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorHoudini\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.homarus.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.homarus\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-homarus\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/homarus/convert\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorHomarus\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\n/opt/karaf/deploy/ca.islandora.alpaca.connector.fits.blueprint.xml | karaf:karaf/644
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<blueprint xmlns=\"http://www.osgi.org/xmlns/blueprint/v1.0.0\"\nxmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\nxmlns:cm=\"http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0\"\nxsi:schemaLocation=\"\n http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd\n http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd\n http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd\">\n<cm:property-placeholder id=\"properties\" persistent-id=\"ca.islandora.alpaca.connector.fits\" update-strategy=\"reload\" >\n<cm:default-properties>\n<cm:property name=\"error.maxRedeliveries\" value=\"5\"/>\n<cm:property name=\"in.stream\" value=\"activemq:queue:islandora-connector-fits\"/>\n<cm:property name=\"derivative.service.url\" value=\"http://localhost/crayfits\"/>\n</cm:default-properties>\n</cm:property-placeholder>\n<reference id=\"broker\" interface=\"org.apache.camel.Component\" filter=\"(osgi.jndi.service.name=fcrepo/Broker)\"/>\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<camelContext id=\"IslandoraConnectorfits\" xmlns=\"http://camel.apache.org/schema/blueprint\">\n<package>ca.islandora.alpaca.connector.derivative</package>\n</camelContext>\n</blueprint>\nBefore we can configure the features we\u2019re going to use, they need to be installed. Some of these installations may take some time.
/opt/karaf/bin/client feature:install camel-blueprint\n/opt/karaf/bin/client feature:install activemq-blueprint\n/opt/karaf/bin/client feature:install fcrepo-service-activemq\n# This again should not be strictly necessary, since this isn't the triplestore\n# we're using, but is being included here to resolve the aforementioned\n# missing link in the dependency chain.\n/opt/karaf/bin/client feature:install jena\n/opt/karaf/bin/client feature:install fcrepo-camel\n/opt/karaf/bin/client feature:install fcrepo-indexing-triplestore\n/opt/karaf/bin/client feature:install islandora-http-client\n/opt/karaf/bin/client feature:install islandora-indexing-triplestore\n/opt/karaf/bin/client feature:install islandora-indexing-fcrepo\n/opt/karaf/bin/client feature:install islandora-connector-derivative\nAt this point, Karaf components should be up and running, but it's a good idea to double-check that this is the case. We can do this from within the Karaf client by taking a look at its component list.
# Until this point, we've been running Karaf commands from outside; we can hop\n# into the client, however, and run commands from directly within.\n/opt/karaf/bin/client\n# This takes us into the Karaf client so we can run commands.\nla | grep islandora\nla | grep fcrepo\n# It may be a good idea to use this to look up to the other components we\n# installed.\nlogout\nFor the above la | grep commands, components that are running should be listed as Active.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-solr/#in-this-section-we-will-install","title":"In this section, we will install:","text":"The Solr binaries can be found at the Solr downloads page; the most recent stable release of Solr 8 should be used.
# While generally we download tarballs as .tar.gz files without version\n# information, the Solr installer is a bit particular in that it expects a .tgz\n# file with the same name as the extracted folder it contains. It's odd, and we\n# can't really get around it.\ncd\nwget SOLR_DOWNLOAD_LINK\ntar -xzvf SOLR_TARBALL\nSOLR_DOWNLOAD_LINK: NOTICE: This will depend on a few different things, not least of all the current version of Solr. The link to the .tgz for the binary on the downloads page will take you to a list of mirrors that Solr can be downloaded from, and provide you with a preferred mirror at the top. This preferred mirror should be used as the SOLR_DOWNLOAD_LINK. - SOLR_TARBALL: The filename that was downloaded, e.g., solr-8.9.0.tgz"},{"location":"installation/manual/installing-solr/#running-the-solr-installer","title":"Running the Solr Installer","text":"Solr includes an installer that does most of the heavy lifting of ensuring we have a Solr user, a location where Solr lives, and configurations in place to ensure it\u2019s running on boot.
sudo UNTARRED_SOLR_FOLDER/bin/install_solr_service.sh SOLR_TARBALL\nUNTARRED_SOLR_FOLDER: This will likely simply be solr-VERSION, where VERSION is the version number that was downloaded. The port that Solr runs on can potentially be configured at this point, but we'll expect it to be running on 8983.
Wait until the command output reaches:
Started Solr server on port 8983 (pid=****). Happy searching!\nsystemd[1]: Started LSB: Controls Apache Solr as a Service.\nAfter which you can press q to quit the output (this won't kill Solr so it's safe).
You can check if Solr is running correctly by going to http://localhost:8983/solr
"},{"location":"installation/manual/installing-solr/#increasing-the-open-file-limit-optional","title":"Increasing the Open File Limit (Optional)","text":"Solr's installation guide recommends that you increase the open file limit so that operations aren't disrupted while Solr is trying to access things in its index. This limit can be increased while the system is running, but doing so won't persist after a reboot. You can hard-increase this limit using your system's sysctl file:
/etc/sysctl.conf
Add the following line to the end of the file:
fs.file-max = 65535\nThen apply your new configuration.
sudo sysctl -p\nInitially, our new Solr core will contain a configuration copied from the example included with the installation, so that we have something to work with when we configure this on the Drupal side. We\u2019ll later update this with generated configurations we create in Drupal.
cd /opt/solr\nsudo mkdir -p /var/solr/data/SOLR_CORE/conf\nsudo cp -r example/files/conf/* /var/solr/data/SOLR_CORE/conf\nsudo chown -R solr:solr /var/solr\nsudo -u solr bin/solr create -c SOLR_CORE -p 8983\nSOLR_CORE: islandora8 You should see an output similar to this:
WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.\n To turn off: bin/solr config -c islandora8 -p 8983 -action set-user-property -property update.autoCreateFields -value false\n\nCreated new core 'islandora8'\nsearch_api_solr","text":"Rather than use an out-of-the-box configuration that won\u2019t be suitable for our purposes, we\u2019re going to use the Drupal search_api_solr module to generate one for us. This will also require us to install the module so we can create these configurations using Drush.
cd /opt/drupal\nsudo -u www-data composer require drupal/search_api_solr:^4.2\ndrush -y en search_api_solr\nYou should see an output similar to this:
The following module(s) will be enabled: search_api_solr, language, search_api\n\n // Do you want to continue?: yes.\n\n [success] Successfully enabled: search_api_solr, language, search_api\nBefore we can create configurations to use with Solr, the core we created earlier needs to be referenced in Drupal.
Log in to the Drupal site at /user using the sitewide administrator username and password (if using defaults from previous chapters this should be islandora and islandora), then navigate to /admin/config/search/search-api/add-server.
Fill out the server addition form using the following options:
SERVER_NAME: islandora8machine_name generated next to the server name you type in; this will be used in the next step.As a recap for this configuration:
Click Save to create the server configuration.
NOTICE You can ignore the error about an incompatible Solr schema; we're going to set this up in the next step. In fact, if you refresh the page after restarting Solr in the next step, you should see the error disappear.
"},{"location":"installation/manual/installing-solr/#generating-and-applying-solr-configurations","title":"Generating and Applying Solr Configurations","text":"Now that our core is in place and our Drupal-side configurations exist, we\u2019re ready to generate Solr configuration files to connect this site to our search engine.
cd /opt/drupal\ndrush solr-gsc SERVER_MACHINE_NAME /opt/drupal/solrconfig.zip\nunzip -d ~/solrconfig solrconfig.zip\nsudo cp ~/solrconfig/* /var/solr/data/SOLR_CORE/conf\nsudo systemctl restart solr\nSERVER_MACHINE_NAME: This should be the machine_name that was automatically generated when creating the configuration in the above step."},{"location":"installation/manual/installing-solr/#adding-an-index","title":"Adding an Index","text":"In order for content to be indexed back into Solr, a search index needs to be added to our server. Navigate to /admin/config/search/search-api/add-index and check off the things you'd like to be indexed.
NOTICE You should come back here later and reconfigure this after completing the last step in this guide. The default indexing configuration is pretty permissive, and you may want to restrict, for example, indexed content to just Islandora-centric bundles. This guide doesn't set up the index's fields either, which are going to be almost wholly dependent on the needs of your installation. Once you complete that configuration later on, re-index Solr from the configuration page of the index we're creating here.
Click Save to add your index and kick off indexing of existing items.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/","title":"Installing Tomcat and Cantaloupe","text":"Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Tomcat runs in a Java runtime environment, so we'll need one to continue. In our case, OpenJDK 11 is open-source, free to use, and can fairly simply be installed using apt-get:
sudo apt-get -y install openjdk-11-jdk openjdk-11-jre\nThe installation of OpenJDK via apt-get establishes it as the de-facto Java runtime environment to be used on the system, so no further configuration is required.
The resultant location of the java JRE binary (and therefore, the correct value of JAVA_HOME when it\u2019s referenced) will vary based on the specifics of the machine it\u2019s being installed on; that being said, you can find its exact location using update-alternatives:
update-alternatives --list java\ntomcat User","text":"Apache Tomcat, and all its processes, will be owned and managed by a specific user for the purposes of keeping parts of the stack segregated and accountable.
sudo addgroup tomcat\nsudo adduser tomcat --ingroup tomcat --home /opt/tomcat --shell /usr/bin\nYou will be prompted to create a password for the tomcat user; all the other information as part of the adduser command can be ignored.
Tomcat 9 itself can be installed in several different ways; while it\u2019s possible to install via apt-get, this doesn\u2019t give us a great deal of control over exactly how we\u2019re going to run and manage it; as a critical part of the stack, it is beneficial for our purposes to have a good frame of reference for the inner workings of Tomcat.
We\u2019re going to download the latest version of Tomcat to /opt and set it up so that it runs automatically. Bear in mind that with the following commands, this is going to be entirely relative to the current version of Tomcat 9, which we\u2019ll try to mitigate as we go.
cd /opt\nsudo wget -O tomcat.tar.gz TOMCAT_TARBALL_LINK\nsudo tar -zxvf tomcat.tar.gz\nsudo mv /opt/TOMCAT_DIRECTORY/* /opt/tomcat\nsudo chown -R tomcat:tomcat /opt/tomcat\nTOMCAT_TARBALL_LINK: No default can be provided here; you should navigate to the Tomcat 9 downloads page and grab the link to the latest .tar.gz file under the \u201cCore\u201d section of \u201cBinary Distributions\u201d. It is highly recommended to grab the latest version of Tomcat 9, as it will come with associated security patches and fixes. - TOMCAT_DIRECTORY: This will also depend entirely on the exact version of tomcat downloaded - for example, apache-tomcat-9.0.50. Again, ls /opt can be used to find this."},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#creating-a-setenvsh-script","title":"Creating a setenv.sh Script","text":"When Tomcat runs, some configuration needs to be pre-established as a series of environment variables that will be used by the script that runs it.
/opt/tomcat/bin/setenv.sh | tomcat:tomcat/755
export CATALINA_HOME=\"/opt/tomcat\"\nexport JAVA_HOME=\"PATH_TO_JAVA_HOME\"\nexport JAVA_OPTS=\"-Djava.awt.headless=true -server -Xmx1500m -Xms1000m\"\nPATH_TO_JAVA_HOME: This will vary a bit depending on the environment, but will likely live in /usr/lib/jvm somewhere (e.g., /usr/lib/jvm/java-11-openjdk-amd64); again, in an Ubunutu environment you can check a part of this using update-alternatives --list java, which will give you the path to the JRE binary within the Java home. Note that update-alternatives --list java will give you the path to the binary, so for PATH_TO_JAVA_HOME delete the /bin/java at the end to get the Java home directory, so it should look something like this: export JAVA_HOME=\"/usr/lib/jvm/java-11-openjdk-amd64\"\nTomcat includes two shell scripts we\u2019re going to make use of - startup.sh and shutdown.sh - which are light wrappers on top of a third script, catalina.sh, which manages spinning up and shutting down the Tomcat server.
Debian and Ubuntu use systemctl to manage services; we\u2019re going to create a .service file that can run these shell scripts.
/etc/systemd/system/tomcat.service | root:root/755
[Unit]\nDescription=Tomcat\n\n[Service]\nType=forking\nExecStart=/opt/tomcat/bin/startup.sh\nExecStop=/opt/tomcat/bin/shutdown.sh\nSyslogIdentifier=tomcat\n\n[Install]\nWantedBy=multi-user.target\nWe\u2019re going to both enable and start Tomcat. Enabling Tomcat will ensure that it starts on boot, the timing of which is defined by the [Install] section\u2019s WantedBy statement, which specifies what it should start after. This is separate from starting it, which we need to do now in order to get Tomcat up and running without requiring a reboot.
sudo systemctl enable tomcat\nsudo systemctl start tomcat\nWe can check that Tomcat has started by running sudo systemctl status tomcat | grep Active; we should see that Tomcat is active (running), which is the correct result of startup.sh finishing its run successfully.
Since version 5, Cantaloupe is released as a standalone Java application and is no longer deployed in Tomcat via a .war file. Even so, we can still fine-tune how it runs and even install it as a service.
"},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#downloading-cantaloupe","title":"Downloading Cantaloupe","text":"Releases of Cantaloupe live on the Cantaloupe release page; the latest version can be found here as a .zip file.
sudo wget -O /opt/cantaloupe.zip CANTALOUPE_RELEASE_URL\nsudo unzip /opt/cantaloupe.zip\nCANTALOUPE_RELEASE_URL: It\u2019s recommended we grab the latest version of Cantaloupe 5. This can be found on the above-linked release page, as the .zip version; for example, https://github.com/cantaloupe-project/cantaloupe/releases/download/v5.0.3/cantaloupe-5.0.3.zip - make sure not to download the source code zip file as that isn't compiled for running out-of-the-box."},{"location":"installation/manual/installing-tomcat-and-cantaloupe/#creating-a-cantaloupe-configuration","title":"Creating a Cantaloupe Configuration","text":"Cantaloupe pulls its configuration from a file called cantaloupe.properties; there are also some other files that can contain instructions for Cantaloupe while it\u2019s running; specifically, we\u2019re going to copy over the delegates.rb file, which can also contain custom configuration. We won\u2019t make use of this file; we\u2019re just copying it over for demonstration purposes.
Creating these files from scratch is not recommended; rather, we\u2019re going to take the default cantaloupe configurations and plop them into their own folder so we can work with them.
sudo mkdir /opt/cantaloupe_config\nsudo cp CANTALOUPE_VER/cantaloupe.properties.sample /opt/cantaloupe_config/cantaloupe.properties\nsudo cp CANTALOUPE_VER/delegates.rb.sample /opt/cantaloupe_config/delegates.rb\nCANTALOUPE_VER: This will depend on the exact version of Cantaloupe downloaded; in the above example release, this would be cantaloupe-5.0.3 The out-of-the-box configuration will work fine for our purposes, but it\u2019s highly recommended that you take a look through the cantaloupe.properties and see what changes can be made; specifically, logging to actual logfiles isn\u2019t set up by default, so you may want to take a peek at the log.application.SyslogAppender or log.application.RollingFileAppender, as well as changing the logging level.
Since it is a standalone application, we can configure Cantaloupe as a systemd service like we did with Tomcat, so it can start on boot:
/etc/systemd/system/cantaloupe.service | root:root/755
[Unit]\nDescription=Cantaloupe\n\n[Service]\nExecStart=java -cp /opt/CANTALOUPE_VER/CANTALOUPE_VER.jar -Dcantaloupe.config=/opt/cantaloupe_config/cantaloupe.properties -Xmx1500m -Xms1000m edu.illinois.library.cantaloupe.StandaloneEntry\nSyslogIdentifier=cantaloupe\n\n[Install]\nWantedBy=multi-user.target\nCANTALOUPE_VER: This will depend on the exact version of Cantaloupe downloaded; in the above example release, this would be cantaloupe-5.0.3 We can now enable the service and run it:
sudo systemctl enable cantaloupe\nsudo systemctl start cantaloupe\nWe can check the service status with sudo systemctl status cantaloupe | grep Active and the splash screen of Cantaloupe should be available at http://localhost:8182
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
Notice
The manual installation guide is not intended to describe the Islandora installation but rather an Islandora installation. The server created using this guide is not hardened, will not be easily scalable, and the components may not be configured in a way you consider easy to work with. A production instance of Islandora should be installed and maintained by a professional with an understanding of Linux and server administration.
This guide will contain generalized steps on installation and configuration of the various components, but will contain specific example commands for executing these steps on an Ubuntu 18.04 Server.
"},{"location":"installation/manual/introduction/#some-prerequisite-knowledge","title":"Some Prerequisite Knowledge","text":"This guide assumes the user has some knowledge:
nano or vi/vim, or creating these files locally and uploading them to the server. It should also generally be assumed that most of these configuration files will have to be created or edited using sudo, and that permissions and ownership may need to be specified on these files.The steps in this guide are listed in chronological order of installation and configuration. Some sections will reference variables and rely on components installed in previous sections. It does not account for skipping over or otherwise changing the order of installation steps; bear this in mind if you decide to do things out of the provided order.
"},{"location":"installation/manual/introduction/#replacement-variables","title":"Replacement Variables","text":"It is expected that the person setting up the site may want to use different usernames, passwords, and other such variables than the ones presented by default in this guide. Additionally, some defaults can't be provided, such as up-to-date version information for externally-provided components. In such cases, the replacement variables will be placed in all capital letters, and a description of the variables, any possible defaults, and how to get up-to-date information, will be listed below.
"},{"location":"installation/manual/introduction/#bash-commands","title":"Bash Commands","text":"Notice
Command blocks are always assumed to start at the home folder of the user originally created during the server installation. They are never run as root; if root access is required, sudo will be specified, and if files are created belonging to root that should not belong to root, chmod and chown will be run against them immediately afterwards to ensure correct permissions. If commands need to be run from a different working directory, an absolute path will be specified to use with cd. If you're concerned about whether or not a code block can be run from your current working directory, run cd ~ before executing any commands in it.
Commands to be run on the command line will be placed in code blocks, with one command per line, and any replacement variables below, e.g.,
sudo run --this-command\npython3 run.py /this/other/command --with-param PARAMETER\nPARAMETER: some_sensible_default, perhaps with an explanation of why, or how to determine alternatives"},{"location":"installation/manual/introduction/#editing-files-in-place","title":"Editing Files In Place","text":"When an individual file needs to be modified in place (as opposed to replacing it outright), a Before and After quote will be provided that identifies one or more lines in the file, what the default installed version of that file looks like on that line, and what the line should look like after it has been modified, like so:
/path/to/file_being_modified
Before:
174 | Here is what line 174 in the file looked like before
175 | And here is what the following line looked like in the file before
After:
174 | Here is what line 174 should look like after modification
175 | And here is what the following line should look like after modification: VARIABLE
VARIABLE: some_value, perhaps with an explanation of whyIt should be noted that configuration files and the like are subject to change by the organizations that maintain their respective applications; this guide generally recommends installing the latest version of these applications, as these generally include security updates. It is expected that the implementer will be able to search through a file and find specific lines in the case where the maintaining organization has moved it in a subsequent patch. In most cases, configuration files will be provided outright to avoid these scenarios.
"},{"location":"installation/manual/introduction/#adding-or-replacing-files","title":"Adding or Replacing Files","text":"When a file needs to be added or replaced, it will be described in three sections:
chmod and chown appropriately to apply the owner, group, and umask/the/path/to/some/file.php | owner:group/umask
<?php\n/**\n * Here is the entire contents of a PHP file\n */\nfunction do_something($to_this) {\n$to_this += THE_NUMBER_TO_ADD_TO_THIS;\n}\n?>\nTHE_NUMBER_TO_ADD_TO_THIS: 12, perhaps with an explanation of why, or other numbers that may be appropriate"},{"location":"installation/manual/introduction/#troubleshooting","title":"Troubleshooting","text":"The most common issues you will likely run into when manually provisioning a server are:
ls -la, and ensure their ownership using chown USER for files, and chown -R USER for directoriesFor any other issues, don't hesitate to email the mailing list to ask for help. If you think that a part of the installation documentation is incorrect or could be improved, please create an issue in the documentation issues queue and give it a documentation tag. Bear in mind that this guide is built for Ubuntu 18.04 and attempts to give generalized instructions; you will likely naturally encounter situations where your own environment needs to differ from the guide.
Needs Maintenance
The manual installation documentation is in need of attention. We are aware that some components no longer work as documented here. If you are interested in helping us improve the documentation, please see Contributing.
"},{"location":"installation/manual/preparing-a-webserver/#in-this-section-we-will-install","title":"In this section, we will install:","text":"Apache can typically be installed and configured outright by your operating system\u2019s package manager:
sudo apt-get -y install apache2 apache2-utils\nThis will install:
systemd service that will ensure Apache can be stopped and started, and will run when the machine is powered on/etc/apache2, including the basic configuration, ports configuration, enabled mods, and enabled sites/var/www/html, configured to be the provided server on port :80 in /etc/apache2/sites-enabled/000-default.conf; we\u2019ll make changes and additions to this file laterwww-data, which we will use to read/write web documents.We\u2019re going to enable a couple of Apache mods that Drupal highly recommends installing, and which are de-facto considered required by Islandora:
sudo a2enmod ssl\nsudo a2enmod rewrite\nsudo systemctl restart apache2\nwww-data Group","text":"Since the user we are currently logged in as is going to work quite a bit inside the Drupal directory, we want to give it group permissions to anything the www-data group has access to. When we run composer, www-data will also be caching data in our own home directory, so we want this group modification to go in both directions.
N.B. This code block uses backticks, not single quotes; this is an important distinction as backticks have special meaning in bash.
Note If doing this in the terminal, replace \"whoami\" with your username and remove the backticks
sudo usermod -a -G www-data `whoami`\nsudo usermod -a -G `whoami` www-data\n# Immediately log back in to apply the new group.\nsudo su `whoami`\nIf you're running Debian 11 you should be able to install PHP 7.4 from the apt packages directly:
sudo apt-get -y install php7.4 php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-imap php7.4-json php7.4-mbstring php7.4-opcache php7.4-xml php7.4-yaml php7.4-zip libapache2-mod-php7.4 php-pgsql php-redis php-xdebug unzip\nIf you're running Debian 10, the repository for the PHP 7.4 packages needs to be installed first:
sudo apt-get -y install lsb-release apt-transport-https ca-certificates\nsudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg\necho \"deb https://packages.sury.org/php/ $(lsb_release -sc) main\" | sudo tee /etc/apt/sources.list.d/php.list\nsudo apt-get update\nsudo apt-get -y install php7.4 php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-imap php7.4-json php7.4-mbstring php7.4-opcache php7.4-xml php7.4-yaml php7.4-zip libapache2-mod-php7.4 php-pgsql php-redis php-xdebug unzip\nThis will install a series of PHP configurations and mods in /etc/php/7.4, including:
mods-available folder (from which everything is typically enabled by default)apache2 foldercli folderunzip, which is important for PHP\u2019s zip module to function correctly despite it not being a direct dependency of the module. We will also need to unzip some things later, so this is convenient to have in place early in the installation process.PostgreSQL can generally be easily installed using your operating system\u2019s package manager. It is typically sensible to install the version the system recognizes as up-to-date. We\u2019re simply going to install the database software:
sudo apt-get -y install postgresql\nThis will install:
postgres; this will be the only user, by default, that has permission to run the psql binary and have access to Postgres configurations/usr/bin/psql, which anyone - even root - will get kicked out of the moment they run it, since only the postgres user has permission to run any Postgres commands/etc/postgresql/11/main which can be used to modify how PostgreSQL works.A modification needs to be made to the PostgreSQL configuration in order for Drupal to properly install and function. This change can be made to the main configuration file at /etc/postgresql/11/main/postgresql.conf:
Before:
558 | #bytea_output = \u2018hex\u2019 # hex, escape
After:
558 | bytea_output = \u2018escape\u2019
(Remove the \"# hex, escape\" comment and change the value from \"hex\" to \"escape\")
The postgresql service should be restarted to accept the new configuration:
sudo systemctl restart postgresql\nIslandora Starter Site uses the built-in Drupal \"Audio\" media type for video. Islandora Starter Site configures it to accept files of type mp3, wav, and aac.
Islandora uses Homarus (ffmpeg as a microservice) to create audio derivatives. Islandora Starter Site sets you up to create:
-codec:a libmp3lame -q:a 5, stored in the public filesystemThese parameters can be changed in the configuration for the Drupal Action that Islandora uses to generate an Audio derivative.
Islandora Starter Site sets up a context to automatically create this derivative when:
http://pcdm.org/use#OriginalFile)http://purl.org/coar/resource_type/c_18cc)The mimetype formats allowed by Homarus are configured in Homarus itself - see Installing Crayfish
"},{"location":"models/audio/#display","title":"Display","text":"Drupal provides an \"Audio\" field formatter for file fields that displays a simple playable audio widget. It works but does not support captions/subtitles. Islandora provides an \"Audio with captions\" formatter that allows for captions.
To use the captions feature out of the box, add the captions track as a WEBVTT file (.vtt) in the Audio media's \"Track\" field (see below regarding which Audio media to use). If you don't have the \"Track\" field (provided by Islandora Starter Site), create a field of type \"Media Track\" (a type provided by Islandora) on the same Media (or more broadly, same entity) as your audio file. Then use the Manage Display page to set your audio file field to render using the \"Audio with captions\" field formatter.
If you're using Islandora Starter Site, you can expect to see an audio player on your node. This is done with a special view (an EVA view) that displays service files, which is configured to show on Repository Item's default display mode (on the \"Manage Display\" page). The EVA view renders the service file media using the \"Source\" view mode. For Audio media, this is configured to show only the audio file using the \"Audio with Captions\" widget. Note that captions, to be displayed, must be on the media that is playing. Thus, when the Service file media is being played, captions on the Original File media are ignored.
"},{"location":"models/video/","title":"Video","text":""},{"location":"models/video/#media-and-file-types","title":"Media and file types","text":"Islandora Starter Site uses the built-in Drupal \"Video\" media type for videos. Islandora Starter Site configures it to accept files of type mp4.
The Install Profile also makes available the \"Remote video\" media type, but its behaviour has not been publicly documented.
"},{"location":"models/video/#derivatives","title":"Derivatives","text":"Islandora uses Homarus (ffmpeg as a microservice) to create video derivatives. Islandora Starter Site sets you up to create:
-ss 00:00:01.000 -frames 1 -vf scale=100:-2. These parameters can be changed in the configuration for the Drupal Actions that Islandora uses to create video derivatives.
Islandora Starter Site sets up a context to automatically create this derivative when:
http://pcdm.org/use#OriginalFile)http://purl.org/coar/resource_type/c_18cc)The mimetype formats allowed by Homarus are configured in Homarus itself - see Installing Crayfish
"},{"location":"models/video/#display","title":"Display","text":"Drupal provides a \"Video\" field formatter for file fields that displays a simple playable video widget. It works but does not support captions/subtitles. Islandora provides a \"Video with Captions\" formatter that allows for captions.
To use the Captions feature out of the box, add the captions track as a WEBVTT file (.vtt) in the Video media's \"Track\" field (see below regarding which Video media to use). If you don't have the \"Track\" field (provided by Islandora Starter Site), create a field of type \"Media Track\" (a type provided by Islandora) on the same Media (or more broadly, same entity) as your audio file. Then use the Manage Display page to set your audio file field to render using the \"Audio with captions\" field formatter.
If you're using Islandora Starter Site, you can expect to see a video player on your node. This is done with a special view (an EVA view) that displays service files, which is configured to show on the default view of Repository Items (see the \"Manage Display\" page). The EVA view renders the service file media using the \"Source\" view mode. For Video media, this is configured to show only the video file using the \"Video with Captions\" formatter. Thus, when the Service file media is being played, captions on the Original File media are ignored.
"},{"location":"release_notes/8.x-2.0/","title":"Islandora 2.0.0 Release Notes / Upgrade Instructions","text":""},{"location":"release_notes/8.x-2.0/#update-to-php-74","title":"Update to PHP 7.4","text":"Upstream dependencies have forced us to upgrade PHP to 7.4. How to do this varies by operating system, but for Ubuntu 18.04 the following will work:
sudo apt install software-properties-commonsudo add-apt-repository ppa:ondrej/phpsudo apt updatesudo apt install php7.4sudo apt install php7.4-curl php7.4-xml php7.4-gd php7.4-imap php7.4-mbstring php7.4-mysqlHere's the steps to update Alpaca code in general. None of this is specific to this release, it's just the process.
Move your blueprint config out of the deploy directory temporarily
mv /opt/karaf/deploy/ca.islandora.* /opt/karafNow we'll uninstall the old features and install the new ones. We're assuming you're currently on 1.0.3 and want to update to 1.0.5, but you may be on an older version. Keep that in mind when copy/pasting these commands.
/opt/karaf/bin/client \"feature:uninstall islandora-connector-derivative\"/opt/karaf/bin/client \"feature:uninstall islandora-indexing-fcrepo\"/opt/karaf/bin/client \"feature:uninstall islandora-indexing-triplestore\"/opt/karaf/bin/client \"feature:uninstall islandora-http-client\"/opt/karaf/bin/client \"feature:repo-remove mvn:ca.islandora.alpaca/islandora-karaf/1.0.3/xml/features\"/opt/karaf/bin/client \"feature:repo-add mvn:ca.islandora.alpaca/islandora-karaf/1.0.5/xml/features\"/opt/karaf/bin/client \"feature:install islandora-http-client\"/opt/karaf/bin/client \"feature:install islandora-indexing-triplestore\"/opt/karaf/bin/client \"feature:install islandora-indexing-fcrepo\"/opt/karaf/bin/client \"feature:install islandora-connector-derivative\"Move the blueprint files back to the deploy folder to deploy the derivative routes
mv /opt/karaf/ca.islandora.* /opt/karaf/deployYou can check that everything is working with
/opt/karaf/bin/client \"la | grep islandora\"If all the bundles have a state of Active, then you're all good!
Some users have experienced timeouts when generating derivatives. We've added the ability to set the timeout length to your routes. In order to do so, edit each ca.islandora.* file in the deploy directory and replace
<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\"/>\nwith
<bean id=\"requestConfigConfigurer\" class=\"ca.islandora.alpaca.connector.derivative.RequestConfigConfigurer\">\n <property name=\"connectionRequestTimeoutMs\" value=\"10000\"/>\n <property name=\"connectTimeoutMs\" value=\"10000\"/>\n <property name=\"socketTimeoutMs\" value=\"10000\"/>\n</bean>\n\n<bean id=\"http\" class=\"org.apache.camel.component.http4.HttpComponent\">\n <property name=\"httpClientConfigurer\" ref=\"requestConfigConfigurer\"/>\n</bean>\n\n<bean id=\"https\" class=\"org.apache.camel.component.http4.HttpComponent\">\n <property name=\"httpClientConfigurer\" ref=\"requestConfigConfigurer\"/>\n</bean>\nYou can set the timeout values to whatever you like, we're using 10000 in this example.
"},{"location":"release_notes/8.x-2.0/#update-drupal-core-and-modules","title":"Update Drupal core and modules","text":"To get the latest Drupal code, edit your composer.json file and change the islandora/* packages versions' to the latest (2.0.0 for most). Then run the following command to update them, your dependencies, and Drupal core:
sudo php -d memory_limit=-1 `which composer` update --with-all-dependencies\nAfterwards, you need to run drush updb or visit /update.php in your browser to trigger the database updates. This includes important changes, like stripping out configuration we no longer use as well as a fix to support large media (>2.2 GB). The database updates may take a while depending on how many media you have.
We removed the Gemini microservice and pushed its code into Crayfish-Commons so that it can be used by the other microservices and Drupal modules. Once you've updated your Drupal installation, Gemini will no longer be utilized. It will still, however, need to be removed. It is recommended to test this method in a staging/development environment before running in your production environment.
2.0.0 version of the islandora module. If you followed the steps above to update Drupal core and its modules, you'll have this.Import two new Islandora views
i. This can be done by coping the views into a new directory and using drush such as
mkdir /var/www/html/drupal/config/1time\ncp /var/www/html/drupal/web/modules/contrib/islandora/modules/islandora_core_feature/config/install/views.view.all_taxonomy_terms.yml /var/www/html/drupal/config/1time/\ncp /var/www/html/drupal/web/modules/contrib/islandora/modules/islandora_core_feature/config/install/views.view.non_fedora_files.yml /var/www/html/drupal/config/1time/\ndrush config:import --partial --source /var/www/html/drupal/config/1time\n/admin/config/development/configuration/single/import 4. Update Crayfish to 2.0.0 1. cd /var/www/html/Crayfish 2. git fetch origin --tags 3. git checkout 2.0.0 6. Run composer install for Milliner such as cd /var/www/html/Crayfish/Milliner && composer install 7. Run composer install for Recast such as cd /var/www/html/Crayfish/Recast && composer install 8. Update the Recast config 1. cd /var/www/html/Crayfish/Recast/cfg 2. open config.yaml 3. Remove gemini_base_url 4. Update base_url in fedora_resource section. It should contain a value something like http://localhost:8080/fcrepo/rest 8. Clear your cache in Drupal using Drush (cd /var/www/html/drupal && drush cr) or the UI. 9. Restart apache sudo systemctl restart apache2 9. Test to make sure you can view existing objects in Fedora by going to the Drupal UI and clicking one of the Fedora URI links 10. Create a new object and make sure it persists to Fedora and the Fedora URI link resolves correctly. Make sure that no new entries get added to Gemini. 11. Once you've confirmed everything is working as expected, you can then remove Gemini like 1. rm /var/www/html/Crayfish/Gemini 2. Remove the gemini database and the associated mysql user 3. Remove the vhost file for gemini: sudo rm /etc/apache2/conf-enabled/Gemini.conf 4. Restart apache: sudo systemctl restart apache2 You can now add captions to A/V content! If you are starting with a new installation, there's nothing to do, but if you have an existing install, here's how to configure Drupal to display captions/
2.0.0 version of islandora and islandora_defaults. If you followed the steps above to update Drupal core and its modules, you'll have this.islandora_defaults using the Features UI (recommended) or by copying them into a temporary directory and using drush to do a partial config import.drush crDrupal requires the use of a _format query parameter to get alternate representations of a node/media.
By default, Islandora deploys with the jsonld module and the Milliner microservice. These two components are configured to strip this _format query parameter off of the end of URIs.
This means that when your content is indexed in Fedora, the triplestore, etc... it's URI will be something like http://localhost:8000/node/1 and not http://localhost:8000/node/1?_format=jsonld.
If you are using a very early version of Islandora \"8\" (pre-release), then you may have URIs with _format=jsonld at the end of them.
If you update to newer code, you will need to ensure that your site is configured to add ?_format=jsonld back to the URLs if you want to maintain consistency.
If you don't do this, you can end up with two copies of your objects in your Fedora repository (one with and one without ?_format=jsonld). You will also have two sets of triples in your triplestore.
To turn the ?_format parameter back on:
admin/config/search/jsonld and confirm the \"Remove jsonld parameter from @ids\" checkbox is unchecked.strip_format_jsonld: false to your Milliner config. If you deployed using the default Islandora-playbook this file would be located at /var/www/html/Crayfish/Milliner/cfg/config.yaml.If you are using Islandora-playbook and are provisioning new environments for your older Islandora, you'll want to lock down the variables in your inventory that control this config.
crayfish_milliner_strip_format_jsonld: truewebserver_app_jsonld_remove_format: 1Alpaca is event-driven middleware based on Apache Camel for Islandora
Currently, Alpaca ships with four event-driven components
This service receives requests from Drupal when it wants to create derivatives and passes that request along to a microservice in Crayfish. When it receives the derivative file back from the microservice, it passes the file back to Drupal.
"},{"location":"technical-documentation/alpaca-tips/#islandora-http-client","title":"islandora-http-client","text":"This service overrides the default http client with Islandora specific configuration.
"},{"location":"technical-documentation/alpaca-tips/#islandora-indexing-fcrepo","title":"islandora-indexing-fcrepo","text":"This service receives requests from Drupal in response to write operations on entities. These requests are passed along to Milliner microservice in Crayfish to convert Drupal entities into Fedora resources and communicate with Fedora (via Chullo).
"},{"location":"technical-documentation/alpaca-tips/#islandora-indexing-triplestore","title":"islandora-indexing-triplestore","text":"This service receives requests from Drupal on indexing and deleting in order to persist/delete content in the triplestore.
"},{"location":"technical-documentation/alpaca-tips/#steps-for-developing-with-alpaca","title":"Steps for developing with Alpaca","text":"Alpaca now runs as a single executable jar which can enable none, some or all of the available services.
To develop your own module, start by cloning the Alpaca code base.
Then create a new directory (for example my-new-module) along side the islandora-indexing-fcrepo, islandora-indexing-triplestore directories
Add your new directory to the settings.gradle file, following the pattern of the others.
include ':islandora-support'\ninclude ':islandora-indexing-triplestore'\ninclude ':islandora-indexing-fcrepo'\ninclude ':islandora-connector-derivative'\ninclude ':islandora-http-client'\ninclude ':islandora-alpaca-app'\n+ include ':my-new-module'\nproject(':islandora-alpaca-app').setProjectDir(\"$rootDir/islandora-alpaca-app\" as File)\nproject(':islandora-support').setProjectDir(\"$rootDir/islandora-support\" as File)\nproject(':islandora-indexing-triplestore').setProjectDir(\"$rootDir/islandora-indexing-triplestore\" as File)\nproject(':islandora-indexing-fcrepo').setProjectDir(\"$rootDir/islandora-indexing-fcrepo\" as File)\nproject(':islandora-connector-derivative').setProjectDir(\"$rootDir/islandora-connector-derivative\" as File)\nproject(':islandora-http-client').setProjectDir(\"$rootDir/islandora-http-client\" as File)\n+ project(':my-new-module').setProjectDir(\"$rootDir/my-new-module\" as File)\nYou can explore the islandora-indexing-fcrepo module to see the pattern to develop your own module.
This module contains three classes.
You can ignore the CommonProcessor class, that is just some processing that is split out for reusability.
The first class is the FcrepoIndexer, this class extends the Apache Camel RouteBuilder and requires a configure method which defines the processing elements of your workflow. This is the Camel \"route\".
The second class is the FcrepoIndexerOptions, this class extends the Alpaca PropertyConfig base class which gets common configuration parameters into your module. It also contains any custom configuration parameters needed for your route.
Lastly it uses the @Conditional(FcrepoIndexerOptions.FcrepoIndexerEnabled.class) to define when this module is enabled.
FcrepoIndexerOptions.FcrepoIndexerEnabled.class refers to the static inner class.
This class is inside of FcrepoIndexerOptions and works like this:
[1] static class FcrepoIndexerEnabled extends ConditionOnPropertyTrue { \n[2] FcrepoIndexerEnabled() {\n[3] super(FcrepoIndexerOptions.FCREPO_INDEXER_ENABLED, false);\n[4] }\n[5] }\nLine 2 is the constructor for this static class
Line 3 passes to the parent constructor two things.
So in this case we check for the property fcrepo.indexer.enabled and if we don't find it, we pass false. So this module is assumed to be \"off\" unless the property fcrepo.indexer.enabled=true is located.
The last thing is to add your new module to the islandora-alpaca-app build.gradle file as a dependencies, like the existing modules. i.e.
dependencies {\n implementation \"info.picocli:picocli:${versions.picocli}\"\n implementation \"org.apache.camel:camel-spring-javaconfig:${versions.camel}\"\n implementation \"org.slf4j:slf4j-api:${versions.slf4j}\"\n implementation \"org.springframework:spring-context:${versions.spring}\"\n implementation project(':islandora-support')\n implementation project(':islandora-connector-derivative')\n implementation project(':islandora-indexing-fcrepo')\n implementation project(':islandora-indexing-triplestore')\n+ implementation project(':my-new-module')\n\n runtimeOnly \"ch.qos.logback:logback-classic:${versions.logback}\"\n\n}\nFinally from the top-level directory of Alpaca execute
./gradlew clean build shadowJar\nThis tells Gradle to clean the modules, then build the modules and finally create a single jar with all needed code (the shadow jar).
The final executable jar is:
<alpaca directory>/islandora-alpaca-app/build/libs/islandora-alpaca-<version>-all.jar\nBefore opening a pull request, you should check your code style. If you are using the Vagrant, you can run phpcs within the Drupal installation directory (on the Vagrant, that is /var/www/html/drupal) or from within the web directory (/var/www/html/drupal/web) as follows:
./vendor/bin/phpcs --standard=./vendor/drupal/coder/coder_sniffer/Drupal modules/contrib/my_module, where modules/contrib/my_module is the relative or full path to the PHP file you want to check.web directory: ../vendor/bin/phpcs --standard=../vendor/drupal/coder/coder_sniffer/Drupal yourfile, where yourfile is the relative or full path to the PHP file you want to check.In both cases:
web: --standard=../vendor/drupal/coder/coder_sniffer/DrupalIslandora runs phpcs in its Github continuous integration environment, and there, it specifies which files to ignore and which files to check. It is a good idea for developers to specify the same options when running phpcs locally, prior to opening a pull request. For example (running phpcs from the within Drupal's web directory), you should use the following --ignore and --extensions options:
../vendor/bin/phpcs --standard=../vendor/drupal/coder/coder_sniffer/Drupal --ignore=*.md --extensions=php,module,inc,install,test,profile,theme,css,info modules/contrib/my_module
Diagram prepared by Bethany Seeger based on work done by Gavin Morris
"},{"location":"technical-documentation/diagram/#components","title":"Components","text":""},{"location":"technical-documentation/diagram/#islandora","title":"Islandora","text":"The following components are microservices developed and maintained by the Islandora community. They are bundled under Islandora Crayfish:
The following components are deployed with Islandora, but are developed and maintained by other open source projects:
This documentation is built using MkDocs, a static site generator that is geared towards building project documentation. The documentation is created in the Markdown format, and it all resides in the docs directory in the repository. The organization of the documentation is controlled by the mkdocs.yml in the root of the repository.
Video version available
Some of the material in this tutorial is presented in our video, How to Build Documentation.
"},{"location":"technical-documentation/docs-build/#prerequisites","title":"Prerequisites","text":"You will need to have mkdocs software installed locally, as well as a required plugin and the MkDocs Material theme. Below we will show you how to install mkdocs using the Python language's pip tool. For more details on installing and using MkDocs visit the MkDocs installation guide.
Open a terminal window.
Install mkdocs:
Windows / Linux:
sudo -H pip install mkdocs
macOS:
pip3 install mkdocs
Install plugin to enable display of the last revision date:
Windows / Linux:
sudo -H pip install mkdocs-git-revision-date-localized-plugin
macOS:
pip3 install mkdocs-git-revision-date-localized-plugin
Install plugin to enable redirects:
Windows / Linux:
sudo -H pip install mkdocs-redirects
macOS:
pip3 install mkdocs-redirects
Install Material theme:
Windows / Linux:
sudo -H pip install mkdocs-material
macOS:
pip3 install mkdocs-material
Make sure you have all the submodules:
git submodule update --init --recursive
Documentation is build by running to the following command in the root of the repository:
mkdocs build --clean
This command will create a static site folder in the root of the repository.
You can preview any changes you have made to the documentation by running the following command:
mkdocs serve
And then visiting http://localhost:8111 in your browser.
To deploy documentation to GitHub Pages, issue the following command:
mkdocs gh-deploy --clean
To stop the mkdocs serve command just type the key combination \"Control-c\".
Islandora can make use of the majority of Drupal modules and themes. Common use cases have been documented in the Islandora Cookbook. There are several ways to download and install Drupal modules. Please refer to this guide on Drupal.org.
Composer is the recommended method to install and update drupal modules and themes in Islandora.
$ composer require \"<vendor>/<package>:<version>\"\n# Example\n$ composer require \"islandora/jsonld:^2\"\nIn the Islandora playbook, you can add a Drupal module's or theme's machine name to the drupal_composer_dependencies variable here. To enable the Drupal module or theme, add the module machine name to the drupal_enable_modules variable as well.
For modules that require additional steps, additional tasks may need to be added to the Ansible playbook. Re-provisioning your instance via Ansible will install the module.
"},{"location":"technical-documentation/install-enable-drupal-modules/#video-walkthroughs-modules-installing-modules-with-composer","title":"Video Walkthroughs: Modules & Installing modules with Composer","text":"Click the image below to open the introduction to Modules video tutorial on the Islandora Youtube channel.
Click the image below to open the Installing modules with Composer video tutorial on the Islandora Youtube channel.
See more videos from the Drupal 101 series here.
"},{"location":"technical-documentation/migrate-7x/","title":"Islandora 7","text":""},{"location":"technical-documentation/migrate-7x/#migrating-from-islandora-legacy-to-islandora-youtube-video","title":"Migrating from Islandora Legacy to Islandora (YouTube Video)","text":"Nov 21, 2019
!! note \"Migration from Islandora Legacy\" For tools to migrate into Islandora from an existing Islandora Legacy instance, please see migrate_7x_claw.
"},{"location":"technical-documentation/migrate-7x/#project-planning-for-migration","title":"Project Planning for Migration","text":"Presented by Mark Jordan and Janice Banser, this session will outline strategies for understanding your data and planning for a move to Islandora.
"},{"location":"technical-documentation/migrate-csv/","title":"CSV","text":""},{"location":"technical-documentation/migrate-csv/#table-of-contents","title":"Table of Contents","text":"This tutorial introduces you to the Drupal Migrate tools available to create Islandora content. Whether you will eventually use CSVs or other sources (such as XML or directly from a Legacy Islandora) this tutorial should be useful as it covers the basics and mechanics of migration.
This tutorial uses the configurations and code available in the migrate_islandora_csv module which, when enabled, will create three example migrations ready for you to use with the Migrate API. Each migration comes from one of the files in the config/install folder. We'll walk through them in detail below. The module is also uses Features which allows you to make changes to the configuration files and sync those changes into your Drupal site.
A note on using Features
This tutorial (and Islandora in general) makes heavy use of Features, which is an easy way to ship and install Drupal configuration. However, after enabling a Feature module, the code in that module's directory is no longer \"live\", as the configuration now resides in the Drupal database. If you change code in the YAML files, it will not take effect until you re-import the Feature. There is a walkthrough in the \"Configuration\" section of the Migrate 7.x to Islandora tutorial.
Sample CSV and images are also included in the module as a convenience so they are easily available on the Drupal server running the migration. (This is not the recommended method for making files available to Drupal in a real migration.)
The module also contains a Migrate process plugin that transforms strings into associative arrays. This is useful for populating multiple Contributor (field_linked_agent) fields. (See \"Typed Relation\" for more information on the Contributor field's type.) It will be available when this module is enabled, and the node migration uses it. It was written generically and will hopefully become part of Migrate Plus, but for now it is here.
When you are ready to create your actual migrations, the contents of this repository can function as a template for you to create the YAML files defining your own migrations.
"},{"location":"technical-documentation/migrate-csv/#introduction","title":"Introduction","text":"Why CSV? CSV files (whether separated by commas, tabs, or other delimiters) are easy to understand and work with, and there's good tooling available for using them with Drupal's Migrate API. The Drupal contrib module migrate_source_csv provides a source plugin that reads from a CSV file, and migrate_plus provides more tools and flexibility for creating migrations, including the ability to create customized migrations using YAML and package them up as Features.
In this tutorial, we'll be inspecting each migration file in detail before running it. You'll start out by migrating the images themselves first, and then you'll create various Drupal entities to describe the files from the metadata in the CSV. It's not as scary as it sounds (especially since this module contains the data we'll be using in a data directory), but you will need a few things before beginning:
vagrant ssh into the box, navigate into Drupal, and use git and drush, etc... If you can copy/paste into a terminal, you'll survive.The configuration referred to in this tutorial comes from the Islandora Starter Site, which defines the content model and metadata fields that we'll be migrating into. You're not required to use the Islandora Starter Site for your repository, but for the purposes of demonstration, it saves you a lot of user interface administrivia so you can focus just on the learning how to migrate. By the time you are done with this exercise, you'll be able to easily apply your knowledge to migrate using any custom metadata profile you can build using Drupal.
"},{"location":"technical-documentation/migrate-csv/#overview","title":"Overview","text":"The Migrate API is the main way to ingest batches of data into Drupal (and because Islandora is Drupal, into Islandora). The Migrate module only provides the framework, it's up to you to create the rules that take data from a source, through a process (i.e. a mapping) to a destination. A set of these rules is called a \"migration\". It has to be set up (as a Configuration Entity, either by importing a YAML file or by installing a Feature) and then it has to be run.
Once a migration has been run, it will have created (or updated) a bunch of Drupal entities of one type - whether that's taxonomy terms, nodes, files, etc. Since an Object in Islandora is made up of several different Drupal entities that refer to each other, it's going to take multiple migrations to create an Islandora object, and it's important to perform these migrations in a sensible order.
A basic Islandora object is at minimum: - a file, which holds the actual binary contents of an item - a node, which holds the descriptive metadata for an item and makes it discoverable - a media, which holds technical metadata and references the file and the node, linking the two together.
Therefore, each row in your CSV must contain enough information to create these.
Using multiple sources for object components
While this tutorial uses a single CSV to describe all the parts of an islandora object, a site can use multiple sources (CSV or otherwise) to migrate the various parts. The only requirements then is to have keys in place matching the parts together. For example, a 'files.csv' can use paths that serve as file identifiers and 'metadata.csv' can use an object identifier. As long as the source for the media migration has the key values for the file and metadata migrations (the file paths and object identifiers, respectively, in this example), we can create the necessary links between the object's components. This is very useful when you need to migrate multiple media/file pairs associated with a single node, such as pre-processed derivatives that need to be migrated.
Buried in your descriptive metadata are often references to other things which aren't repository items themselves, but records still need to be kept for them. Authors, publishers, universities, places, etc... can be modeled as Drupal Entities, so that they can be referenced by other Entities. So there's the potential to have a lot of different entity types described in a single row in a CSV.
In this tutorial, we will use the three migrations defined in this module to migrate into Drupal entities. The entities we are migrating content into have already been defined by the Islandora Starter Site:
We can do this because subjects, persons, and corporate bodies are (in this example) represented by simple 'name' strings. We create them on the fly, as if we were adding new tags to a tag vocabulary. If we wanted to model subjects, people, or corporate bodies as entities with multiple fields (first name, last name, date, subheadings, URI, etc.) then we would need up to six migrations.
Migrations follow the Extract-Transform-Load pattern. You extract the information from a source, process the data to transform it into the format you need, and load it into the destination system (i.e. Drupal). Migrations are stored in Drupal as configuration, which means they can be represented in YAML, transferred to and from different sites, and are compatible with Drupal's configuration synchronization tools. And the structure of each YAML file is arranged to follow the Extract-Transform-Load pattern.
To perform the migrations, we'll be using drush. We will be able to run each of the file, node, and media migrations separately or all at once in a group. We will also learn how to roll back a migration in case it didn't go as planned.
The sample migrations here will give us a chance to show off some techniques for working with multi-valued fields, entity reference fields, and complex field types like controlled_access_terms's typed_relation field. We'll also see how the migrate framework can help de-duplicate, and at the same time, linked data-ize :tm: your data by looking up previously migrated entities.
So hold on to your hats. First, let's get this module onto your Islandora instance.
"},{"location":"technical-documentation/migrate-csv/#installation","title":"Installation","text":"From your islandora-playbook directory, issue the following commands to enable this module: - vagrant ssh to open a shell in your Islandora instance. - cd /var/www/html/drupal/web/modules/contrib to get to your modules directory. - git clone https://github.com/Islandora/migrate_islandora_csv to clone down the repository from GitHub. - drush en -y migrate_islandora_csv to enable the module, installing the migrations as configuration.
Optionally, flush the cache (drush cr), so the migrations become visible in the GUI at Manage > Structure > Migrations > migrate_islandora_csv (http://localhost:8000/admin/structure/migrate/manage/migrate_islandora_csv/migrations)
Now lets go migrate some files.
Caution
As you saw, you can git clone into the modules directory, but if you're installing a custom module that's intended to stay installed for the long term (unlike a migration feature, which you should probably uninstall and delete when you're done with it) then you may want to check with your devops folks and use Composer instead. However, using Git directly allows you to be more flexible when iterating and testing.
To migrate files (i.e. just the raw binaries) from a CSV, you need: - a column in the CSV containing paths to the files you wish to ingest, and - the files need to be accessible from the server that's running Drupal so that the Migrate framework can find them.
This tutorial assumes you're working with the sample images provided in the module, which will be located at /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/images. When you're migrating for real, the files will have to be uploaded or otherwise made accessible to the server before this point.
Open up the CSV file at data/migration.csv and you'll see a file column containing paths to the sample images. You can use your favorite shell-based utility to open it at /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv, or browse to it on GitHub, or just look at it pasted below as this tutorial does not require you to edit the files.
Open up the \"files\" migration at config/install/migrate_plus.migration.file.yml. You'll see the following migration config:
id: file\nlabel: Import Image Files\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: '/var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv'\n delimiter: ','\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique key (\"id\") per row in the CSV. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # Islandora uses Flysystem and stream wrappers to work with files. What we're really saying here is\n # to put these files in Fedora in a 'csv_migration' folder. It doesn't matter if the directory\n # doesn't exist yet, it will get created for you automatically.\n destination_dir: 'fedora://csv_migration'\n\nprocess:\n\n ##\n # The following two fields are temporary, and just used to generate a destination for the file.\n ##\n\n # Hack the file name out of the full path provided in the 'file' column.\n filename:\n -\n plugin: callback\n callable: pathinfo\n source: file\n -\n plugin: extract\n index:\n - basename\n\n # Construct the destination URI using the file name.\n destination:\n plugin: concat\n delimiter: /\n source:\n - constants/destination_dir\n - '@filename'\n\n ##\n # Here's where we copy the file over and set the URI of the file entity.\n ##\n uri:\n plugin: file_copy\n source:\n - file # The source column in the CSV\n - '@destination' # The destination entry from above\n\ndestination:\n # These are Drupal 'image' entities we're making, not just plain 'file' entities.\n plugin: 'entity:file'\n type: image\nIt seems like a lot to take in at first, but there's a pattern to Drupal migrations. They always contain core identification information and three key sections: source, process, and destination. And these sections correspond exactly to Extract, Transform, and Load.
The first section of a migration contains metadata for Drupal about the migration itself. The id parameter is the machine name of this migration, and must not conflict with existing migrations. Note: use alphanumeric characters and underscores. Hyphens (-) will cause the migration to fail. The label is a human-readable string to identify this migration, and the migration_group is where this migration will be grouped in the GUI.
The source section configures a Drupal source plugin that will extract the data. A source plugin provides \"rows\" of data to processing plugins so that they can be worked on. In this case, we're using the csv source plugin, which very literally uses rows, however you can have source plugins that work with other data formats like XML and JSON. Look at the config from this section.
source:\n plugin: csv\n path: '/var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/migration.csv'\n delimiter: ','\n header_row_count: 1\n ids:\n - file\n constants:\n destination_dir: 'fedora://csv_migration'\nWe're going to dive into the details of this step below, but in summary: the process section is where we extract the desired bits from that row, transform them as desired, and populate them into an associative array. This section is a series of named steps, that call one or more process plugins. These plugins are executed in sequence, with the results getting passed from one to the next, forming a pipeline. By the end of the step, you have transformed some element of data (perhaps through text manipulation, concatenation, etc...) into the form that Drupal is expecting. The resulting value gets associated with the name of the step.
If the name of a step is the same as a field or property name on the target entity, the migrated entity will have that value for that field or property. This is how you can apply metadata from the CSV to an entity. If the step name is not the name of a field or property on the target entity, the migrate framework assumes it's a temporary value you're using as part of more complex logic. It won't wind up on the entity when the migration is done, but it will be available for you to use within other process plugins. You can always spot when a temporary value is being used by the fact that it's prefixed with an @ and surrounded by quotes. You can also pass constants into process plugins, which are prefixed with constants/.
The destination section contains the configuration that describes what gets loaded into Drupal.
destination:\n plugin: 'entity:file'\n type: image\nIn the process section of the migration, we're copying the images over into a Drupal file system and setting the uri property on the corresponding File entity.
uri:\n plugin: file_copy\n source:\n - file\n - '@destination'\nfile_copy process plugin. But first, we have to know where a file is located and where it should be copied to. We know where the file resides, we have that in the CSV's file column. But we're going to have to do some string manipulation in order to generate the new location where we want the file copied. We're trying to convert something like /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/data/images/Free Smells.jpg to fedora://csv_migration/Free Smells.jpg. The URI we're constructing is a stream wrapper of the form scheme://path/to/file. Islandora uses Flysystem, which allows for integration with many different types of filesystems, both remote and local. With Flysystem, the scheme part of the URI is the name of a filesystem. By default, Fedora is exposed using the scheme fedora://. So by setting uri to fedora://csv_migration/Free Smells.jpg, we're saying \"put Free Smells.jpg in the csv_migration directory in Fedora.\"
Now, to perform this string manipulation in PHP, we'd do something like
$info = pathinfo($filepath);\n$filename = $info['basename'];\n$destination = \"fedora://csv_migration/\" . $filename;\nWe will mimic this exactly in the process section of our migration config. Just like we declare variables and call functions with PHP code, we can make entries in the process section to store the output of Drupal process plugins. We'll build up a destination \"variable\" using a filename \"variable\" and pass it into the file_copy process plugin.
To start, we'll get the filename using two process plugins, which do the same steps as the first two lines of the PHP above:
filename:\n -\n plugin: callback\n callable: pathinfo\n source: file\n -\n plugin: extract\n index:\n - basename\ncallback, lets you execute any PHP function that takes a single input and returns an output. It's not as flexible as making your own custom process plugin, but it's still pretty useful in a lot of situations. Here we're using it to call pathinfo(), telling it to use the file column in the CSV as input. We pass the resulting array from pathinfo() to the extract process plugin, which pulls data out of arrays using the keys you provide it under index. Now that we have the file name, we have to prepend it with fedora://csv_migration/ to make the destination URI. In our PHP code above, we used . to concatenate the strings. In the migration framework, we use the concat process plugin. You provide it with two or more strings to concatenate, as well as a delimiter.
destination:\n plugin: concat\n delimiter: /\n source:\n - constants/destination_dir\n - '@filename'\nIn our PHP code, we concatenated the $filename variable with a string literal. In our process plugin, we can provide the variable, e.g. the output of the filename process step, by prefixing it with an @. We can't, however, pass in fedora://csv_migration directly as a string. At first glance, you might think something like this would work, but it totally doesn't:
# Can't do this. Won't work at all.\n destination:\n plugin: concat\n delimiter: /\n source:\n - 'fedora://csv_migration'\n - '@filename'\nsource values as names of columns from the CSV or names of other process steps. Even if they're wrapped in quotes. It will never try to use the string directly as a value. To circumvent this, we declare a constant in the source section of the migration config. constants:\n destination_dir: 'fedora://csv_migration'\nThis constant can be referenced as constants/destination_dir and passed into the concat process plugin as a source.
There are a lot more process plugins available through the (core) Migrate and Migrate Plus modules, and they are documented on Drupal.org.
"},{"location":"technical-documentation/migrate-csv/#running-the-file-migration","title":"Running the File Migration","text":"Migrations can be executed via drush using the migrate:import command. You specify which migration to run by using the id defined in its YAML. You also need to set parameters to tell Drush who you are and what your site's URL is. Failing to do so will result in derivatives not being generated and malformed/improper RDF. So don't forget them! To run the file migration from the command line, make sure you're within /var/www/html/drupal/web (or any subdirectory) and enter
drush -y --userid=1 --uri=localhost:8000 migrate:import file\n--update flag. drush -y --userid=1 --uri=localhost:8000 migrate:import file --update\nmigration_group in their configuration. You can execute an entire group of migrations using the --group flag. For example, to run the entire group defined in this module drush -y --userid=1 --uri=localhost:8000 migrate:import --group migrate_islandora_csv\nmigrate:rollback command to delete all migrated entities. Like migrate:import, it also respects the --group flag and --uri parameter. So to rollback everything we just did: drush -y --uri=localhost:8000 migrate:rollback --group migrate_islandora_csv\nmigrate:reset command to put a migration back to Idle. For example, with the file migration, use drush -y --uri=localhost:8000 migrate:reset file\nMake sure you've run (and not rolled back) the file migration. It should tell you that it successfully created 5 files. You can confirm its success by visiting http://localhost:8000/admin/content/files. You should see 5 images of neon signs in the list.
Those five images are nice, but we need something to hold their descriptive metadata and show them off. We use nodes in Drupal to do this, and that means we have another migration file to work with. Nestled in with our nodes' descriptive metadata, though, are more Drupal entities, and we're going to generate them on the fly while we're making nodes. While we're doing it, we'll see how to use pipe delimited strings for multiple values as well as how to handle typed_relation fields that are provided by controlled_access_terms. Open up /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/config/install/migrate_plus.migration.node.yml and check it out.
# Uninstall this config when the feature is uninstalled\ndependencies:\n enforced:\n module:\n - migrate_islandora_csv\n\nid: node\nlabel: Import Nodes from CSV\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: modules/contrib/migrate_islandora_csv/data/migration.csv\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique id per row in the csv. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # We're tagging our nodes as Images\n model: Image\n\n # Everything gets created as admin\n uid: 1\n\n# Set fields using values from the CSV\nprocess:\n title: title\n uid: constants/uid\n\n # We use the skip_on_empty plugin because\n # not every row in the CSV has subtitle filled\n # in.\n field_alternative_title:\n plugin: skip_on_empty\n source: subtitle\n method: process\n\n field_description: description\n\n # Dates are EDTF strings\n field_edtf_date: issued\n\n # Make the object an 'Image'\n field_model:\n plugin: entity_lookup\n source: constants/model\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_models\n\n # Split up our pipe-delimited string of\n # subjects, and generate terms for each.\n field_subject:\n -\n plugin: skip_on_empty\n source: subject\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: entity_generate\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: subject\n\n # If you have multiple values of a complex\n # field, iterate over them using sub_process.\n # But sub_process requires structured data\n # i.e. an associative array, not a string\n # or list of strings. To turn strings into\n # associative arrays, use the custom\n # process plugin str_to_assoc.\n\n # Extract a list of names from the column\n # called photographer, and transform it into\n # an array of associative arrays.\n photographers:\n -\n source: photographer\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\n\n # Iterate over the array of associative arrays.\n # We create the taxonomy terms here so that we\n # can specify the bundle - other columns which\n # might feed into field_linked_agent may contain\n # corporate bodies or families. The resulting\n # array contains the resulting term id (tid)\n # under the key 'target_id'.\n # We also add a key-value pair\n # 'rel_type' => 'relators:pht'. Other columns\n # might use different relators.\n linked_agent_pht:\n plugin: sub_process\n source: '@photographers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n rel_type:\n plugin: default_value\n default_value: 'relators:pht'\n\n # Extract an array of names from the column\n # called provider\n providers:\n -\n source: provider\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\n # Generate/lookup taxonomy terms in the\n # corporate body vocab, and add the relator.\n linked_agent_prv:\n plugin: sub_process\n source: '@providers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: 'corporate_body'\n rel_type:\n plugin: default_value\n default_value: 'relators:prv'\n\n # Write to the linked agent field. In this case\n # we first want to merge the info from the\n # photographer and provider columns. Since we\n # already prepared our structured array using\n # the components of the typed_relation field as\n # keys ('target_id' and 'rel_type'), we can just\n # pass this array into field_linked_agent.\n field_linked_agent:\n plugin: merge\n source:\n - '@linked_agent_pht'\n - '@linked_agent_prv'\n\n# We're making nodes\ndestination:\n plugin: 'entity:node'\n default_bundle: islandora_object\nThe source section looks mostly the same other than some different constants we're defining - the string \"Image\" (no quotes needed) and the drupal ID of the user who will be assigned as the author. If values contained special characters such as colons, quotes would be needed.
If you look at the process section, you can see we're taking the title, description, and issued columns from the CSV and applying them directly to the migrated nodes without any manipulation.
title: title\n field_description: description\n field_edtf_date: issued\nsubtitle, we're passing it through the skip_on_empty process plugin because not every row in our CSV has a subtitle entry. It's very useful when you have spotty data, and you'll end up using it a lot. The method: process bit tells the migrate framework only skip that particular field if the value is empty, and not to skip the whole row. It's important, so don't forget it. The full YAML for setting field_alternative_title from subtitle looks like this: field_alternative_title:\n plugin: skip_on_empty\n source: subtitle\n method: process\nNow here's where things get interesting. We can look up other entities to populate entity reference fields. For example, all Repository Items have an entity reference field that holds a taxonomy term from the islandora_models vocabulary. All of our examples are images, so we'll look up the Image model in the vocabulary since it already exists (it gets made for you when you use islandora-playbook). We use the entity_lookup process plugin to do this.
field_model:\n plugin: entity_lookup\n source: constants/model\n entity_type: taxonomy_term\n # 'name' is the string value of the term, e.g. 'Original file', 'Thumbnail'.\n value_key: name\n bundle_key: vid\n bundle: islandora_models\nThe entity_lookup process plugin looks up an entity based on the configuration you give it. You use the entity_type, bundle_key, and bundle configurations to limit which entities you search through. entity_type is, as you'd suspect, the type of entity: node, media, file, taxonomy_term, etc... bundle_key tells the migrate framework which property holds the bundle of the entity, and bundle is the actual bundle id you want to restrict by. In this case we specify the vid (vocabulary id) has to be islandora_models - which is the machine name of the vocabulary we're interested in. In this plugin, source is the value to search for - in this case we're looking for the string \"Image\", which we've defined as a constant. And we're comparing it to the name field on each term by setting the value_key config.
This approach applies the same taxonomy term to all objects. If you want to assign a taxonomy term at the node level (that is, potentially a different term for each node) rather than to all the nodes being imported, you can use a configuration as illustrated next. In this example, we have a column in our CSV input file with the header 'model', which we define as the source of the field_model values:
field_model:\n plugin: entity_lookup\n # 'model' is the header of a field in our input CSV that contains the string value of the taxonomy term.\n source: model\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_models\nIf you're not sure that the entities you're looking up already exist, you can use the entity_generate plugin, which takes the same config, but will create a new entity if the lookup fails. We use this plugin to create subject taxonomy terms that we tag our nodes with. A node can have multiple subjects, so we've encoded them in the CSV as pipe delimited strings.
We can hack those apart easily enough. In PHP we'd do something like
$subjects = explode($string, '|');\n$terms = [];\nforeach ($subjects as $name) {\n $terms[] = \\Drupal::service('entity_type.manager')->getStorage('taxonomy_term')->create([\n ...\n 'vid' => 'subject',\n 'name' => $name,\n ...\n ]);\n}\n$node->set('field_subject', $terms);\nWith process plugins, that logic looks like
field_subject:\n -\n plugin: skip_on_empty\n source: subject\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: entity_generate\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: subject\nskip_on_empty process plugin, which we've already seen, followed by explode. The explode process plugin operates exactly like its PHP counterpart, taking an array and a delimiter as input. The combination of skip_on_empty and explode behave like a foreach loop on the explode results. If we have an empty string, nothing happens. If there's one or more pipe delimited subject names in the string, then entity_generate gets called for each name that's found. The entity_generate process plugin will try to look up a subject by name, and if that fails, it creates one using the name and saves a reference to it in the node. So entity_generate is actually smarter than our pseudo-code above, because it can be run over and over again and it won't duplicate entities. :champagne:"},{"location":"technical-documentation/migrate-csv/#complex-fields","title":"Complex Fields","text":"Some fields don't hold just a single type of value. In other words, not everything is just text, numbers, or references. Using Drupal 8's Typed Data API, fields can hold groups of named values with different types. Consider a field that holds an RGB color. You could set it with PHP like so:
$node->set('field_color', ['R' => 255, 'G' => 255, 'B' => 255]);\nYou could even have a multi-valued color field, and do something like this
$node->set('field_color', [\n ['R' => 0, 'G' => 0, 'B' => 0],\n ['R' => 255, 'G' => 255, 'B' => 255],\n]);\nIn the migrate framework, you have two options for handling these types of fields. You can build up the full array they're expecting, which is difficult and requires a custom process plugin. Or, if you only have one value going into a complex field, you can set each named component in the field with separate process pipelines.
In controlled_access_terms, we define a new field type of typed_relation, which is an entity reference coupled with a MARC relator. It expects an associative array that looks like this:
[ 'target_id' => 1, 'rel_type' => 'relators:ctb']\nThe target_id portion takes an entity id, and rel_type takes the predicate for the MARC relator we want to use to describe the relationship the entity has with the repository item. This example would reference taxonomy_term 1 and give it the relator for \"Contributor\".
If we have a single name to deal with, we can set those values in YAML, accessing field_linked_agent/target_id and field_linked_agent/rel_type independently.
field_linked_agent/target_id:\n plugin: entity_generate\n source: photographer\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n\n field_linked_agent/rel_type: constants/relator\nHere we're looking at the photographer column in the CSV, which contains the names of the photographers that captured these images. Since we know these are photographers, and not publishers or editors, we can bake in the relator constant we set to relators:pht in the source section of the migration. So all that's left to do is to set the taxonomy term's id via entity_generate. If the lookup succeeds, the id is returned. If it fails, a term is created and its id is returned. In the end, by using the / syntax to set properties on complex fields, everything gets wrapped up into that nice associative array structure for you automatically.
However, if you have multiple names, and/or multiple columns that contain values that will go into the same typed_relation field, you do need to build up a full array of structured data.
We start by extracting the values in the photographer column, and busting them into a list. (In this case, given the sample data, the lists will all have length 1). Then, we use a custom process plugin to make each value the value in an associative array (see example data below).
photographers:\n -\n source: photographer\n plugin: skip_on_empty\n method: process\n -\n plugin: explode\n delimiter: '|'\n -\n plugin: str_to_assoc\n key: 'name'\nSo, if we started with a column containing
'Alice|Bob|Charlie'\nphotographers temporary variable would contain [\n ['name' => 'Alice'],\n ['name' => 'Bob'],\n ['name' => 'Charlie]\n]\nNext, we use the sub_process plugin. It takes an array of associative arrays (as seen above) and iterates over them. From within subprocess' process parameter, you can access only what's defined in that associative array. Here, when we do our entity_generate lookup, our source is name, the (only) key in that array.
linked_agent_pht:\n plugin: sub_process\n source: '@photographers'\n process:\n target_id:\n plugin: entity_generate\n source: name\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: person\n rel_type:\n plugin: default_value\n default_value: 'relators:pht'\nsub_process, we cannot access the temporary variables or constants that we've created in the outer migration. This is why we use the default_value plugin when for the rel_type. It would have been simpler to define a constant as we did with 'Image', but we wouldn't be able to access it. The output of this pipeline is now formatted as the structured data expected by a typed_relation field: [\n ['target_id' => 42, 'rel_type' => 'relators:pht' ],\n ['target_id' => 43, 'rel_type' => 'relators:pht' ],\n ['target_id' => 44, 'rel_type' => 'relators:pht' ],\n]\nfield_linked_agent. We don't have to assign the sub-components of field_linked_agent here, because this is already the structured data it is looking for. field_linked_agent:\n plugin: merge\n source:\n - '@linked_agent_pht'\n - '@linked_agent_prv'\nClear as mud? Great. Now let's run that migration.
"},{"location":"technical-documentation/migrate-csv/#running-the-node-migration","title":"Running the node migration","text":"Like with the file migration, run drush -y --userid=1 --uri=http://localhost:8000 migrate:import node from anywhere within the Drupal installation directory will fire off the migration. Go to http://localhost:8000/admin/content and you should see five new nodes. Click on one, though, and you'll see it's just a stub with metadata. The CSV metadata is there, links to other entities like subjects and photographers are there, but there's no trace of the corresponding files. Here's where media entities come into play.
Media entities are Drupal's solution for fieldable files. Since you can't put fields on a file, what you can do is wrap the file with a Media entity. In addition to a reference to the file (binary), technical metadata and structural metadata for the file go on the Media entity (e.g. MIME type, file size, resolution). Media also have a few special fields that are required for Islandora, field_media_of and field_use, which denote what node owns the media and what role the media serves, respectively. Since the Media entity references both the file it wraps and the node that owns it, Media entities act as a bridge between files and nodes, tying them together. And to do this, we make use of one last process plugin, migration_lookup. Open up /var/www/html/drupal/web/modules/contrib/migrate_islandora_csv/config/install/migrate_plus.migration.media.yml and give it a look.
# Uninstall this config when the feature is uninstalled\ndependencies:\n enforced:\n module:\n - migrate_islandora_csv\n\nid: media\nlabel: Import Media from CSV\nmigration_group: migrate_islandora_csv\n\nsource:\n plugin: csv\n path: modules/contrib/migrate_islandora_csv/data/migration.csv\n\n # 1 means you have a header row, 0 means you don't\n header_row_count: 1\n\n # Each migration needs a unique key per row in the csv. Here we're using the file path.\n ids:\n - file\n\n # You can't enter string literals into a process plugin, but you can give it a constant as a 'source'.\n constants:\n # We're tagging our media as Original Files\n use: Original File\n\n # Everything gets created as admin\n uid: 1\n\nprocess:\n\n name: title\n uid: constants/uid\n\n # Make the media an 'Original File'\n field_media_use:\n plugin: entity_lookup\n source: constants/use\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_media_use\n\n # Lookup the migrated file in the file migration.\n field_media_image:\n plugin: migration_lookup\n source: file\n migration: file\n no_stub: true\n\n # Lookup the migrated node in the node migration\n field_media_of:\n plugin: migration_lookup\n source: file\n migration: node\n no_stub: true\n\ndestination:\n # These are 'image' media we're making.\n plugin: 'entity:media'\n default_bundle: image\n\nmigration_dependencies:\n required:\n - migrate_plus.migration.file\n - migrate_plus.migration.node\n optional: { }\nThe Breakdown
Compared to the other migrations, this one is very straightforward. There's no string or array manipulation in YAML, and at most there's only one process plugin per field. Title and user are set directly, with no processing required
name: title\n uid: constants/uid\nfield_media_use field is a tag that's used to denote the purpose of a file with regard to the node it belongs to. E.g. is this the original file? a lower quality derivative? thumbnail? etc... In many ways it bears a resemblance to DSID in Islandora 7.x. Like field_model with nodes, the vocabulary already exists in your Islandora install, so all you have to do is look it up with the entity_lookup plugin. # Make the media an 'Original File'\n field_media_use:\n plugin: entity_lookup\n source: constants/use\n entity_type: taxonomy_term\n value_key: name\n bundle_key: vid\n bundle: islandora_media_use\nfield_media_image and field_media_of fields are how the media binds a file to a node. You could use entity_lookup or entity_generate, but we've already migrated them and can very easily look them up by the id assigned to them during migration. But what's the benefit of doing so? The entity_lookup and entity_generate process plugins do the job fine, right? The main advantage of using migration_lookup and defining migrations whenever possible, is that migrated entities can be rolled back. If you were to hop into your console and execute
drush -y --uri=http://localhost:8000 migrate:rollback --group migrate_islandora_csv\nmigration_lookup to set entity reference fields."},{"location":"technical-documentation/migrate-csv/#running-the-media-migration","title":"Running the media migration","text":"Run drush -y --uri=http://localhost:8000 migrate:import media from anywhere within the Drupal installation directory. You should now be able to see the media files attached to the nodes you created earlier. At this point, you might want to create derivatives, such as thumbnails, using the appropriate Drupal actions on the main content admin window.
If you've made it all the way to the end here, then you've learned that you can migrate files and CSV metadata into Islandora using only YAML files. You've seen how to transform data with pipelines of processing plugins and can handle numeric, text, and entity reference fields. You can handle multiple values for fields, and even more complicated things like typed_relation fields. And as big as this walkthrough was, we're only scratching the surface of what can be done with the Migrate API.
There's certainly more you can do with Drupal 8's Migrate API. There's a plethora of source and processing plugins out there that can handle pretty much anything you throw at it. XML and JSON are fair game. You can also request sources using HTTP, so you can always point it at an existing systems REST API and go from there. If you can't make the Migrate API's existing workflow make the necessary changes to your data, you can expand its capabilities by writing your own process plugins. Reading the Drupal.org documentation on the Migrate API would be a good place to start.
But really the best thing to do is try and get your data into Islandora! We intend to create a boilerplate branch of this repository that will allow you to clone down a migration template, ready for you to customize to fit your data. And as you assemble it into CSV format, keep in mind that if you have more than just names for things like subjects and authors, that you can always make more CSVs. Think of it like maintaining tables in an SQL database. Each CSV has unique keys, so you can lookup/join entities between CSVs using those keys. And you can still pipe-delimit the keys like we did in our example to handle multi-valued fields.
In some repositories, these CSVs can be used to make bulk updates to metadata. Just make your changes to maintain the CSVs, then run the migration(s) again with the --update flag. This will not always be efficient, as you'll update every entity, even if it didn't change. But, by breaking down your CSVs per collection or object type, you could keep them small enough to use this process for a small repository.
There is also a tool for migrating directly from a Legacy Islandora to Islandora (migrate_7x_claw), using Solr, XPaths, and Fedora calls to pull files and objects directly into Islandora. It may be worth checking out, and/or using in conjunction with a CSV migration.
"},{"location":"technical-documentation/migration-islandora-workbench/","title":"Islandora Workbench","text":""},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench","title":"Islandora Workbench","text":"Islandora Workbench provides a command-line solution to create, update, and delete Islandora content from CSV data. The Islandora Workbench repository can be found on Github. Full details and documentation for Islandora Workbench are maintained separately (Islandora Workbench Documentation).
"},{"location":"technical-documentation/migration-islandora-workbench/#overview","title":"Overview","text":"Islandora Workbench is tailored towards end users with less technical knowledge or limited server access.
Workbench provides an alternative to Drupal's Migrate framework, as it does not need to be run on the Drupal server. Islandora Workbench uses Islandora's REST API and offers cross-platform support (Windows, Mac, Linux) to run on your computer, using your provided CSVs and files.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench-highlights","title":"Islandora Workbench highlights","text":"Islandora Workbench:
Using Islandora Workbench, you can assign both existing and new taxonomy terms to nodes. Within the CSVs you provide, the values of the taxonomy field/columns can:
If you need to create terms with multiple fields (such as an External URI) or which use term hierarchy, you can create terms in a separate task.
"},{"location":"technical-documentation/migration-islandora-workbench/#other-field-types","title":"Other Field Types","text":"There are multiple ways to create paged content with Islandora Workbench. More information on each option is available here. You may:
Click the video previews shown below to open the corresponding video in Youtube.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandoracon-2022-migration-strategies","title":"IslandoraCon 2022: Migration Strategies","text":"This video (August 2022) is an overview of how to harvest data from an existing Islandora 7 site using Workbench, how to sanitize and prep that data with custom Python tools, and how to pull that data into a new Islandora 2 site.
"},{"location":"technical-documentation/migration-islandora-workbench/#summer-of-islandora-workbench-introduction-to-islandora-workbench","title":"Summer of Islandora Workbench: Introduction to Islandora Workbench","text":"This video (July 29, 2021) is an introduction of Islandora Workbench.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-workbench-demo","title":"Islandora Workbench Demo","text":"This video (Dec 17, 2020) is a demo of Islandora Workbench.
"},{"location":"technical-documentation/migration-islandora-workbench/#islandora-online-islandora-migration-tools","title":"Islandora Online: Islandora Migration Tools","text":"This video (Aug 10, 2020) provides an overview of the Islandora Workbench and the two other options available to migrate data into an Islandora installation.
"},{"location":"technical-documentation/migration-migrate-api/","title":"Migrate API","text":""},{"location":"technical-documentation/migration-migrate-api/#migrate-api","title":"Migrate API","text":"Uses the Drupal Migrate API, which \"provides services for migrating data from a source system\" to Drupal 8, 9, or 10.
The \"source system\" can be almost anything:
Why use the Migrate API?
drush mim node --updateCheck out this video that demonstrates the Drupal Migrate API migration process: Islandora Webinar: Migrating from Islandora Legacy to Islandora (Nov 21, 2019)
"},{"location":"technical-documentation/migration-overview/","title":"Migration Overview","text":""},{"location":"technical-documentation/migration-overview/#migration-and-batch-loading","title":"Migration and Batch Loading","text":"This video (Aug 10, 2020) provides an overview of the various options available to migrate data into an Islandora installation.
The three main migration options are:
Why use the rest API?
Visit the REST API migration documentation section for more details.
"},{"location":"technical-documentation/migration-overview/#migrate-api","title":"Migrate API","text":"Why use the Migrate API?
drush mim node --updateTwo tools that use the Migrate API are migrate_islandora_csv and migrate_7x_claw.
Visit the Migrate API migration documentation section for more details.
"},{"location":"technical-documentation/migration-overview/#islandora-workbench","title":"Islandora Workbench","text":"Why use the Migrate API?
Visit the Islandora Workbench migration documentation section for more details.
"},{"location":"technical-documentation/migration-rest-api/","title":"REST API","text":""},{"location":"technical-documentation/migration-rest-api/#rest-api","title":"REST API","text":"Why use the REST API?
BONUS: It\u2019s just Drupal\u2019s REST API
"},{"location":"technical-documentation/migration-rest-api/#islandora-only-provides-two-additional-api-endpoints","title":"Islandora only provides two additional API endpoints","text":"Just be aware, you are writing everything yourself! (In other words you are making all of the migration decisions yourself.)
"},{"location":"technical-documentation/migration-rest-api/#videos","title":"Videos","text":"This video (Aug 10, 2020) provides an overview of the REST API and the two other options available to migrate data into an Islandora installation.
"},{"location":"technical-documentation/ppa-documentation/","title":"Updating adeb and adding it to LYRASIS PPA","text":""},{"location":"technical-documentation/ppa-documentation/#this-page-is-here-for-archival-purposes","title":"This page is here for archival purposes.","text":"If you are running Ubuntu 20.04 or higher, installing a specially-compiled imagemagick is no longer necessary. We are leaving this page available in case in the future we have a need to package our own deb and add it to the Lyrasis PPA.
Ubuntu removed JPEG 2000 support from ImageMagick since Vivid Vervet's 8:6.8.9.9-5 version. The PPA that we have created adds JPEG 2000 support back into ImageMagick for Ubuntu 16.04 and 18.04. More information on why JPEG 2000 support was removed can be found here, and more information on openjpeg in Ubuntu can be found here.
Review the Ubuntu Packaging Guide. Key items needed are in the Getting Set Up section:
sudo apt install gnupg pbuilder ubuntu-dev-tools apt-filepbuilder:pbuilder-dist bionic createdebuild:imagemagick is used an example. If you need to only change or patch the actual source code, then you will need to use quilt. More information on using quilt can be found in the Patches to Packages section. If you are only change dependencies, or information in the debian directory of a package, quilt is not required, and if used will cause build failures.
mkdir imagemagick-bionic-jp2cd imagemagick-bionic-jp2pull-lp-source imagemagick bioniccd imagemagick-6.9.7.4+dfsgdch (Make sure to change UNRELEASED to the Ubuntu release name. For example: bionic)debuild -SIndex: imagemagick-6.9.7.4+dfsg/debian/control\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control\n+++ imagemagick-6.9.7.4+dfsg/debian/control\n@@ -26,8 +26,7 @@ Build-Depends: debhelper (>= 10),\n libx11-dev, libxext-dev, libxt-dev,\n # for plugins\n ghostscript, libdjvulibre-dev, libexif-dev,\n- libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libjpeg-dev, libopenjp2-7-dev,\n libopenexr-dev, libperl-dev, libpng-dev, libtiff-dev,\n libwmf-dev,\n # libgraphviz-dev, incompatible license against fftw\n@@ -273,8 +272,7 @@ Depends: libmagickcore-6-headers (= ${so\n libmagickcore-6.q16-3 (= ${binary:Version}),\n libmagickcore-6.q16-3-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\n@@ -483,8 +481,7 @@ Depends: libmagickcore-6-headers (= ${so\n libmagickcore-6.q16hdri-3 (= ${binary:Version}),\n libmagickcore-6.q16hdri-3-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\nIndex: imagemagick-6.9.7.4+dfsg/debian/control.d/noquantum.in\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control.d/noquantum.in\n+++ imagemagick-6.9.7.4+dfsg/debian/control.d/noquantum.in\n@@ -26,8 +26,7 @@ Build-Depends: debhelper (>= 10),\n libx11-dev, libxext-dev, libxt-dev,\n # for plugins\n ghostscript, libdjvulibre-dev, libexif-dev,\n- libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libjpeg-dev, libopenjp2-7-dev,\n libopenexr-dev, libperl-dev, libpng-dev, libtiff-dev,\n libwmf-dev,\n # libgraphviz-dev, incompatible license against fftw\nIndex: imagemagick-6.9.7.4+dfsg/debian/control.d/quantum.in\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/control.d/quantum.in\n+++ imagemagick-6.9.7.4+dfsg/debian/control.d/quantum.in\n@@ -78,8 +78,7 @@ Depends: libmagickcore-${IMVERSION}-head\n libmagickcore-${IMVERSION}.${QUANTUMDEPTH}-${CORESOVERSION} (= ${binary:Version}),\n libmagickcore-${IMVERSION}.${QUANTUMDEPTH}-${CORESOVERSION}-extra (= ${binary:Version}),\n libbz2-dev, libdjvulibre-dev,\n- libexif-dev, libfreetype6-dev, libjpeg-dev,\n-# libopenjp2-7-dev, Needed for JPEG 2000 but not in main see MIR #711061\n+ libexif-dev, libfreetype6-dev, libjpeg-dev, libopenjp2-7-dev,\n liblcms2-dev, liblqr-1-0-dev, libltdl-dev, libopenexr-dev, libpng-dev,\n librsvg2-dev, libtiff-dev, libwmf-dev, libx11-dev, libxext-dev, libxml2-dev,\n libxt-dev, zlib1g-dev,\nIndex: imagemagick-6.9.7.4+dfsg/debian/rules\n===================================================================\n--- imagemagick-6.9.7.4+dfsg.orig/debian/rules\n+++ imagemagick-6.9.7.4+dfsg/debian/rules\n@@ -98,7 +98,7 @@ STATIC_CONFIGURE_OPTIONS := \\\n --with-gs-font-dir=/usr/share/fonts/type1/gsfonts \\\n --with-magick-plus-plus \\\n --with-djvu \\\n- --with-openjp2 \\\n+ --with-openjp2 \\\n --with-wmf \\\n --without-gvc \\\n --enable-shared \\\nOnce the package is built successfully, you'll upload the changes file to Launchpad. For example:
dput ppa:lyrasis/imagemagick-jp2 imagemagick_6.9.7.4+dfsg-16ubuntu6.8_source.changesAfter the package is successfully uploaded to the PPA, you'll receive an email from Launchpad. Something like this:
Accepted:\n OK: imagemagick_6.9.7.4+dfsg.orig.tar.xz\n OK: imagemagick_6.9.7.4+dfsg-16ubuntu6.8.debian.tar.xz\n OK: imagemagick_6.9.7.4+dfsg-16ubuntu6.8.dsc\n -> Component: main Section: graphics\n\nimagemagick (8:6.9.7.4+dfsg-16ubuntu6.8) bionic; urgency=medium\n\n * Add jp2 support.\nAnd you will see it in the interface for the Lyrasis PPA:
"},{"location":"technical-documentation/resizing-vm/","title":"Resizing a VM","text":""},{"location":"technical-documentation/resizing-vm/#resize-vagrant-machine","title":"Resize vagrant machine","text":"To expand virtual machine's hard drive for testing of larger files. Once the VM has started, you'll need to halt the VM, download and run the script, tell it what size (in MB) and then start the VM. The last step vagrant ssh --command \"sudo resize2fs /dev/sda1\" is a check. It should return there was nothing to do. If you already provisioned you VM you can skip the 2 steps with provisioning in them.
# Skip this if you VM is already provisioned.\n$ vagrant up --no-provision <-- Exclude if already running and provisioned.\n\n$ vagrant halt\n\n# Download and run. This will default to the correct name (just press enter) then give the size.\n# Example: `350000` is equal to 350GB\n$ wget https://gist.githubusercontent.com/DonRichards/6dc6c81ae9fc22cba8d7a57b90ab1509/raw/45017e07a3b93657f8822dfbbe4fc690169cdabc/expand_disk.py\n$ chmod +x expand_disk.py\n$ python expand_disk.py\n$ vagrant up --no-provision\n\n# This step isn't needed but acts as a check to verify it worked.\n$ vagrant ssh --command \"sudo resize2fs /dev/sda1\"\n# Skip this if you VM is already provisioned.\n$ vagrant provision <-- Exclude if already provisioned.\nYou may need to remove the \"resized\" version. Assuming your VM location is ~/VirtualBox\\ VMs
$ rm -rf ~/VirtualBox\\ VMs/Islandora\\ CLAW\\ Ansible_resized\nIf your resources are restricted (hidden) you will need to have authorization to access them.
You can specify which types of authentication are allowed for which HTTP methods.
These are common to all HTTP methods against the REST API.
In the above screenshot we have 3 allowed methods. 1. basic_auth 1. jwt_auth 1. cookie
"},{"location":"technical-documentation/rest-authorization/#basic-authentication-basic_auth","title":"Basic authentication (basic_auth)","text":"To use basic authentication with a client like cURL use the -u username:password argument.
For example:
curl -u admin:islandora http://localhost:8000/node/3\nBy default JWTs are passed internally from Drupal to various microservices and Fedora.
To use a JWT yourself you need to enable the JWT Authentication Issuer module.
Once enabled this module makes a /jwt/token endpoint. You can perform a GET against this endpoint as an authenticated user to receive a JWT.
For example:
curl -i -u admin:islandora http://localhost:8000/jwt/token\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 22:08:37 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 620\nContent-Type: application/json\n\n{\n \"token\" : \"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE1NTE3MzczMTcsImV4cCI6MTU1MTc0NDUxNywiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.QUTrMiK_DyBxqQY4LnibLYtieEW3-MyjjQO9NSFI7bPylNm1S5ZY0uvzjDob3ckYgRN4uCyMFZO4BPytpQVA_jyeSuZyUA_10v33ItpoKyjrJ_S057iykNd_rWmxe8tT8T1fPypq_-Z7Th_PkyZWrYBqoBBVO1iVQt5txxfGWMqhxd2FgsXw6N-aR9sYOSc4UrLmFRmPP5Zk_CNIZP6NtBaM9JNr8CnWyPEFSAR75xfyH3ge5qjBqLlDS389k07pyJFB5rOT59txzLE9WLvpp9JK3oQv821Q1Bp-PMiASghXc0dBCHxM8o41BzLE88UstRA7agBAkUqG3hMpoNZqfA\"\n}\nYou can then take the same token and re-use it.
curl -H\"Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE1NTE3MzczMTcsImV4cCI6MTU1MTc0NDUxNywiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.QUTrMiK_DyBxqQY4LnibLYtieEW3-MyjjQO9NSFI7bPylNm1S5ZY0uvzjDob3ckYgRN4uCyMFZO4BPytpQVA_jyeSuZyUA_10v33ItpoKyjrJ_S057iykNd_rWmxe8tT8T1fPypq_-Z7Th_PkyZWrYBqoBBVO1iVQt5txxfGWMqhxd2FgsXw6N-aR9sYOSc4UrLmFRmPP5Zk_CNIZP6NtBaM9JNr8CnWyPEFSAR75xfyH3ge5qjBqLlDS389k07pyJFB5rOT59txzLE9WLvpp9JK3oQv821Q1Bp-PMiASghXc0dBCHxM8o41BzLE88UstRA7agBAkUqG3hMpoNZqfA\" http://localhost:8000/node/3?_format=jsonld\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 22:10:02 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/3>; rel=\"canonical\"\nLink: <http://localhost:8000/node/3/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=3>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/3/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/3/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/3>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=3>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/3?_format=json>; rel=\"alternate\"; type=\"application/json\"\nX-Drupal-Dynamic-Cache: HIT\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 858\nContent-Type: application/ld+json\n\n{\n \"@graph\": [\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/node\\\\/3?_format=jsonld\",\n \"@type\": [\n \"http:\\\\/\\\\/pcdm.org\\\\/models#Object\"\n ],\n \"http:\\\\/\\\\/purl.org\\\\/dc\\\\/terms\\\\/title\": [\n {\n \"@value\": \"Custom item\",\n \"@language\": \"en\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/author\": [\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/user\\\\/1?_format=jsonld\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/dateCreated\": [\n {\n \"@value\": \"2019-03-01T19:42:54+00:00\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#dateTime\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/dateModified\": [\n {\n \"@value\": \"2019-03-01T19:43:12+00:00\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#dateTime\"\n }\n ],\n \"http:\\\\/\\\\/purl.org\\\\/dc\\\\/terms\\\\/extent\": [\n {\n \"@value\": \"1 item\",\n \"@type\": \"http:\\\\/\\\\/www.w3.org\\\\/2001\\\\/XMLSchema#string\"\n }\n ],\n \"http:\\\\/\\\\/schema.org\\\\/sameAs\": [\n {\n \"@value\": \"http:\\\\/\\\\/localhost:8000\\\\/node\\\\/1?_format=jsonld\"\n }\n ]\n },\n {\n \"@id\": \"http:\\\\/\\\\/localhost:8000\\\\/user\\\\/1?_format=jsonld\",\n \"@type\": [\n \"http:\\\\/\\\\/schema.org\\\\/Person\"\n ]\n }\n ]\n}\nThis allows you to use a cookie stored in your web browser when you log in to Drupal to access these REST endpoint pages.
This is what allows you to access the URIs like http://localhost:8000/node/1?_format=json with your web browser.
You will need to use one of the configured authorization methods to create content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-create/#content-nodes","title":"Content (Nodes)","text":"The above setup shows a configuration where the JSON format is enabled for GET, PATCH, DELETE, and POST, with authentication types \"basic_auth\" and \"jwt_auth\" enabled for each method. Thus, with this configuration, you can perform a POST request against a node at the /node endpoint with a body in the JSON format.
To create a node, you need to provide two elements in your message body: the node type and any required field values.
For the Repository Item content type included in the Islandora Starter Site, these are:
A good way to make your first POST request is to perform a GET request against an existing node and erase all the extra content.
You can find more information about GET requests here.
Again we are using the json format.
\ud83c\udf4e For example curl -X GET 'http://localhost:8000/node/3?_format=json
Look for the type element
\"type\" : [\n {\n \"target_id\" : \"islandora_object\",\n \"target_type\" : \"node_type\",\n \"target_uuid\" : \"62189bec-3ef3-4196-b847-b17e5ce61fd5\"\n }\n]\nIn our example \"islandora_object\" is the machine name of the content type \"Repository Item\". If you have created a new type you will have a different target_id.
You will not need the target_uuid.
Next look for the title element
\"title\" : [\n {\n \"value\" : \"An example Islandora object\"\n }\n]\nLastly look for the field_model element
\"field_model\": [\n {\n \"target_id\": 24,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e7560b68-e95a-4e76-9671-2a3041cd9800\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/24\"\n }\n]\nYou can find the models by browsing the taxonomy terms available at http://localhost:8000/admin/structure/taxonomy/manage/islandora_models/overview
In my example installation, term 24 is an \"Image\", but let's create a collection which is term 23.
Note: Taxonomy terms may vary between instances and you should verify the correct number for your installation.
So the body of the request will be:
{\n \"type\": [\n {\n \"target_id\": \"islandora_object\",\n \"target_type\": \"node_type\"\n }\n ],\n \"title\": [\n {\n \"value\": \"Created a collection with POST\"\n }\n ],\n \"field_model\": [\n {\n \"target_id\": 23,\n \"target_type\": \"taxonomy_term\"\n }\n ]\n}\nNote: You must include an appropriate Content-type header for the format you're requesting
Other Note: You must include some authentication credentials to say who you are and so Drupal can check if you are allowed to create this object. Otherwise you will receive a 401 Unauthorized response.
If you do provide credentials but don't have permission, you will receive a 403 Forbidden response.
You can find more information about Authorization here
\ud83c\udf4e For example:
curl -i -X POST -u admin:islandora -H\"Content-type: application/json\" --data '{\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\"}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\"}]}' 'http://localhost:8000/node?_format=json'\n\nHTTP/1.1 201 Created\nDate: Tue, 05 Mar 2019 18:07:00 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nLocation: http://localhost:8000/node/3\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1564\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":3}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nThe parts of the above request are:
-i - return the response headers-X POST - send a POST request-u admin:islandora - use these basic authentication credentials-H\"Content-type: application/json\" - send the content-type header--data '{...}' - send the request body (seen above)'http://localhost:8000/node?_format=json' - the endpoint of the requestThe Drupal REST UI is supposed to have a way to upload files, but this seems to require the use of an X-CSRF-Token, which can only be retrieved using Cookie authentication and even then does not allow you to upload.
However there is a special REST endpoint created by Islandora, which is less configurable and is not part of the above-mentioned REST UI.
This endpoint is available at http://localhost:8000/node/{node id}/media/{media type}/{media use}
It only accepts PUT requests. If the media and file don't exist they are created, if they exist the file is updated with the new body.
The node id and taxonomy term id are used to search (via an entity query) for a matching media. If this media exists the body of the file is replaced with the new content, otherwise a new file and media are created to hold the content.
The tokens to this URI are as follows:
You can find the media use taxonomy terms at http://localhost:8000/admin/structure/taxonomy/manage/islandora_media_use/overview
The body of the request is the actual binary file to upload.
\ud83c\udf4e For example:
With a local file called my-image.png that I wanted to link to a node with ID 3.
I am using the taxonomy term \"Original file\", which on my machine is 16
> curl -i -X PUT -u admin:islandora -H\"Content-type: image/png\" --data-binary \"@my-image.png\" -H\"Content-Location: public://images/my-image.png\" 'http://localhost:8000/node/3/media/image/16'\n\nHTTP/1.1 100 Continue\n\nHTTP/1.1 201 Created\nDate: Tue, 05 Mar 2019 22:01:39 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLocation: http://localhost:8000/media/4\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 0\nContent-Type: text/html; charset=UTF-8\nThe parts of the above request are:
-i - return the response headers-X PUT - send a PUT request-u admin:islandora - use these basic authentication credentials-H\"Content-type: image/png\" - send the content-type header--data-binary \"@my-image.png\" - send the contents of the file located at my-image.png as binary-H\"Content-Location: public://images/my-image.png\" - store the file in the public scheme (ie. in Drupal) at the path images/my-image.png, to store the file in Fedora use the fedora// scheme (ie. fedora://images/my-image.png)'http://localhost:8000/node/3/media/image/16' - the endpoint of the request specifying the node, media type and taxonomy term.Deleting is as easy as getting resources and more difficult than creating resources.
If you can perform a GET request then you have the information required to perform a DELETE request.
Check out the examples below.
If you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-delete/#content-nodes","title":"Content (Nodes)","text":"You will need your node id, you can find more information in the GET documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
Our example node has a node id of 2
> curl -i -u admin:islandora -X GET 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 15:02:00 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/2>; rel=\"canonical\"\nLink: <http://localhost:8000/node/2/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=2>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/2/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/2/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/2>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=2>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/dc/dcmitype/Collection>; rel=\"tag\"; title=\"Collection\"\nLink: <http://localhost:8000/node/2?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1564\nContent-Type: application/json\n\n{\"nid\":[{\"value\":2}],\"uuid\":[{\"value\":\"413135a6-0bd1-4d6b-8bcb-059cf7784d83\"}],\"vid\":[{\"value\":2}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Created a collection with POST\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-05T18:04:43+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}%\nThen we switch GET to DELETE
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 15:02:30 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nAll subsequent requests to the above URI will return a 404 Not Found status code.
> curl -i -u admin:islandora -X GET 'http://localhost:8000/node/2?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 15:12:58 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 149\nContent-Type: application/json\n\n{\"message\":\"The \\u0022node\\u0022 parameter was not converted for the path \\u0022\\/node\\/{node}\\u0022 (route name: \\u0022rest.entity.node.GET\\u0022)\"}\nYou will need a media id as used in GET documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
With a media id of 1 for our example, I'll perform a GET
> curl -i -u admin:islandora -X GET 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 14:53:54 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/media/add>; rel=\"https://drupal.org/link-relations/add-page\"\nLink: <http://localhost:8000/media/add/image>; rel=\"https://drupal.org/link-relations/add-form\"\nLink: <http://localhost:8000/media/1>; rel=\"canonical\"\nLink: <http://localhost:8000/admin/content/media>; rel=\"collection\"\nLink: <http://localhost:8000/media/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/media/delete?media=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/media/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/media/1>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node/1>; rel=\"related\"; title=\"Media of\"\nLink: <http://pcdm.org/use#OriginalFile>; rel=\"tag\"; title=\"Original File\"\nLink: <http://localhost:8000/media/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nLink: <http://localhost:8000/media/1/source>; rel=\"edit-media\"\nLink: <http://localhost:8000/_flysystem/fedora/2019-03/Louis_Riel.jpg>; rel=\"describes\"; type=\"image/jpeg\"\nX-Drupal-Dynamic-Cache: HIT\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1937\nContent-Type: application/json\n\n{\"mid\":[{\"value\":1}],\"uuid\":[{\"value\":\"d8893926-ddb7-4125-b2da-30428af0fe3d\"}],\"vid\":[{\"value\":1}],\"langcode\":[{\"value\":\"en\"}],\"bundle\":[{\"target_id\":\"image\",\"target_type\":\"media_type\",\"target_uuid\":\"3860e653-201b-4509-89dd-628c446d81cb\"}],\"revision_created\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_user\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log_message\":[],\"status\":[{\"value\":true}],\"name\":[{\"value\":\"An image\"}],\"thumbnail\":[{\"target_id\":2,\"alt\":\"A portrait of Louis Riel\",\"title\":null,\"width\":800,\"height\":1333,\"target_type\":\"file\",\"target_uuid\":\"b0625129-c592-463a-93c3-3eff7cd3567e\",\"url\":\"http:\\/\\/localhost:8000\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-01T19:43:22+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_file_size\":[{\"value\":166613}],\"field_height\":[{\"value\":1333}],\"field_media_image\":[{\"target_id\":2,\"alt\":\"A portrait of Louis Riel\",\"title\":\"\",\"width\":800,\"height\":1333,\"target_type\":\"file\",\"target_uuid\":\"b0625129-c592-463a-93c3-3eff7cd3567e\",\"url\":\"http:\\/\\/localhost:8000\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"field_media_of\":[{\"target_id\":1,\"target_type\":\"node\",\"target_uuid\":\"8322e36e-f8ec-4fd9-919d-52aed7b17a52\",\"url\":\"\\/node\\/1\"}],\"field_media_use\":[{\"target_id\":16,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"08e01ff9-eb72-42f5-ae3a-8b21ba0c0bc3\",\"url\":\"\\/taxonomy\\/term\\/16\"}],\"field_mime_type\":[{\"value\":\"image\\/jpeg\"}],\"field_width\":[{\"value\":800}]}\nThen we replace GET with DELETE.
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 14:54:55 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nSubsequent requests to the media return 404 Not Found statuses.
> curl -i -u admin:islandora -X GET 'http://localhost:8000/media/1?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 15:13:45 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 153\nContent-Type: application/json\n\n{\"message\":\"The \\u0022media\\u0022 parameter was not converted for the path \\u0022\\/media\\/{media}\\u0022 (route name: \\u0022rest.entity.media.GET\\u0022)\"}\nYou'll need the file id, there is more information at the top of the GET requests for files documentation.
A delete is simply the same request as a GET but sending a DELETE http verb.
With a file id of 2 for our example, I can perform a test GET
> curl -i -u admin:islandora 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 200 OK\nDate: Fri, 15 Mar 2019 14:40:40 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 617\nContent-Type: application/json\n\n{\"fid\":[{\"value\":2}],\"uuid\":[{\"value\":\"b0625129-c592-463a-93c3-3eff7cd3567e\"}],\"langcode\":[{\"value\":\"en\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"filename\":[{\"value\":\"Louis_Riel.jpg\"}],\"uri\":[{\"value\":\"fedora:\\/\\/2019-03\\/Louis_Riel.jpg\",\"url\":\"\\/_flysystem\\/fedora\\/2019-03\\/Louis_Riel.jpg\"}],\"filemime\":[{\"value\":\"image\\/jpeg\"}],\"filesize\":[{\"value\":166613}],\"status\":[{\"value\":true}],\"created\":[{\"value\":\"2019-03-01T19:43:35+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-01T19:43:46+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}]}\nIf this is the correct file, I can delete it.
> curl -i -u admin:islandora -X DELETE 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 204 No Content\nDate: Fri, 15 Mar 2019 14:43:22 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Type: text/html; charset=UTF-8\nSubsequent requests to the URI now return a 404 Not Found status.
> curl -i -u admin:islandora 'http://localhost:8000/entity/file/2?_format=json'\n\nHTTP/1.1 404 Not Found\nDate: Fri, 15 Mar 2019 14:43:33 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 157\nContent-Type: application/json\n\n{\"message\":\"The \\u0022file\\u0022 parameter was not converted for the path \\u0022\\/entity\\/file\\/{file}\\u0022 (route name: \\u0022rest.entity.file.GET\\u0022)\"}\nIf you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
"},{"location":"technical-documentation/rest-get/#content-nodes","title":"Content (Nodes)","text":"The above setup shows that you can perform a GET request against a node at the /node/{id} endpoint. This is the same as the URL when viewed through a web browser. For example http://localhost:8000/node/2
It also defines the formats you can access. In this example json and jsonld are available.
To access a resource in a specific format use the _format= argument which is appended to the end of the URI after a ?.
To get the resource located at http://localhost:8000/node/3 in JSON, you can go to http://localhost:8000/node/3?_format=json.
\ud83c\udf4e For example:
You can do this in your web browser or with a command line client like cURL. (Note: response body here has been formatted, yours will look different)
vagrant@claw:~$ curl -i http://localhost:8000/node/3?_format=json\n\nHTTP/1.1 200 OK\nDate: Mon, 04 Mar 2019 21:53:50 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nLink: <http://localhost:8000/node/3>; rel=\"canonical\"\nLink: <http://localhost:8000/node/3/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=3>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/3/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/3/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/3>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=3>; rel=\"https://drupal.org/link-relations/create\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/3?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nX-Drupal-Cache: MISS\nContent-Length: 1521\nContent-Type: application/json\n\n{\n \"nid\": [\n {\n \"value\": 1\n }\n ],\n \"uuid\": [\n {\n \"value\": \"8322e36e-f8ec-4fd9-919d-52aed7b17a52\"\n }\n ],\n \"vid\": [\n {\n \"value\": 1\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"type\": [\n {\n \"target_id\": \"islandora_object\",\n \"target_type\": \"node_type\",\n \"target_uuid\": \"62189bec-3ef3-4196-b847-b17e5ce61fd5\"\n }\n ],\n \"revision_timestamp\": [\n {\n \"value\": \"2019-03-01T19:43:12+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"revision_uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"revision_log\": [],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"title\": [\n {\n \"value\": \"Custom item\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:42:54+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:12+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"promote\": [\n {\n \"value\": true\n }\n ],\n \"sticky\": [\n {\n \"value\": false\n }\n ],\n \"default_langcode\": [\n {\n \"value\": true\n }\n ],\n \"revision_translation_affected\": [\n {\n \"value\": true\n }\n ],\n \"content_translation_source\": [\n {\n \"value\": \"und\"\n }\n ],\n \"content_translation_outdated\": [\n {\n \"value\": false\n }\n ],\n \"field_alternative_title\": [],\n \"field_description\": [],\n \"field_display_hints\": [],\n \"field_edtf_date\": [],\n \"field_edtf_date_created\": [],\n \"field_edtf_date_issued\": [],\n \"field_extent\": [\n {\n \"value\": \"1 item\"\n }\n ],\n \"field_identifier\": [],\n \"field_linked_agent\": [],\n \"field_member_of\": [],\n \"field_model\": [\n {\n \"target_id\": 24,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e7560b68-e95a-4e76-9671-2a3041cd9800\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/24\"\n }\n ],\n \"field_pid\": [],\n \"field_resource_type\": [],\n \"field_rights\": [],\n \"field_subject\": []\n}\nThe above steps are the same for media, it is just the URI that has changed.
For media you use /media/{id}
\ud83c\udf4e For example:
curl -u admin:islandora http://localhost:8000/media/2?_format=json\n\n{\n \"mid\": [\n {\n \"value\": 2\n }\n ],\n \"uuid\": [\n {\n \"value\": \"e75e609d-510c-4764-9280-665bb7026161\"\n }\n ],\n \"vid\": [\n {\n \"value\": 2\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"bundle\": [\n {\n \"target_id\": \"image\",\n \"target_type\": \"media_type\",\n \"target_uuid\": \"3860e653-201b-4509-89dd-628c446d81cb\"\n }\n ],\n \"revision_created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"revision_user\": [],\n \"revision_log_message\": [],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"name\": [\n {\n \"value\": \"1-Service File.jpg\"\n }\n ],\n \"thumbnail\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"default_langcode\": [\n {\n \"value\": true\n }\n ],\n \"revision_translation_affected\": [\n {\n \"value\": true\n }\n ],\n \"content_translation_source\": [\n {\n \"value\": \"und\"\n }\n ],\n \"content_translation_outdated\": [\n {\n \"value\": false\n }\n ],\n \"field_access_terms\": [],\n \"field_file_size\": [\n {\n \"value\": 166608\n }\n ],\n \"field_height\": [\n {\n \"value\": 1333\n }\n ],\n \"field_media_image\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"field_media_of\": [\n {\n \"target_id\": 1,\n \"target_type\": \"node\",\n \"target_uuid\": \"8322e36e-f8ec-4fd9-919d-52aed7b17a52\",\n \"url\": \"\\\\/node\\\\/1\"\n }\n ],\n \"field_media_use\": [\n {\n \"target_id\": 18,\n \"target_type\": \"taxonomy_term\",\n \"target_uuid\": \"e8c698a7-8c15-47c4-8567-1c27f8f0c19e\",\n \"url\": \"\\\\/taxonomy\\\\/term\\\\/18\"\n }\n ],\n \"field_mime_type\": [\n {\n \"value\": \"image\\\\/jpeg\"\n }\n ],\n \"field_width\": [\n {\n \"value\": 800\n }\n ]\n}\nfield_media_of indicates which node(s) the media is associated with. The node can be referenced using the value of the target_id key (in this example, 1) or the target_uuid key (8322e36e-f8ec-4fd9-919d-52aed7b17a52).
Through the REST API you are accessing metadata about files, but not the actual binary object.
Based on the above configuration the path for a GET request is /entity/file/{id}
The file ID refers to the integer counter and not the UUID of a file.
Through the UI you can go to the File listing at http://localhost:8000/admin/content/files.
For the file you want to access, choose the USED IN link.
That URI has the format http://localhost:8000/admin/content/files/usage/{file id}
Alternatively the file ID can be found by viewing the media metadata (above section). In the above example if you look through the response from a GET request to the /media/{id} you will find a section like:
\"field_media_image\": [\n {\n \"target_id\": 3,\n \"alt\": null,\n \"title\": null,\n \"width\": 800,\n \"height\": 1333,\n \"target_type\": \"file\",\n \"target_uuid\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\",\n \"url\": \"http:\\\\/\\\\/localhost:8000\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\nNote: Depending on the \"type\" of file (image, video, audio, file) this field will have a different name.
This has both the url which is the URI to retrieve the binary content of the file as well as the target_id which is the file ID.
So we can retrieve the file with a GET request to the URI http://localhost:8000/sites/default/files/2019-03/1-Service%20File.jpg.
But more importantly we can retrieve the file metadata by making a request to http://localhost:8000/entity/file/3 with one of the allowed formats
\ud83c\udf4e For example:
> curl -i -X GET \"http://localhost:8000/entity/file/3?_format=json\"\n\nHTTP/1.1 200 OK\nDate: Tue, 05 Mar 2019 16:52:06 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-Drupal-Dynamic-Cache: MISS\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nX-Drupal-Cache: HIT\nContent-Length: 634\nContent-Type: application/json\n\n{\n \"fid\": [\n {\n \"value\": 3\n }\n ],\n \"uuid\": [\n {\n \"value\": \"e97ad038-dcb3-4d81-aa91-c945b2fe092c\"\n }\n ],\n \"langcode\": [\n {\n \"value\": \"en\"\n }\n ],\n \"uid\": [\n {\n \"target_id\": 1,\n \"target_type\": \"user\",\n \"target_uuid\": \"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\n \"url\": \"\\\\/user\\\\/1\"\n }\n ],\n \"filename\": [\n {\n \"value\": \"1-Service File.jpg\"\n }\n ],\n \"uri\": [\n {\n \"value\": \"public:\\\\/\\\\/2019-03\\\\/1-Service File.jpg\",\n \"url\": \"\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\"\n }\n ],\n \"filemime\": [\n {\n \"value\": \"image\\\\/jpeg\"\n }\n ],\n \"filesize\": [\n {\n \"value\": 166608\n }\n ],\n \"status\": [\n {\n \"value\": true\n }\n ],\n \"created\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ],\n \"changed\": [\n {\n \"value\": \"2019-03-01T19:43:48+00:00\",\n \"format\": \"Y-m-d\\\\TH:i:sP\"\n }\n ]\n}\nThe uri: url: is the path to your file, so for the above example of \"url\": \"\\\\/sites\\\\/default\\\\/files\\\\/2019-03\\\\/1-Service%20File.jpg\".
I can perform a GET against <my hostname>/sites/default/files/2019-03/1-Service%20File.jpg and get the contents of this file.
PATCH requests allow you to update resources in place via a REST call. In this case you send a few required fields and then any changed fields. PATCH requests are available for nodes and media entities, but not for files. PATCH requests are very consistent between node and media entities so we will just show one set of examples here.
Our example node is at http://localhost:8000/node/3.
If you perform a GET against another node or media you can see all of the fields, some of these are calculated (change based on others, i.e. \"changed\"), some are references to other entities (nodes, media, files in Drupal. i.e. \"field_model\") and others are pure text fields (i.e. \"title\").
The fields will be different between different node types and media types, but the syntax to update them is the same.
If you have restricted access to view your content, you will need to use one of the configured authorization methods to access your content, media and/or files.
These are defined under Authorization on the overview.
This with assume you have already created a node or media at some location.
"},{"location":"technical-documentation/rest-patch/#change-a-text-field","title":"Change a text field","text":"To change the title of a node, you need to supply the new title and the node_type. The node_type is required as this defines what fields are available to the node.
A successful PATCH request will return a 200 status code and the body will contain the newly updated body.
curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"title\": [{\"value\":\"Updated with a PATCH request\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 17:01:23 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1562\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":3}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T17:01:23+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":23,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"6a3b293d-4617-417b-99d2-23d75b57f7c2\",\"url\":\"\\/taxonomy\\/term\\/23\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nThis example is how to change a field that references some other entity. For this example we will use the field_model field, this is a reference to the taxonomy term that holds the \"model\" of the resource (ie. Image, Collection, Audio, Video, etc)
On our example installation taxonomy term 22 is \"Binary\", so to change a node from what it was to a Binary you would do.
> curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"field_model\": [{\"target_id\": 22, \"target_type\": \"taxonomy_term\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 17:51:47 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1562\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":4}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-11T17:36:10+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T17:51:47+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[],\"field_model\":[{\"target_id\":22,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"e1f167e1-124d-4db4-96ab-30641ca4e21b\",\"url\":\"\\/taxonomy\\/term\\/22\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nTo patch an object and make it part of a collection, you need the id number of the collection object. In this example node 2 will be our collection.
target_type can be a confusing one, if you are ever unsure have a look at the returned values for an existing object.
> curl -i -u admin:islandora -H\"Content-type: application/json\" -X PATCH -d '{ \"type\": [{\"target_id\": \"islandora_object\"}], \"field_member_of\": [{\"target_id\": 2, \"target_type\": \"node_type\"}]}' 'http://localhost:8000/node/3?_format=json'\n\nHTTP/1.1 200 OK\nDate: Mon, 11 Mar 2019 18:01:40 GMT\nServer: Apache/2.4.18 (Ubuntu)\nX-Powered-By: PHP/7.1.26-1+ubuntu16.04.1+deb.sury.org+1\nCache-Control: must-revalidate, no-cache, private\nX-UA-Compatible: IE=edge\nContent-language: en\nX-Content-Type-Options: nosniff\nX-Frame-Options: SAMEORIGIN\nExpires: Sun, 19 Nov 1978 05:00:00 GMT\nVary:\nX-Generator: Drupal 8 (https://www.drupal.org)\nContent-Length: 1669\nContent-Type: application/json\n\n{\"nid\":[{\"value\":3}],\"uuid\":[{\"value\":\"3f618cdc-3f2a-4e77-b932-9ff1d461a57a\"}],\"vid\":[{\"value\":4}],\"langcode\":[{\"value\":\"en\"}],\"type\":[{\"target_id\":\"islandora_object\",\"target_type\":\"node_type\",\"target_uuid\":\"62189bec-3ef3-4196-b847-b17e5ce61fd5\"}],\"revision_timestamp\":[{\"value\":\"2019-03-11T17:36:10+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"revision_uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"revision_log\":[],\"status\":[{\"value\":true}],\"title\":[{\"value\":\"Updated with a PATCH request\"}],\"uid\":[{\"target_id\":1,\"target_type\":\"user\",\"target_uuid\":\"46a47057-de2d-4ce2-87ae-dbe4551209b8\",\"url\":\"\\/user\\/1\"}],\"created\":[{\"value\":\"2019-03-05T18:07:00+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"changed\":[{\"value\":\"2019-03-11T18:01:40+00:00\",\"format\":\"Y-m-d\\\\TH:i:sP\"}],\"promote\":[{\"value\":true}],\"sticky\":[{\"value\":false}],\"default_langcode\":[{\"value\":true}],\"revision_translation_affected\":[{\"value\":true}],\"content_translation_source\":[{\"value\":\"und\"}],\"content_translation_outdated\":[{\"value\":false}],\"field_access_terms\":[],\"field_alternative_title\":[],\"field_description\":[],\"field_display_hints\":[],\"field_edtf_date\":[],\"field_edtf_date_created\":[],\"field_edtf_date_issued\":[],\"field_extent\":[{\"value\":\"1 item\"}],\"field_identifier\":[],\"field_linked_agent\":[],\"field_member_of\":[{\"target_id\":2,\"target_type\":\"node\",\"target_uuid\":\"413135a6-0bd1-4d6b-8bcb-059cf7784d83\",\"url\":\"\\/node\\/2\"}],\"field_model\":[{\"target_id\":22,\"target_type\":\"taxonomy_term\",\"target_uuid\":\"e1f167e1-124d-4db4-96ab-30641ca4e21b\",\"url\":\"\\/taxonomy\\/term\\/22\"}],\"field_pid\":[],\"field_resource_type\":[],\"field_rights\":[],\"field_subject\":[]}\nSignposting is a technique used in RESTful APIs where other relevant resources are exposed to clients as Link headers in GET and HEAD requests. These Link headers follow a standard format as specified in RFC8288. Drupal already makes use of this technique for content entities, and Islandora takes it even further by providing additional Link headers that enable the client to navigate the repository and discover additional information about various resources. Because the links are returned in response headers, they can be relied upon without having to parse the message body. This makes them consistent across all serialization formats that can be returned in a message body (XML, JSON, JSONLD, etc...).
As a general precaution, link headers for Drupal entities are not exposed to users that do not have the permissions to view the entity linked in the header. So making GET and HEAD requests anonymously will yield a different set of headers than what an authenticated user would see. For example, anonymous users don't have the view media permission, so they will not see the link headers for media associated with a node.
Other representations generated by different serializers available through Drupal's REST API are exposed as link headers with rel=\"alternate\" and type equal to the mimetype that will be received when dereferencing the link. For example, if an entity in Drupal has a JSONLD representation, then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/node/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"
Entity reference fields are exposed as link headers with rel=\"related\" and a title equal to the entity reference field's display label. For example, if http://example.org/node/1 has an entity reference field name \"Associated Content\" that references http://example.org/node/2, then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/node/2>; rel=\"related\"; title=\"Associated Content\"
Entity reference fields for taxonomy terms get special handling. The taxonomy terms used to tag content are exposed as link headers with rel=\"tag\" and a title equal to the taxonomy term's display label. If the term has an external URI in a controlled vocabulary, then that URI is provided. Otherwise, the local Drupal URI is provided. For example, if a piece of content is tagged with taxonomy/term/1, which has a display label of \"Example Term\", then the link header returned in a GET or HEAD response would look like
Link: <http://example.org/taxonomy/term/1>; rel=\"tag\"; title=\"Example Term\"
If instead the term were to have the field_external_uri field with a value of http://exampletwo.org/vocab#term then the link header would look like
Link: <http://exampletwo.org/vocab#term>; rel=\"tag\"; title=\"Example Term\".
Media entities belonging to nodes are exposed as link headers with rel=\"related\" and a title equal to the display label of their field_media_use taxonomy term. For example, if a media is tagged as Original File indicating that it is the initial file uploaded, the link header returned in a GET or HEAD response for a node would look like
Link: <http://example.org/media/1>; rel=\"related\"; title=\"Original File\".
Files that are the source for media entities are exposed as Link headers in the GET and HEAD responses with rel=\"describes\". The endpoint to edit the contents of the source file is also exposed using rel=\"edit-media\". For example, if http://example.org/media/1 has the source file http://example.org/file.txt, then a GET or HEAD response would contain both
Link: <http://example.org/file.txt>; rel=\"describes\"Link: <http://example.org/media/1/source>; rel=\"edit-media\"After creating a node, adding it to a Collection, uploading a file and kicking off derivatives, the link headers returned for said node would look like the following. Note that non-Link headers have been removed for brevity:
vagrant@claw:~$ curl -I http://localhost:8000/node/1?_format=json\nHTTP/1.1 200 OK\n...\n# These are provided by Drupal core\nLink: <http://localhost:8000/node/1>; rel=\"canonical\"\nLink: <http://localhost:8000/node/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/admin/content/node/delete?node=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/node/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/node/1/revisions>; rel=\"version-history\"\nLink: <http://localhost:8000/node/1>; rel=\"https://drupal.org/link-relations/revision\"\nLink: <http://localhost:8000/node?node=1>; rel=\"https://drupal.org/link-relations/create\"\n# These are provided by Islandora\nLink: <http://localhost:8000/node/2>; rel=\"related\"; title=\"Member of\"\nLink: <http://purl.org/coar/resource_type/c_c513>; rel=\"tag\"; title=\"Image\"\nLink: <http://localhost:8000/media/1>; rel=\"related\"; title=\"Original File\"\nLink: <http://localhost:8000/media/2>; rel=\"related\"; title=\"Service File\"\nLink: <http://localhost:8000/media/3>; rel=\"related\"; title=\"Thumbnail Image\"\nLink: <http://localhost:8000/node/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nIf we were to inspect one of the Media associated with this node (which we would've gotten in the response above), the results would look like:
vagrant@claw:~$ curl -I http://localhost:8000/media/1?_format=json\nHTTP/1.1 200 OK\n...\n\n# These are provided by Drupal core\nLink: <http://localhost:8000/media/add>; rel=\"https://drupal.org/link-relations/add-page\"\nLink: <http://localhost:8000/media/add/image>; rel=\"https://drupal.org/link-relations/add-form\"\nLink: <http://localhost:8000/media/1>; rel=\"canonical\"\nLink: <http://localhost:8000/admin/content/media>; rel=\"collection\"\nLink: <http://localhost:8000/media/1/delete>; rel=\"https://drupal.org/link-relations/delete-form\"\nLink: <http://localhost:8000/media/delete?media=1>; rel=\"https://drupal.org/link-relations/delete-multiple-form\"\nLink: <http://localhost:8000/media/1/edit>; rel=\"edit-form\"\nLink: <http://localhost:8000/media/1>; rel=\"https://drupal.org/link-relations/revision\"\n# These are provided by Islandora\nLink: <http://localhost:8000/node/1>; rel=\"related\"; title=\"Media of\"\nLink: <http://pcdm.org/use#OriginalFile>; rel=\"tag\"; title=\"Original File\"\nLink: <http://localhost:8000/media/1?_format=jsonld>; rel=\"alternate\"; type=\"application/ld+json\"\nLink: <http://localhost:8000/media/1/source>; rel=\"edit-media\"\nLink: <http://localhost:8000/_flysystem/fedora/2019-03/IF-Org-Chart_0.jpg>; rel=\"describes\"; type=\"image/jpeg\"\nSee the Drupal Documentation on PHPUnit in Drupal.
"},{"location":"technical-documentation/running-automated-tests/#setting-up-phpunit","title":"Setting up PhpUnit","text":"Before you can use phpunit, you must first install the following:
composer require --dev -W phpspec/prophecy-phpunit drupal/core-dev
In ISLE, you need to make the database port available to PHPUnit. To do that, edit docker-compose.yml and find the section including image: islandora/mariadb[version number]. Shortly below is a labels: heading; set the value of the traefik.enable: to \"true\". Apply the changes made to the docker_compose.yml using docker compose up -d.
Follow the Configure PHPUnit and Create a directory for HTML output sections in Drupal Documentation on running phpunit tests to make a phpunit.xml file.
If you place the phpunit.xml file in any directory other than [drupal root]/web/core, you need to change the 'bootstrap' in the <phpunit> tag near the top of the file to point to the relative or absolute location of the [drupal root]/web/core folder.
When setting the SIMPLETEST_DB database credentials in ISLE, * the default username and db_name are drupal_default * your db_password can be found in codebase/web/sites/default/settings.php
Unless you changed the default values, just swap out [password] for your actual db password in the following:
mysql://drupal_default:[password]@islandora.traefik.me:3306/drupal_default`.\nIf you are in the Drupal root directory (codebase on ISLE; the one containing web) and your phpunit.xml file is also in the Drupal root directory, use the following command to run phpunit for a single test file (here, Islandora's DeleteNodeWithMediaAndFile.php):
vendor/bin/phpunit web/modules/contrib/islandora/tests/src/Functional/DeleteNodeWithMediaAndFile.php
If your phpunit.xml is in a different directory, such as web/core, then use the -c flag to specify the path to the directory containing phpunit.xml:
vendor/bin/phpunit -c web/core web/modules/contrib/islandora/tests/src/Functional/DeleteNodeWithMediaAndFile.php
For any Drupal URI, the corresponding Fedora URIs is computed by the Crayfish Commons library using the Drupal-assigned UUID. In Drupal, a \"pseudo-field\" is available that will display the corresponding Fedora URI on the page of a node, taxonomy term, or media object. The presence of this URI does NOT guarantee the existence of the corresponding entity in Fedora.
"},{"location":"technical-documentation/testing-notes/#enabling-the-fedora-uri-pseudo-field","title":"Enabling the Fedora URI Pseudo-field","text":"To display the Fedora URI pseudo-field on a Drupal node, media, or taxonomy term, go to Manage > Configuration > Islandora and select all the bundles for which you would like the Fedora URI displayed. Once you have selected the bundles, and cleared the cache, the new pseudo-field will appear at the bottom of all display modes. You can alter where in the display the Fedora URI field appears, by going to the \"Manage Display\" page for the bundle. For example, for a Repository Item, you'd go to Manage > Structure > Content Types, and under the dropdown for \"Repository Item, select \"Manage Display\". In that list you should see Fedora URI which you can move around (or hide) as desired. This will need to be repeated in each Display mode (tab). Clearing cache may be necessary to refresh the node display.
Note: This information used to be stored in a service called Gemini, which kept track of corresponding minted Fedora URIs and their minted equivalents. Gemini was removed for Islandora 2.0.0.
"},{"location":"technical-documentation/testing-notes/#how-do-i-search-for-a-object-in-the-solr","title":"How do I search for a object in the Solr?","text":"http://localhost:8983/solr/#/islandora/queryExample
ss_search_api_id:\"entity:node/4:en\"\nhttp://localhost:8080/bigdata/#queryhttp://localhost:8080/bigdata/#namespaces), make sure islandora is selected.select ?p ?o where { <drupal_url> ?p ?o }\nExample:
select ?p ?o where { <http://localhost:8000/media/8?_format=jsonld> ?p ?o }\nselect ?s where { ?s <http://pcdm.org/models#memberOf> <drual_url_of_the_collection?_format=jsonld> }\nExample:
select ?s where { ?s <http://pcdm.org/models#memberOf> <http://localhost:8000/node/7?_format=jsonld> }\nselect ?s where { ?s <http://pcdm.org/models#fileOf> <drupal_url_of_the_object?_format=jsonld> }\nExample:
select ?s where { ?s <http://pcdm.org/models#fileOf> <http://localhost:8000/node/4?_format=jsonld> }\nThis section describes how to update Drupal and its modules using Composer. If you installed Islandora using the Islandora Playbook or ISLE, then your Drupal was installed by Composer, so it is best practice to continue using Composer for updates. The method on this section is not specific to Islandora, and does not (yet) include how to update Islandora Features.
How to upgrade Drupal in ISLE
For specific instructions on how to upgrade Drupal core and the Drupal modules installed within ISLE, please refer to the documentation page: Maintaining Your Drupal Site
"},{"location":"technical-documentation/updating-drupal/#what-is-composer","title":"What is Composer","text":"It is recommended by Drupal.org and the Islandora community to use Composer with Drupal for various tasks.
\"Composer is a dependency manager for PHP. Drupal core uses Composer to manage core dependencies like Symfony components and Guzzle.\" [Source]
"},{"location":"technical-documentation/updating-drupal/#always-create-backs-ups-db-and-files-before-updating","title":"Always create backs ups (DB and files) before updating","text":"Before updating either Drupal core or Drupal modules:
Warning: Always revert to a backup if you get a fatal error in the update process.
"},{"location":"technical-documentation/updating-drupal/#updating-drupal-core","title":"Updating Drupal Core","text":"Over time new versions of Drupal \u201ccore\u201d are released, and Islandora users are encouraged to install official Drupal core updates and security patches. On the other hand \u201calpha\u201d and \u201cbeta\" versions of Drupal core should only be installed by advanced users for testing purposes.
The Islandora community STRONGLY recommends that the \"Composer\" method of upgrading Drupal core be used with Islandora as mentioned here.
"},{"location":"technical-documentation/updating-drupal/#here-is-an-overview-of-the-steps-for-updating-drupal-core-using-composer","title":"Here is an overview of the steps for updating Drupal core using Composer","text":"Back Up
First make sure you have made database and file back ups.
1) First, verify that an update of Drupal core actually is available:
composer outdated \"drupal/*\"
If there is no line starting with drupal/core, Composer isn't aware of any update. If there is an update, continue with the commands below.
2) Assuming you are used to updating Drupal and know all the precautions that you should take, the update is as simple as:
composer update drupal/core --with-dependencies
If you want to know all packages that will be updated by the update command, use the --dry-run option first.
Alternate syntax for Islandora 8 needed
If you are running the older Islandora 8 codebase that predates the Islandora 2 release, note that Islandora 8 is configured to use a fork of drupal-composer/drupal-project which requires this specific composer syntax compared to other Drupal 8+ sites:
composer update drupal/core webflo/drupal-core-require-dev \"symfony/*\" --with-dependencies
In addition, if you are upgrading from 8.5 to 8.7, you need to replace \"~8.5.x\" with \"^8.7.0\" for drupal/core and webflo/drupal-core-require-dev in composer.json. [Source]
3) Apply any required database updates using drush updatedb, or use the web admin user interface.
drush updatedb
4) Clear the cache using drush cache:rebuild, or use the web admin user interface.
drush cache:rebuild
For stepwise update instructions visit this page: https://www.drupal.org/docs/8/update/update-core-via-composer#s-stepwise-update-instructions
"},{"location":"technical-documentation/updating-drupal/#updating-drupal-modules","title":"Updating Drupal Modules","text":"Islandora uses several general Drupal modules and some specialized Islandora Drupal modules, and over time new versions of these modules are released. There are two approaches to updating Drupal modules in Islandora: using Composer or updating modules individually. Islandora uses Composer to determine which Drupal module versions should be installed for each release of Islandora. Therefore if you update the Islandora specific Drupal modules using Composer you will also update any dependent general Drupal modules as well. The second method is to individually update Drupal modules.
For more information about how to update Drupal modules visit:
https://www.drupal.org/docs/8/extending-drupal-8/updating-modules
Back Up
First make sure you have made database and file back ups.
"},{"location":"technical-documentation/using-rest-endpoints/","title":"Islandora via REST","text":"Each node, media and file in Drupal has its own URI and we can GET the resources, some in a variety of formats.
We can also create nodes, media and files in Drupal by using PUT and/or POST requests.
We can update the field content by using a PATCH request and lastly we can DELETE a node, media or file resource.
To perform these actions, you will need the RESTful Web Services module enabled.
To configure your setup via the UI, you also need the RESTful UI module enabled.
Then you can configure your REST services at https://<yourmachine>/admin/config/services/rest
This screenshot shows the setup for resources, you can see the various HTTP methods and what formats they will respond in and what authentication methods they support.
Islandora uses semantic versioning for all non-Legacy code in the github.com/Islandora organization. This allows us to be compatible with Composer and with Drupal's release naming conventions for contributed modules.
"},{"location":"technical-documentation/versioning/#semantic-versioning","title":"Semantic Versioning","text":"Semantic Versioning is a common versioning standard. Versions have the form: Major version, Minor version, or Patch.
1.2.3 => 1.2.4 - Just a bug fix, should be a drop-in replacement.1.2.3 => 1.3.0 - Adds in new features, should be a drop-in replacement to get new functionality.1.2.3 => 2.0.0 - Major changes, may require a migration or changes to your set-up.The following Islandora components use semantic versioning:
Drupal submodules, which are included in several of the above modules, share versions with their parents.
Drupal module versions switched from 8.x-1.x to 2.x
In October 2021, Islandora switched from the \"core compatibility\" based numbering scheme (8.x-1.x) to a pure semantic versioning scheme for its Drupal modules. In accordance with Drupal's requirements, this transition required us to bump the major version, from 1.x to 2.x, despite there not being any major API-breaking changes to the code itself.
"},{"location":"technical-documentation/versioning/#implications-for-release-process","title":"Implications for Release Process","text":"Committers should now create (i.e. \"tag\") new versions of components when new bug fixes, features, or API changes are successfully added. This means that \"releases\" (new versions) will be happening individually, continually, and far more frequently than before. See Releasing Islandora.
"},{"location":"technical-documentation/versioning/#module-interdependencies","title":"Module Interdependencies","text":"When Islandora components require other Islandora components in their composer.json files, we prefer the version specification syntax ^2 to point to the latest-released compatible version within the specified major version.
In Drupal, blocks are snippets of content that can be placed on pages within your website's layout. Some examples of blocks are your site's menu, breadcrumbs, search block, etc... You have a lot of control over when and where you see blocks using Drupal's administrative interface. You can create and delete blocks, as well as move them around in different locations on the page (called 'regions' in Drupal theme terminology). You can also control on which pages the blocks are visible, allowing you to conditionally show/hide them as appropriate. Blocks are made available to place by enabling the Drupal modules that provide them, and there's lots out there to choose from. Placing blocks is an essential skill for any site administrator, so let's jump right in and get our hands dirty. There are two methods discussed here: the \"Block Layout\" interface, and using \"Contexts\". There is a third option, \"Display Suite,\" that is not discussed here.
"},{"location":"tutorials/blocks/#using-block-layout","title":"Using Block Layout","text":"Using the admin toolbar, navigate to Admin > Structure > Block layout (admin/structure/block). You will see a table listing all of the available regions provided by the current theme for block placement. You can switch between available themes using the tabs above the table. For each region, enabled blocks are listed and can be re-arranged. Take a look at the \"Main Content\" region for example.
If you wanted to move the content above its tabs, you could drag the \"Main page content\" block above the \"Tabs\" block. If you scroll all the way down to the bottom and click the \"Save blocks\" button, now when you go to view content you should see something like this
If you want to add a new block to a region, hit the \"Place block\" button for that region and you'll be brought to a modal that will let you select which block to place.
After selecting your block, you will be brought to its configuration form.
Here you can give the block a different title, change its region using a dropdown, and control its visibility settings by configuring various conditions. In our case, we're adding the \"Development\" block to the \"Footer\" region. And since we want to see this block everywhere, we'll leave the visibility settings alone for now. Click the \"Save block\" button, and now if we go view some content we should see the \"Development\" menu in the footer of our site.
If we want to limit this eyesore to only the front page of our site, we can revisit the visibility settings of the block. Navigate to the block we just placed on the block placement page, and click its \"Configure\" button. Here you can limit the block to show only for certain roles or content types. You can also explicitly set a white or black list of pages, which is what we'll do.
Click the Pages tab and you'll have a text area you can list pages in. We're going to enter <front>, which is a special value that means \"The front page of your site\". Underneath the text area, you can select whether you are showing or hiding the block for this list of pages. We want to show this only on the front page, so we'll leave it set at \"Show for the listed pages\". Click \"Save block\" and go back to a piece of content. You shouldn't see the development menu.
But go to the frontpage and...
"},{"location":"tutorials/blocks/#using-context","title":"Using Context","text":"The visibility conditions in the block placement user interface are simple and effective. They cover the majority of use cases, however, there are times when you need even more control. This is where the Context module really shines. Any condition available to the Context module can be used to control block visibility. And you can even combine conditions using Boolean (AND/OR) logic.
For example, in the Islandora Starter Site, we provide a context specifically for showing a list of members of a collection. All of our content is the same content type (Repository Item), and Collections are distinguished by being tagged with the \"Collection\" taxonomy term. Using the admin toolbar, if you navigate to Admin > Structure > Context (admin/structure/context), you should see a list of all contexts available to the site.
Click the \"Configure\" button on the \"Collection\" context.
You can see in its \"Condition\" section, we have the \"Node has term\" condition, which checks for content tagged with the term provided by the user. In this case, it's set to look for the \"Collection\" tag. Underneath, in the \"Reaction\" section, you can see that the \"Block\" reaction has been selected, and its configuration looks a lot like the core block placement UI. The only difference here is that when the \"Include blocks from block layout\" box is checked, you are only placing the blocks you want to add in addition to an already existing block layout. If you uncheck the box, you are creating an entirely new block layout from scratch, and will have to replace basic things like breadcrumbs and menus.
Here you can see we've added only the members block to this block layout. If we go and navigate to a collection, you'll see its list of members underneath its content.
If we edit the collection and change its model tag from \"Collection\" to anything else, you'll see the block disappears!
As you can see, block placement can be very flexible, and allow you to customize your site in a very granular fashion. Between using core block placement and the context module, there's no block you can't get into the right place on your site.
"},{"location":"tutorials/blocks/#video-walkthrough-blocks","title":"Video Walkthrough: Blocks","text":"Click the image below to open the Blocks video tutorial on the Islandora Youtube channel.
See more videos from the Drupal 101 series here.
Islandora Quick Lessons (2020)
Learn more with this video on Creating a Custom Block.
"},{"location":"tutorials/create-a-resource-node/","title":"Create a Resource Node","text":""},{"location":"tutorials/create-a-resource-node/#overview","title":"Overview","text":"This tutorial will walk you through adding a piece of digital content (an image and some metadata) into an Islandora repository, creating an Islandora Resource Node. At the end, you will have metadata in a Drupal node, a file stored in Fedora, and derivatives automatically created.
Video version available
The material in this tutorial is presented in our video, Adding Content.
"},{"location":"tutorials/create-a-resource-node/#tools","title":"Tools","text":"For this tutorial you will need an instance of Islandora Starter Site. You can also use the online sandbox or one of the sandbox VMs.
Online Sandbox
login credentials for online sandbox
"},{"location":"tutorials/create-a-resource-node/#step-1-create-an-islandora-node","title":"Step 1: Create an Islandora Node","text":"To add content to Islandora, start by creating a node. When logged in to Drupal as an administrator, navigate with the top menu bar to Manage >> Content >> Add content or click on Add content under the Tools menu.
Next, select a content type that is configured with Islandora behaviours. In the Islandora Starter Site, there is an Islandora content type. It is called Repository Item.
At the top of the form, the Model field is a mandatory Islandora field. Select Image from the Model drop box. This will enable image display behaviours, and cause image derivatives to be generated once we upload a file.
The rest of the form contains a large number of metadata fields, grouped into collapsible sections. Fill out the Title (mandatory) and whichever other fields are applicable to you.
Keep it simple
For this tutorial, do not attempt to fill out any fields with an autocomplete symbol - a small circle at the right-hand-side of the text box.
Then, click Save. You have now created a node, and since it is of an Islandora content type, we call it a \"resource node.\"
The new node page displays, with the title, any metadata that we entered, and a Fedora URI which you can click on to see if this new node has been synced into Fedora (this only works if you have permission to access Fedora). Congratulations, you have created a resource node!
"},{"location":"tutorials/create-a-resource-node/#step-2-upload-an-original-file","title":"Step 2: Upload an Original File","text":"Alas, your node has no files. To upload a file, click on the node's Media tab.
Concept: Media tab
The \"Media\" tab shows Media that we say \"belong to\" or are \"media of\" that node. The idea of media \"belonging to\" nodes, and the \"Media\" tab, are part of Islandora but not part of standard Drupal.
The Media tab shows nothing listed, because this node has no files. So click on Add Media. You will be taken to a list of media types to choose from.
We want to add a jpg image, so click on Image. You are now presented with a form to create a media.
Images vs TIFFs
Drupal considers any type of image that can be viewed natively in the browser as an Image. For other image types that require special viewers, such as TIFFs, you would have to choose File. Learn more about Media in Islandora.
This form contains fields for metadata about the file, including \"technical metadata\". There are three important parts of the form:
With the mandatory fields filled out (do not edit the \"Media of\" field), click Save, and a new Media will be created attached to your resource node. You will land on the administrator's list of Media (i.e. all media on your site, not just those attached to the resource node), and the new media object you created is at the top of the list.
Select the Media you just created, and you will see a page containing that Media's information and a rendering of the image that you uploaded.
"},{"location":"tutorials/create-a-resource-node/#step-3-verify-derivatives-were-created","title":"Step 3: Verify derivatives were created","text":"Now return to the node you created, and you will see the image along with its (minimal) descriptive metadata. The image you are viewing is not the one you uploaded, but a lower-resolution service file that was created and stored in the Drupal public filesystem.
If you open the node's Media tab, you should see new Media have been added - these are derivatives have been created automatically, including a service file, a thumbnail, and a FITS technical metadata file.
You now have an Islandora resource node, which is a special case of a Drupal node, and it has Media attached to it including an Original File, Thumbnail, Service File, and FITS Technical Metadata file.
"},{"location":"tutorials/create-a-resource-node/#learn-more","title":"Learn More","text":"Views are powerful content filters that enable you to present Islandora (and other) content in interesting and exciting ways. For more documentation on views:
Islandora Starter Site ships with some views already created and turned on. The Islandora home page displays content items that have been added to Islandora. This view is named Frontpage, and it lists items that meet the following filter criteria:
This view will display all content items added to Islandora, as the checkbox Promoted to front page is on by default.
As you develop your Islandora Website it is likely that you will need to change the default behaviour of the Frontpage View. As an example, the following describes how to edit the Frontpage page view to only show content items and not collections.
For this example, we added six collection items to Islandora. In total there are eight items in the repository. In addition to the six collection items, there is one audio item and one image item.
To edit the front page view, hover over the view (Frontpage view) and select Edit view when displayed.
Select Add under the filter criteria section.
We do not want to display collections, so we need to add a filter criteria that does not filter for the Islandora model type 'Collection'.
Select Model from the list and then Apply (all displays).
Select Islandora Model to select filters on Islandora model types and select Apply and continue.
Select the operator Is none of and the Collection model (autocomplete should work here to help you). To finish click Apply (all displays).
Save the view. Now the 'Frontpage' View does not display collections.
For this example, we create a new view that only shows collections. It will be created as a Block (also see the tutorial on Configuring Blocks) that will only display on the front page. We will add the new collection list block below the existing frontpage view that lists items.
Name the view and select Create a block. Give the block a title and decide how you want it to display (Grid, Table, List, Paging). To progress, select Save and edit.
5. Customize the view format and sorting as required. 6. Add a filter criteria to only show the Islandora model type of 'Collection' and Save the view.
7. To place the view on front page, the new block must be added to the 'Main page content' area (using 'Block layout') and set to display on the front page. 1. Navigate to Administration >> Structure >> Block layout (/admin/structure/block). Under Main content select Place block
\nRestrict the block to only display on the frontpage by adding the text '' to the Page vertical tab. Then select Save block.
4. Review the block placement and move if required.
8. The 'Collection list' now only displays on the front page. It displays below the Main page content.
Islandora Quick Lessons
Learn more with videos on Basic Views and Advanced Views.
"},{"location":"tutorials/how-to-create-collection/","title":"How to create and add to a Collection","text":"This how-to demonstrates creating a collection and adding items to it in the Islandora Starter Site. For more about collections, see Concept: Collection.
"},{"location":"tutorials/how-to-create-collection/#introduction","title":"Introduction","text":"In the Islandora Starter Site, nodes that have \"Collection\" in their Model field will show a view of their member (\"child\") objects. A member is any object that is \"Member of\" that object, and can be added via the \"Children\" tab. This is the mechanism in the Islandora Starter Site, and your individual instances may vary.
Collections and Deleting
Collections and their members are independent of each other, and removing something from a collection does not delete it. Similarly, deleting a collection does not delete its members.
"},{"location":"tutorials/how-to-create-collection/#creating-a-collection","title":"Creating a Collection","text":"When logged in, click Manage > Content on the admin toolbar. Then, click on Add content.
Then click on 'Repository Item' to give your collection the default metadata profile for Islandora.
Fill out the form.
At the top of the form, select \"Collection\" from the Model dropdown list.
When done filling out the form, click Save.
The collection has been created. Now let's add some members to this collection.
"},{"location":"tutorials/how-to-create-collection/#add-existing-items-to-a-collection","title":"Add Existing Items to a Collection","text":"To populate a collection with existing items, return to any existing content and click on its Edit tab. This brings up the form for this item.
Scroll down to the bottom of the form to find the System section. In the Member of field, start typing in the name of the collection this item should belong to. Select the name of the collection you want from the autocomplete. It is important that you select it from the dropdown, not just type in the correct title, because selecting it causes the node id to appear beside the title and allows Drupal to create a relationship.
The correct collection is now selected. Click Save when ready.
To confirm, return to the collection and verify the new item appears in the collection's list of members.
"},{"location":"tutorials/how-to-create-collection/#add-a-new-item-as-a-member-of-a-collection","title":"Add a New Item as a Member of a Collection","text":"To create an item and add it as a member to a collection in one step, visit a collection and click on its Children tab. From the Children tab, you can manage the members of a collection and perform actions on them.
Click on the +Add Child button, and then select Repository Item as your new item's Content Type. Only content types that have the field_member_of field will be available from this list.
You are taken to the creation form for a Repository Item, but if you scroll down to the \"System\" section, you should see the widget for \"Member Of\" is already filled out for you with the appropriate collection.
Click 'Save' at the end of the form to create the new item and add it as a member of the collection.
Islandora Quick Lessons
Learn more with this video on Making a Collection.
"},{"location":"tutorials/switch-homepage-to-twig/","title":"Format Homepage with TWIG","text":""},{"location":"tutorials/switch-homepage-to-twig/#twig-debugging","title":"TWIG Debugging","text":"It's helpful to identify which TWIGs are available to use and where they're stored but not required use TWIGs to format the homepage.
# Copy the default service.\n$ cp web/sites/default/default.services.yml web/sites/default/services.yml\n\n# fix permissions (just in case)\n$ chown nginx:nginx web/sites/default/services.yml\n\n# I use nano to edit but you can pick whichever editor you want.\n# For this example we'll install the editor\n$ apk add nano\n\n# Now open the newly created service file and set these 3 values under the TWIG config section.\n$ nano web/sites/default/services.yml\n\n...yml\ntwig.config:\n debug: true\nauto_reload: true\ncache: true\n# Now save and exit (in NANO it's CTRL + x)\nCopy the default TWIG into your theme's template directory.
$ cp web/themes/contrib/bootstrap/templates/node/node.html.twig web/themes/contrib/solid/templates/node--6--full.html.twig\n\n# Clear cache\n$ drush cr\nX next to the loaded TWIG file. Please note that the file name corresponds to the node number. To use the URL alias instead of the node ID requires additional work. Here's a tutorial on this topic. <!-- FILE NAME SUGGESTIONS:\n x node--6--full.html.twig\n * node--6.html.twig\n * node--page--full.html.twig\n * node--page.html.twig\n * node--full.html.twig\n * node.html.twig\n-->\nNow edit the TWIG file (web/themes/contrib/solid/templates/node--6--full.html.twig) to say whatever you want, and it should show up immediately without needing to clear cache.
"},{"location":"tutorials/switch-homepage-to-twig/#clean-up","title":"Clean up","text":"Don't forget to turn off TWIG debugging in config file (web/sites/default/services.yml). This will likely have unexpected consequences on production system performance.
twig.config:\n debug: false\n auto_reload: false\nThis page is about controlling who can view or edit your Islandora content (metadata and binaries).
Islandora recommends using Drupal's access control features. Contributed modules such as those described below, can provide additional flexibility and configurability. However, these only apply when content is accessed through Drupal, and are not applied to data in Fedora, the Triplestore, or Solr.
Known strategies for implementing access control in Drupal include:
Exposed Endpoints
If access control is a concern, you will want to lock down access to the following services that are part of the Islandora Suite:
as anyone accessing data through those services are bypassing Drupal's access control. In all out-of-the-box sandboxes and demo instances, these services are wide open.
"},{"location":"user-documentation/access-control/#access-control-in-core-drupal","title":"Access control in Core Drupal","text":"Summary: Core Drupal's access control features include the \"published\"/\"unpublished\" states and some basic permissions that can be granted globally or at the bundle level.
Drupal's concepts of \"published\" and \"unpublished\" for nodes and media apply to Islandora just as they do in Drupal. Usually, published content is visible to the entire world (without any authentication), while unpublished content can only be accessed by selected users - usually administrators, other privileged roles, and the user who created the content. This is configurable through the Drupal Permissions interface.
All Drupal permissions are granted to roles, not individual users. Read more documentation on Users, roles, and permissions.
The extent of the configurability of Drupal Core's access control (excluding revisions) is provided by the following permissions:
View permissions (apply to all content or all media regardless of bundle) - View all published content - View own unpublished content - View (published) media - View own unpublished media \"Editing\" permissions (can be granted for a specific bundle, or for all nodes or media): - Create new content - Edit (your) own content - Edit any(one's) content - Delete (your) own content - Delete any(one's) content Administrative permission (includes all the above and more, give to trusted users only): - Administer contentThese are the basic access control options built into Drupal Core. There are many contributed modules that do access control, some are described below. Before using access control modules, please see Comparison and Overview of Access Control modules on drupal.org.
Contributed modules are required for the following cases:
Given the [Collection] \u2192 Node \u2192 Media \u2192 File structure of Islandora resources, it is reasonable to wonder if access to a Node, Media, or File is influenced by the permissions on its \"parent\" resource.
There is not yet a mechanism for Drupal's access control to be \"inherited\" from parent resources to child resources. An exception is non-public files, as described below.
"},{"location":"user-documentation/access-control/#sidebar-access-to-files-in-drupal","title":"Sidebar: Access to files in Drupal","text":"Drupal's access control applies to files except those stored in Drupal's public filesystem.
If you upload a file to Drupal, it's usually stored in a field on an entity. In Islandora, that entity is a Media. Drupal's access checks mean that a user must have permission to view the Media, and (if configured) the file field on that media, to see the file. However, file fields can be configured with various forms of file storage, and the file storage determines whether access control checks are applied at all. Files stored in Drupal's public filesystem are always public, viewable by anyone in the world.
Drupal's access control applies to files accessed through Drupal where the file storage is:
Direct access to file storage
Files stored using Flysystem are actually located on a separate service. It is the administrator's responsibility to implement appropriate access control on the target storage system, as there is a way to link directly to files in Fedora, S3, or Dropbox. These direct links bypass Drupal and therefore Drupal's access control.
The filesystem used by a field is usually configured through \"Manage Fields\" on the appropriate entity. However, it is possible for a field to be configured to use a certain filesystem, but for actual files created through back-end processes to use a different one. For example, derivatives generated by Islandora as configured in the Islandora Starter Site are written to the public filesystem, despite the field configuration which points to Fedora.
To determine the actual location of a file, right-click the existing file in the Media Edit page and select \"copy location\". A file that is saved in flysystem will include /_flysystem/ in the URL. These links (you can test it!) only work if you have permission to view the media. A file with /sites/default/files/ is probably publicly accessible (this is the public filesystem on most Drupal instances).
In the Islandora Starter Site, All Derivatives Are Public
Out of the box using the Islandora Starter Site, derivatives are created for all Islandora media (as long as the conditions of the contexts are met), and they are stored in the public Drupal filesystem. This happens even if the media and/or node are not published.
"},{"location":"user-documentation/access-control/#access-control-in-solr","title":"Access control in Solr","text":"The Search API Module, which connects Drupal to Solr, provides configuration so that access considerations are respected by the Drupal solr results display. The \"Processors\" configuration (e.g. at admin/config/search/search-api/index/default_solr_index/processors) provides checkboxes for the following options:
Both are enabled out of the box in the Islandora Starter Site. This will ensure that queries through Drupal never show content that the active user shouldn't see, as well as preventing information about unpublished entities from ever being entered into Solr.
Solr Admin Client
Anyone with access to the Solr Admin Client may see the full contents of the index, regardless of permissions. If this is not desired, you should restrict access to the Solr endpoint and GUI. By default, the Islandora Playbook exposes Solr at http://localhost:8983/solr/. ISLE-DC on the other hand can expose or not expose individual services, including Solr. Since December 2021 this has been available as an option in the .env file and since that time, Solr (and some other services) are not exposed by default.
\"The Group module allows you to create arbitrary collections of your content and users on your site and grant access control permissions on those collections.\"
Opinion
Group is one of the more hefty modules, and is difficult to learn. It is good if you have static or semi-static groups of users who need access to a static or semi-static group of content, but sharing an item with an arbitrary group of users is cumbersome.
"},{"location":"user-documentation/access-control/#permissions-by-term-contributed-module","title":"Permissions By Term (contributed module)","text":"\"The Permissions by Term module extends Drupal by functionality for restricting view and edit access to single nodes via taxonomy terms. [...] Taxonomy term permissions can be coupled to specific user accounts and/or user roles.\" By default, this module only affects nodes. To enable Permissions by Term for Media and other entities, enable the \"Permissions by Entity\" submodule.
The Islandora Starter Site includes an empty vocabulary called \"Islandora Access\", which is intended to hold such taxonomy terms. However, permissions_by_term or a similar mechanism to control access must be installed and configured on your own.
This module is known to work, and appears to be supported well by the Drupal community.
Warning
This module is known in the Islandora community to cause performance degradation when large numbers of nodes are involved. Seth Shaw describes this in a blog post, Content Access Control Solutions Investigation.
Examples Wanted
Do you have experience setting up Permissions By Term? We'd love some illustrative examples here.
"},{"location":"user-documentation/access-control/#field-permissions-contributed-module","title":"Field Permissions (contributed module)","text":"\"The Field Permissions module allows site administrators to set field-level permissions to edit, view and create fields on any entity.\"
"},{"location":"user-documentation/access-control/#other-contributed-modules","title":"Other contributed modules","text":"Workflow Participants allows for granting permissions on individual nodes or media to individual users, as an extension to the Workflow suite of modules. However, it is not well-supported and the \"manage workflow participants\" permission should not be given to untrusted users as it may grant users the ability to add participants even to content they are otherwise not able to see or edit.
"},{"location":"user-documentation/access-control/#access-control-in-fedora","title":"Access control in Fedora","text":"In Fedora, it is possible to control access to resources using Access Control Lists (ACLs) per the Fedora API Specification. ACLs are inherited through Fedora's LDP Containment relationships.
Islandora does not create customized ACLs when syncing content into Fedora.
Islandora does not create Fedora resources in hierarchies that use LDP containment relationships.
At this time, access control in Fedora is NOT reflective of access control in Drupal. In the sandboxes and demo installations, all resources in Fedora are open for the world to view. This includes unpublished nodes and media, as well as the files uploaded into Fedora. Access controls configured in Drupal are not synced in any way to Fedora.
Sites concerned with access control wish to \"lock down\" their Fedora to only be accessible through Drupal.
"},{"location":"user-documentation/access-control/#access-control-in-the-triplestore","title":"Access control in the Triplestore","text":"Anyone with access to Blazegraph can access all the repository information within Blazegraph. If this is not desired, you should restrict access to the Blazegraph endpoint and GUI. By default, the Islandora Playbook exposes Blazegraph at http://localhost:8080/bigdata/. ISLE-DC on the other hand can expose or not expose individual services, including Blazegraph. Since December 2021 this has been available as an option in the .env file and since that time, Blazegraph (and some other services) are not exposed by default.
Meta-Issue: Access Restrictions and Embargoes
"},{"location":"user-documentation/accessibility/","title":"Accessibility","text":"Accessibility is the ability of a site to be used fully by all users, including those using screen reader technologies or keyboard navigation.
"},{"location":"user-documentation/accessibility/#drupal-documentation","title":"Drupal documentation","text":"Islandora's accessibility features are currently provided by Drupal and contributed Drupal modules. These pages within the Accessibility section of the Drupal documentation can provide guidance:
Much of the accessibility of a website is dependent on how specific HTML tags and attributes are used, thus falls largely into the realm of Drupal Themes. The Olivero theme and the Claro admin theme were designed by the Drupal community with accessibility as a guiding principle.
"},{"location":"user-documentation/accessibility/#automatic-alt-text","title":"Automatic alt-text","text":"When creating image media, alt-text is a required attribute. An Islandora function currently automatically populates the alt-text, if that media is the \"media of\" a node, with the respective node's title.
This is not considered good accessibility as the node's title will likely be already on the page, and it doesn't add anything to describing the image itself. Please fill out the alt text with something meaningful when adding image content to Islandora.
"},{"location":"user-documentation/advanced-search/","title":"Islandora advanced search","text":"Advanced Search adds additional functionality beyond the basic Solr search. It enables the use of Ajax with search blocks, facets, and search results.
"},{"location":"user-documentation/advanced-search/#requirements","title":"Requirements","text":"Use composer to download the required libraries and modules.
composer require drupal/facets \"^1.3\"\ncomposer require drupal/search_api_solr \"^4.1\"\ncomposer require drupal/search_api \"^1.5\"\nHowever, for reference, islandora_advanced_search requires the following drupal modules:
To download/enable just this module, use the following from the command line:
composer require islandora/islandora\ndrush en islandora_advanced_search\nYou can set the following configuration at Administration >> Configuration >> Advanced Search Settings (admin/config/search/advanced):
"},{"location":"user-documentation/advanced-search/#configuring-solr","title":"Configuring Solr","text":"Please review Configure Search before continuing. The following assumes you already have a working Solr and the Drupal Search API setup.
"},{"location":"user-documentation/advanced-search/#configure-collection-search","title":"Configure collection search","text":"To support collection based searches you need to index the field_member_of for every repository item as well define a new field that captures the full hierarchy of field_member_of for each repository item.
Add a new Content solr field field_descendant_of to the solr index at admin/config/search/search-api/index/default_solr_index/fields.
Then under admin/config/search/search-api/index/default_solr_index/processors enable Index hierarchy and set up the new field to index the hierarchy.
The field can now be used limit a search to all the descendants of a given object.
Re-Indexing
You may have to re-index to make sure the field is populated.
"},{"location":"user-documentation/advanced-search/#configure-views","title":"Configure views","text":"The configuration of views is outside the scope of this document, please read the Drupal Documentation, as well as the Search API Documentation.
"},{"location":"user-documentation/advanced-search/#collection-search","title":"Collection search","text":"It will be typical that you require the following Relationships and Contextual Filters when setting up a search view to enable Collection Search searches.
Here a relationship is set up with Member Of field and we have two contextual filters:
field_member_of (Direct descendants of the Entity)field_descendant_of (All descendants of the Entity)Both of these filters are configured the exact same way.
These filters are toggled by the Advanced Search block to allow the search to include all descendants or just direct descendants (documented below).
"},{"location":"user-documentation/advanced-search/#paging","title":"Paging","text":"The paging options specified here can have an effect on the pager block (documented below).
"},{"location":"user-documentation/advanced-search/#sorting","title":"Sorting","text":"Additionally, the fields listed as Sort Criteria as Exposed will be made available in the pager block (documented below).
"},{"location":"user-documentation/advanced-search/#configure-facets","title":"Configure facets","text":"The facets can be configured at admin/config/search/facets. Facets are linked to a Source which is a Search API View Display so it will be typically to have to duplicate your configuration for a given facet across each of the displays where you want it to show up.
To be able to display exclude facet links as well as include links in the facets block we have to duplicate the configuration for the facet like so.
Both the include / exclude facets must use the widget List of links that allow the user to include / exclude facets
The excluded facet also needs the following settings to appear and function correctly.
The URL alias must match the same value as the included facet except it must be prefixed with ~ character that is what links to the two facets to each other.
And it must also explicitly be set to exclude:
You may also want to enable Hide active items and Hide non-narrowing results for a cleaner presentation of facets.
"},{"location":"user-documentation/advanced-search/#configure-blocks","title":"Configure blocks","text":"For each block type:
There will be one block per View Display. The block should be limited to only appear when the view it was derived from is also being displayed on the same page.
This requires configuring the visibility of the block as appropriate. For collection based searches be sure to limit the display of the Facets block to the models you want to display the search on, e.g:
For any valid search field, you can drag / drop and reorder the fields to display in the advanced search form on. The configuration resides on the block so this can differ across views / displays if need be. Additionally, if the View the block was derived from has multiple contextual filters you can choose which one corresponds to direct children, this will enable the recursive search checkbox.
"},{"location":"user-documentation/breadcrumbs/","title":"Breadcrumbs","text":"Breadcrumbs are a Drupal concept. They provide a hierarchical path of links to \"ancestors\" of the current content item.
"},{"location":"user-documentation/breadcrumbs/#islandora-breadcrumbs","title":"Islandora Breadcrumbs","text":"Islandora provides a module, \"Islandora Breadcrumbs\" (a submodule of the Islandora module) that creates breadcrumbs based on the value of configured reference fields (by default, field_member_of). To use Islandora Breadcrumbs, simply enable the module. Islandora breadcrumbs will apply to nodes that have the configured Entity Reference fields.
There are a few configuration options for this module, accessible at Manage > Configuration > Islandora > Breadcrumbs Settings (/admin/config/islandora/breadcrumbs). These include:
Breadcrumbs are cached, so if you aren't seeing the results that you expect, try clearing the Drupal cache.
"},{"location":"user-documentation/content-models/","title":"Content models in Islandora","text":""},{"location":"user-documentation/content-models/#resource-nodes","title":"Resource Nodes","text":"This section describes the Islandora concept of a Resource Node. For a step-by-step demonstration, see the tutorial Create a resource node.
A resource node holds the descriptive metadata for an Islandora object, as well as groups together the various files that are part of the object for preservation or display, such as the original file and various derivative files generated from it.
The model for exactly what constitutes an object in Islandora is flexible and can be adapted to the needs of specific users. For example, the Islandora Starter Site configuration considers an object as a resource node of the type \"Repository Item\" which contains descriptive metadata about the object. Attached to that Node are one or more Media, each representing a file that is part of this object, such as \"Original File\", \"Thumbnail\", \"Preservation Master\", etc. With this model, every original file uploaded into Islandora has its own resource node.
Multi-file Media configurations also attach Media to a parent node, but allow for that node to be represented by multiple \"Original File\"s. In this model, a Media contains the original file as well as any derivative files created from it (thumbnail, service file, etc.).
For an example of where these two different approaches could apply, the basic configuration might make sense for a book that has rich page-level metadata, so that each page would be its own Node with its own metadata record; the multi-file media configuration might be a better fit for a book that does not have page-level metadata (except an ordering or page numbers), so that each Media would represent one page, and all pages (Media) would be attached to a single parent Node/metadata record for the entire book.
As we learned in the introduction, objects in an Islandora repository are represented as a combination of resource nodes, media, and files in Drupal. Because of this, their metadata profile, display, form (and much more) are configurable through the Drupal UI. This gives repository administrators a huge degree of control over their repository without any need for coding. Much more so than ever before. And since we're using a core Drupal solution for modeling our resource nodes and media, compatibility with third-party modules is virtually guaranteed. This opens up a plethora of solutions from the Drupal community that will save you untold time and effort when implementing your repository with Islandora.
"},{"location":"user-documentation/content-models/#properties","title":"Properties","text":"Resource nodes, as Drupal nodes, have some common basic properties regardless of content type. These properties are not fields. This means that they cannot be removed and have limited configurability. Their name, what type of data they hold, etc... are all baked in. Here's an example of the basic properties on nodes:
nid: 1\nuid: 1\ntitle: \"I am an Islandora object\"\ncreated: 1550703004\nchanged: 1550703512\nuuid: 02932f2c-e4c2-4b7e-95e1-4aceab78c638\ntype: islandora_object\nstatus: 1\nAs you can see, it's all system data used at the Drupal level to track the basics.
Property Value nid The local ID for the node uid The ID of the Drupal user who created the node uuid The global ID for any entity title The title for the node created Timestamp of when the node was created changed Timestamp of when the node was last updated type Content type (e.g. which group of fields are present on the node) status Published, unpublished, etc...Compared to Islandora Legacy
These node properties are analogous to following Islandora Legacy object properties:
Islandora Legacy Islandora owner uid dc.title title PID uuid status statusThe small amount of configurability available for these properties is found on the content type editing form where a user can choose to change the label of the title field, whether to display author information on the node's page, etcetera. These settings will only apply to nodes of that particular content type.
To view all of a node's property and field values administrators can use the 'Devel' tab's 'Load' section:
"},{"location":"user-documentation/content-models/#fields","title":"Fields","text":"In addition to the basic node properties identified above, resource nodes (like all Drupal nodes) can have fields. Most of what we would think of as descriptive metadata is stored as fields. Resource nodes use 'content types' to define a specific set of required and optional fields it has; we can think of content types as metadata profiles for our objects. For example, you might have a content type for a set of repository objects that have very specialized metadata requirements but another content type for generic repository objects that share a more general set of metadata fields. A resource node's content type is set on its creation and is immutable. The section on metadata describes in more detail how fields on Islandora objects work.
Configuring fields (adding, editing, removing) is usually done through the Manage > Content types interface, as is described in the tutorial, Create/Update a Content Type.
Islandora has a notion of a content model, which is used to identify what type of content is being represented by a node (e.g. an image, a video, a collection of other items, etc...). This is done using a special field, Model, which accepts taxonomy terms from the Islandora Models vocabulary. By applying a term from the Islandora Models vocabulary to a node, Islandora will become aware of how to handle the node in response to certain events, like choosing a viewer or generating derivatives.
Compared to Islandora Legacy
Content models in Islandora Legacy were immutable and contained restrictions as to what types of datastreams could be associated with an object. Islandora imposes no such restrictions. Content models can be changed at any time, and they in no way dictate what types of media can be associated with a node.
"},{"location":"user-documentation/content-models/#media","title":"Media","text":"All resource nodes can be linked to any number of media. The media associated with a resource node can be managed using the \"Media\" tab when viewing a node. Much like the \"Members\" tab, Actions can be performed in bulk using the checkboxes and Actions dropdown.
See the media section for more details.
"},{"location":"user-documentation/content-models/#display-modes","title":"Display modes","text":"Drupal uses \"display modes\" (also called \"view modes\") as alternative ways to present content to users. You may be familiar with the \"full\" and \"teaser\" versions of nodes, which are rendered using two corresponding display modes. Islandora makes use of display modes to control how media content is displayed. Islandora Starter Site provides two display modes for Media, one which renders the OpenSeadragon viewer and the other which renders the pdf.js viewer. These two display modes can be enabled by using \"Display hints\" in the node edit form, or you can configure Islandora to use a specific display mode for all media based on the file's Mime type. Both methods make use of Contexts.
To set the display mode on the resource node's edit form, select the display mode you want to use for that node in the Display hints field:
Due to the associated Context configurations (\"OpenSeadragon\" and \"PDFjs Viewer\") that are shipped with the Islandora Starter Site, the selected display mode will then be used when the resource node's page is rendered.
At a global level, there are a couple of ways to tell Drupal to use the PDFjs viewer to render the content of the media field whenever the media has a Mime type of application/pdf.
The first way is to edit the \"PDFjs Viewer\" Context. By default, this Context tells Drupal to use the PDFjs viewer if the node has the term \"PDFjs\" (yes, that's a taxonomy term):
If you add the Condition \"Node has Media with Mime type\" and configure it to use application/pdf as the Mime type, like this:
Context will use whichever Condition applies (as long as you don't check \"Require all conditions\"). That is, if the \"PDFjs\" display hint option in the node edit form is checked, or if the node's media has a Mime type of application/pdf, the media content will be rendered using the PDFjs viewer.
The second way to use the media's Mime type to render its content with the PDFjs viewer is to create a separate Context that will detect the media's Mime type and use the configured display mode automatically. To do this, create a new Context. Add a \"Node has Media with Mime type\" condition and specify the Mime type, and then add a \"Change View mode\" Reaction that selects the desired display mode:
Finally, save your Context. From that point on, whenever the media for a node has the configured Mime type, Drupal will render the media using the corresponding display mode.
The node-level and global approaches are not exclusive to one another. One Context can override another depending on the order of execution. Contexts are applied in the order they are displayed on the Contexts page, which is editable through a drag-and-drop interface. Whichever Condition appears last in the list of Contexts between the node-level Condition (which in this case is the \"Node has term\" condition) the global Condition (which is \"Node has Media with Mime type\"), that one will override the other. An example of having the display mode specified in the node edit form intentionally override the display mode based on Mime type is to have media with the image/jp2 mime-type configured to use the OpenSeadragon viewer, but to manually select the OpenSeadragon display mode for nodes with JPEG media (for example, a very large JPEG image of a map, where the OpenSeadragon's pan and zoom features would be useful).
Islandora has a notion of membership, which is used to create a parent/child relationship between nodes. Membership is denoted using another special field, \"Member Of\". This is used to create the link between members and their parent collection, pages and their book (\"paged content\"), or members of a compound object and the compound object itself.
Any two nodes can be related in this way, though typically, the parent node has a content model of Collection or Paged Content (see their respective pages for more details). The \"Member Of\" field can hold multiple references, so it is possible for a single child to belong to multiple parents, but may also complicate the creation of breadcrumbs.
Compared to Islandora Legacy
In Islandora Legacy, there was a distinction between belonging to a collection and belonging to a compound object. In Islandora, this distinction is not present. Since all nodes can have members , essentially every node has the potential to be a compound object or collection.
Child v. Member
Islandora uses the \"child\" and \"member\" descriptor for resource nodes that store a reference to another resource node in the \"Member Of\" field interchangeably. Administrators will more often see the \"member\" terminology more often while front-end users will usually see \"child\" terminology.
For any node, its Children tab can be used to see all its members. You can also perform Actions in bulk on members using the checkboxes and the Actions dropdown as well as clicking on the Reorder Children tab to adjust the order in which they display.
"},{"location":"user-documentation/content-models/#more-information","title":"More information","text":"The following pages expand on the concepts discussed above:
This document was originally developed by Alex Kent and has been adapted for general use by the Islandora community.
[^1] In the Islandora Starter Site, this is the field_model field, which is populated by taxonomy terms in the islandora_models taxonomy vocabulary provided by the islandora_core_feature submodule of Islandora/islandora
[^2] In the Islandora Starter Site, this is the field_member_of field.
The conventional Islandora Legacy definition of an object is a file loaded in the repository with associated derivatives. In Islandora Legacy, objects (video files, audio files, PDFs, etc.) are loaded through the user interface, and Datastreams are generated automatically. These consist of access and display copies, the metadata, OCH/HOCR, technical metadata, and more. All of these Datastreams are directly connected to the object and accessed through the admin interface.
In Islandora, the traditional Islandora Legacy objects (video files, audio files, etc. that were represented in different content models) are now Drupal nodes. Islandora object nodes are a special kind of Drupal node, distinct from nodes that exist for other content types such as a blog post, an article, a page (like the About page on a site), and others. These Islandora objects are still loaded through the interface and described with the data entry form, and derivatives are still generated. However, the Datastreams are no longer connected to the original object in the same immutable way. Each of these Datastreams can be manipulated through Drupal by non-developers. You can create a variety of ways to view this metadata and information related to the objects. Doing so requires knowledge of Drupal 8, but this essentially means that there are many ways to view the metadata and access the related objects in Islandora.
In Islandora it is therefore helpful to think of objects as resource nodes. The term reflects the new nature of objects in Islandora. A resource node does not just refer to the individual object file, but encompasses multiple elements that all relate to each other, even if they are no longer directly connected like objects in Islandora Legacy.
The typical elements of a resource node:
These resource nodes are what the librarian, student, archivist, technician, or general non-developer creates through the data entry form. It is possible to configure all elements of a resource node in Islandora through Drupal. This fact allows control over how one accesses the node and how nodes are displayed and discovered online by non-developers. It also allows a repository to take full advantage of all third-party Drupal modules, themes, and distributions available.
"},{"location":"user-documentation/content-types/","title":"Creating and updating content types","text":""},{"location":"user-documentation/content-types/#overview","title":"Overview","text":"Since metadata in Islandora is stored as fields in Nodes, the standard Drupal Content Types system provides our 'ingest forms'. For more information about Content Types in general, please see Content Types in Drupal. If you are already familiar with Drupal Field UI, you\u2019re already well-equipped to create and modify your own ingest forms in Islandora.
This page will address how to create and modify ingest forms by editing fields and form display settings on Content Types via the graphical user interface (GUI). This page will also cover editing the RDF mapping to accommodate changes to fields.
Islandora forms are Drupal forms, and for help working with forms via the API, please check out the Further Reading section for links to more advanced Drupal documentation.
"},{"location":"user-documentation/content-types/#before-you-start","title":"Before you start","text":"If you have deployed your Islandora with the Islandora Starter Site configuration, you will already have a Repository Item content type available, with pre-configured fields and repository behaviours.
There are multiple tabs with different options to configure your Content Type:
Changes not displaying?
If you make changes under Manage display and don't see them reflected in your Node, double check that you have edited the right view mode
This example adds a new field where a user can indicate if the repository item needs to be reviewed:
The new field has been added:
It appears in the ingest form when creating a new repository object. To test this, go to Content >> Add content >> Repository item:
RDF Mappings
New fields, except for Typed Relation fields, are not automatically indexed in Fedora and the triple-store. Update the Content Type's RDF Mapping to enable indexing the field (see below).
Search
New fields will not automatically be searchable. They need to be added to the Solr index configuration. See the 'Setup and Configure Search' page for more information.
Context
To add new behavior based on the results of this new field, check out Context.
"},{"location":"user-documentation/content-types/#change-the-form-display","title":"Change the form display","text":"To change where in the form a field is displayed, go to the Admin menu, return to Structure >> Content Types, and find the Repository Item content type again. Select Manage form display from the dropdown menu or select the Manage form display tab.
When creating a new Repository Item, the new field appears at the top, as a set of radio buttons.
"},{"location":"user-documentation/content-types/#change-the-content-display","title":"Change the content display","text":"Finally, change how the results of this example field are displayed. Initially the new field shows up at the bottom of repository object pages:
In the Admin menu, return to Structure >> Content Types and find the Repository Item content type again. Select Manage display from the dropdown menu or select the Manage display tab.
To create your own custom content type from scratch, please refer to this guide on Drupal.org.
Your custom content types can contain whatever fields you like, but there are two mandatory fields that all Islandora content types should contain:
In order for a custom content type to be considered an Islandora Object, it needs to have the field \"Member of\" ('field_member_of'). This allows it to be included in contexts that have the \"Node is an Islandora node\" condition. Nodes that have this field will automatically be synced to Fedora and indexed by the triple store if you are using the context provided by the Islandora Starter Site. Having this field present in your content type also gives you tabs for adding children and media when viewing an item of that content type.
The other mandatory field is \"Model\" ('field_model'). This is used in several of the contexts that the Islandora Starter Site provides. This field determines how Islandora objects are displayed, and how media derivatives are created.
RDF mapping aligns Drupal fields with RDF ontology properties. For example, the title field of a content model can be mapped to dcterms:title and/or schema:title. In Islandora, triples expressed by these mappings get synced to Fedora and indexed in the Blazegraph triplestore. RDF mappings are defined/stored in Drupal as a YAML file (to learn more about YAML, there are several tutorials on the web. Currently, Drupal 8 does not have a UI to create/update RDF mappings to ontologies other than Schema.org. This requires repository managers to update the configuration files themselves. Consider using the RDF mappings included in the Islandora Starter Site as templates by copying and modifying one to meet your needs.
The Drupal 8 Configuration Synchronization export (e.g. http://localhost:8000/admin/config/development/configuration/single/export) and import (e.g. http://localhost:8000/admin/config/development/configuration/single/import) can be used to get a copy of the mappings for editing in a text editor before being uploaded again. Alternatively, a repository manager can update the configuration on the server and use Features to import the edits.
An RDF mapping configuration file has two main areas: the mapping's metadata and the mapping itself. Most of the mapping's metadata should be left alone unless you are creating a brand-new mapping for a new Content Type or Taxonomy Vocabulary. A partial example from islandora_default's islandora_object (Repository Item) is included below:
langcode: en\nstatus: true\ndependencies:\n config:\n - node.type.islandora_object\n enforced:\n module:\n - islandora_demo\n module:\n - node\nid: node.islandora_object\ntargetEntityType: node\nbundle: islandora_object\ntypes:\n - 'pcdm:Object'\nfieldMappings:\n title:\n properties:\n - 'dc:title'\n field_alternative_title:\n properties:\n - 'dc:alternative'\n field_edtf_date:\n properties:\n - 'dc:date'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_description:\n properties:\n - 'dc:description'\nThe required mapping metadata fields when creating a brand-new mapping include the id, status, targetEntityType, and bundle. (uuid and _core, not seen in the example above but may be present in exported copies, will be added by Drupal automatically.) bundle is the machine name for the Content Type or Taxonomy Vocabulary you are creating the mapping for. targetEntityType is node for Content Types or taxonomy_term for Taxonomy Vocabularies. The id configuration is a concatenation of target entity type and bundle ('node' and 'islandora_object' in the example above). The id is also used to name the configuration file: e.g. rdf.mapping.node.islandora_object.yml is rdf.mapping. plus the id (node.islandora_object) and then .yml.
The mapping itself consists of the types' and fieldMappings configurations.
All the mappings use RDF namespaces instead of fully-qualified URIs. For example, the type for islandora_object is entered in the RDF config as pcdm:Object instead of http://pcdm.org/models#Object. The available namespaces are defined in module hooks (hook_rdf_namespaces) but can also be entered manually in a configuration interface. Repository managers wanting to add additional namespaces need to go to Configuration > Search and Metadata > JSONLD and enter their desired namespaces in the \"Additional RDF Namespaces\" box.
Namespaces currently supported (ordered by the module that supplies them) include:
The types corresponds to the rdf:type predicate (which corresponds to JSON-LD's @type) and can have multiple values. This type value will be applied to every node or taxonomy term using the mapped content type or vocabulary.
In some cases a repository may want a node or taxonomy term's rdf:type to be configurable. For example, the Corporate Body Vocabulary (provided by the Controlled Access Terms Default Configuration module) has schema:Organization set as the default type in the RDF mapping. However, more granular types may apply to one organization and not another, such as schema:GovernmentOrganization or schema:Corporation. The alter_jsonld_type Context reaction allows Content Types and Taxonomy Vocabularies to add a field's values as rdf:types to its JSON-LD serialization (the format used to index a node or taxonomy term in Fedora and the triple-store).
fieldMappings specifies the fields to be included, their RDF property mappings, and any necessary data converters (the datatype_callback). One field can be mapped to more than one RDF property by adding them to the field's properties list. The datatype_callback is defined by the 'callable' key and the fully qualified static method used to convert it to the desired data format. For example, fields of the Drupal datetime type need to be converted to ISO 8601 values, so we use the Drupal\\rdf\\CommonDataConverter::dateIso8601Value function to perform the conversion.
Islandora Quick Lessons
Learn more with this video on Customizing a Form.
"},{"location":"user-documentation/context/","title":"Context","text":""},{"location":"user-documentation/context/#context-in-islandora","title":"Context in Islandora","text":"Context is a Drupal contrib module that allows configuration of \"if this, then that\" logic using an administrative user interface. Context defines \"conditions\" and \"reactions\" to enact the \"if this [condition], then that [reaction]\" logic. Some Islandora Legacy repositories use the community-contributed Islandora Context module to apply this sort of logic to Islandora objects. An example Context from Islandora Legacy is
If an object's namespace is 'customnamespace', render the block containing the right's statement \"Please contact Special Collections before reusing this item\".
The Context project page on drupal.org says \"You can think of each context as representing a 'section' of your site\", but that doesn't tell the whole story, since it gives the impression that Context is all about defining subsites. Context in Drupal 8 (and above) is much more powerful than in Drupal 7 due to how it interacts with the rest of Drupal (specifically, through Drupal 8's new plugin API). Because of this increased flexibility and power, and because Context provides a ready-made user interface for site administrators to combine conditions and reactions, Islandora uses Context to drive the logic underlying many important aspects of Islandora, including which derivatives get generated and how objects are displayed. In Islandora Legacy, many of these things are managed (and often hard-coded) within solution packs and utility modules.
"},{"location":"user-documentation/context/#an-example","title":"An example","text":"Let's look at the example of how Context can determine how an object is displayed. Drupal 8 has the idea of \"view modes\", which allow site builders to choose what happens when an object is viewed by the user (it has nothing to do with Drupal Views). In the node edit form for Islandora objects, there is a checkbox that, if checked, tells Drupal to render the image using the OpenSeadragon viewer:
This functionality is accomplished via the \"OpenSeadragon\" Context, which, as its configuration suggests, checks as its condition whether the node as the \"OpenSeadragon\" tag and if so, reacts by using the view mode \"OpenSeadragon\":
"},{"location":"user-documentation/context/#context-and-actions","title":"Context and actions","text":"Context in Drupal 8 uses Drupal Actions to execute the \"then that\" part of their logic. An implication of this is some configuration options relevant to the outcome of the \"then that\" reaction are configured in the Action admin interface (located at admin/config/system/actions), not within the Context admin interface (located at admin/structure/context). While this makes for a clunky admin experience, the advantage is that it allows the reuse of a single piece of Drupal code in multiple Contexts.
One important group of functionality in Islandora repositories that admins might want to control is what types of media get persisted to Fedora. Islandora allows administrators to choose what types of media get persisted to Fedora and what types of media get persisted to the Drupal public file system. For example, you may want to persist your thumbnails to Fedora along with your \"Preservation Master\" and \"Original File\" files. This can be accomplished using Context.
By default, Islandora stores thumbnails in Drupal's public file system, not in Fedora. Thumbnails are automatically generated by a Context (the \"Image Service File\" Context to be specific), but if you want to change the file system where your thumbnails are stored to the Fedora file system, you need to change that setting in the \"Image - Generate a thumbnail from a service file\" Action, not in the Context that uses that action:
This configuration should be done as part of your initial Islandora setup, since there is currently no way to move thumbnails that were persisted to Drupal's public file system to Fedora, or vice versa. But, at least site admins can choose one or the other using this Action.
You can create custom Actions and then use them in Contexts. For example, if you want to present a message to the user every time they upload a media tagged as \"Preservation Master File\", create an Action of type \"Display a message to the user\" and enter your message:
Then, create a Context that uses this Action as its Reaction, and as its Condition, uses \"Media has term\" with a value of \"Preservation Master File\":
Now, when a user creates a Media and tags it as a \"Preservation Master File\", they will be shown the message you created in your Action:
"},{"location":"user-documentation/context/#tips-on-using-islandora-specific-contexts","title":"Tips on using Islandora-specific contexts","text":"Islandora provides some Conditions and Reactions that aren't available on other Drupal sites. When using them, you may find the following tips useful:
,) after the first term in the autocomplete field and continue typing.Most Islandora repository administrators will not need to alter or configure any Contexts. But since much of Islandora's underlying functionality is governed by this powerful Drupal contrib module, administrators should become comfortable using it, and Actions, to customize and refine their repositories. Also, since Islandora objects are ordinary Drupal nodes, Islandora objects can take advantage of standard Drupal 8 Context functionality out of the box.
"},{"location":"user-documentation/extending/","title":"Extending Islandora: The Islandora Cookbook","text":"Because newer Islandora is very tightly integrated with Drupal, many use cases that required custom Islandora modules in Islandora Legacy and earlier can now be solved with a combination of configuration and modules from the Drupal community. Other use cases may still require custom code, but are not common enough, or complete enough, for that custom code to be a part of the core software.
Both scenarios are covered by the Islandora Cookbook below, which contains an annotated list of custom Islandora tools and modules developed by our community, a list of Drupal contributed modules that may be useful in an Islandora context, and a set of recipes which each provide a detailed outline for solving a particular use case.
Islandora is also compatible with most Drupal themes.
"},{"location":"user-documentation/extending/#introduction","title":"Introduction","text":"In the spirit of Islandora Awesome, Islandora Cookbook is a curated list of great modules and other tools for Islandora. Because Islandora is more tightly integrated with Drupal, most of these 'recipes' require only Drupal modules as ingredients.
We offer this list for discovery, but do not officially provide support for any of these modules.
"},{"location":"user-documentation/extending/#recipes","title":"Recipes","text":"Below are modules and tools that might be useful to solve common use cases, presented without specific recipes.
"},{"location":"user-documentation/extending/#islandora-contributed-modules","title":"Islandora contributed modules","text":"Warning! - All modules are under development.
Permissions by Term
By default, Drupal allows you only to restrict access to Drupal nodes by coupling node content types to user roles. The Permissions by Term module extends Drupal by functionality for restricting view and edit access to single nodes via taxonomy terms. Since Islandora objects can have taxonomy terms, this can be used to control access at the node and collection level. The submodule Permissions by Entity extends this control to the media level.
"},{"location":"user-documentation/extending/#setting-a-date-and-time-for-publication","title":"Setting a date and time for publication","text":"Moderation Scheduler
Moderation Scheduler gives content editors the ability to schedule Islandora nodes to be published at specified dates and times in the future. Can be combined with Views Bulk Edit to set the same scheduled publication date/time on multiple nodes at once.
"},{"location":"user-documentation/extending/#tombstoning","title":"Tombstoning","text":"Tombstones
A module that informs users who attempt to view deleted content that the resource has been removed instead of showing a generic 404 page. Fielded so that the date, reason, and alternative links can be given, along with a citation for the deleted object.
"},{"location":"user-documentation/extending/#displays","title":"Displays","text":""},{"location":"user-documentation/extending/#image-slideshow","title":"Image slideshow","text":"Views Slideshow
Views Slideshow can be used to create a slideshow of any content (not just images) that can appear in a View. Powered by jQuery, it is heavily customizable: you may choose slideshow settings for each View you create. It can be used to create an easy, adjustable slideshow of images from an Islandora repository. Views Slideshow can be seen in action with Islandora objects on the front page here
"},{"location":"user-documentation/extending/#ingest","title":"Ingest","text":""},{"location":"user-documentation/extending/#batch-uploading-via-csv","title":"Batch uploading via CSV","text":"Migrate Islandora CSV
This repository is a tutorial that will introduce you to using the Drupal 8 Migrate tools to create Islandora content. Whether you will eventually use CSVs or other sources (such as XML or directly from a 7.x Islandora) this tutorial should be useful as it covers the basics and mechanics of migration.
This repository is also a Drupal Feature that, when enabled as a module, will create three example migrations ready for you to use with the Migrate API.
"},{"location":"user-documentation/extending/#alternative-way-to-batch-upload-via-csv","title":"Alternative way to batch upload via CSV","text":"Islandora Workbench
Command-line tool for ingesting (and updating) nodes and media from anywhere - you don't need to access to the Drupal server's command line. Provides robust data validation, flexible organization of your input data (can use CSV, Google Sheets, or Excel files) plus creation of taxonomy terms on the fly.
"},{"location":"user-documentation/extending/#search","title":"Search","text":""},{"location":"user-documentation/extending/#custom-search-weighting","title":"Custom search weighting","text":"Search Overrides
This module provides a method for users with the necessary permissions to manually override the results being returned by Search API Solr. They will be able to choose a specific search term, and pick which nodes should be at the top, and also choose to exclude nodes so they will not be shown in the results. Currently, only nodes are supported.
"},{"location":"user-documentation/extending/#other","title":"Other","text":""},{"location":"user-documentation/extending/#batch-editing","title":"Batch editing","text":"Views Bulk Edit
A powerful tool that turns Views into a means of batch editing nodes, including Islandora repository objects. Once installed, create a view, add the fields that you would like to edit, add a Views bulk operations (Edit) field to the view, and select which actions you would like to have available. The Modify field values action will allow you to batch edit the value for the same field across multiple objects. A demonstration of a simple bulk-editing view with a few fields and actions can be found here
Content Type Clone
A tool that allows you to clone an existing content type. Can be used to copy and easily make your own version of the Repository Item Content Type with fewer or edited fields, without starting over. Has options to also copy all nodes from the old type to the new, and to delete from the old type when copying.
"},{"location":"user-documentation/extending/#gather-user-feedback","title":"Gather user feedback","text":"Content Feedback
The Content Feedback module allows users and visitors to quickly send feedback messages about the currently displayed content, and can be applied to Islandora nodes. All content feedback messages are listed and grouped by status in an administrative feedback list.
"},{"location":"user-documentation/extending/#sending-your-content-to-archiveorg","title":"Sending your content to Archive.org","text":"Wayback Submit to Archive.org
A tool for automatically submitting the contents of your site to Archive.org. The Wayback Submit module will submit all node types on schedule, according to criteria set by the site admin (only certain node types, only certain views, etc).
"},{"location":"user-documentation/extending/#other-resources","title":"Other resources","text":"This tutorial contains are step-by-step instructions for adding a facet on a vocabulary reference field. After a search, a block containing the configured facets appears letting the user filter their search.
This is a walk-through. For a more in-depth discussion of creating facets, see Configure Advanced Search
This tutorial currently works best with the \"standard\" install of the Islandora and Islandora Default modules which can be obtained from the playbook (as opposed to the install profile, which handles facet blocks differently).
"},{"location":"user-documentation/faceting/#step-1-add-vocabulary-bundles","title":"Step 1: Add vocabulary bundles","text":"Steps 1 and 2 add the field to the Solr index.
Now that you've added the field, you need to rebuild your Solr index.
Step 4 adds and configures the facet itself.
This step adds the facets in a single block.
At this point, searching for content that has facet values should cause the block to appear. For more in-depth overview of search, see Configure Advanced Search
Facets aren't necessarily searchable
While this will create facets, the values that appear won't work (won't necessarily bring back any content) if you type them in the search box. This is because the fulltext search box uses only fulltext fields, and facets, as mentioned above, requires string fields.
If you want to be able to search for taxonomy term values and bring up the related nodes, you could either include the full rendered item for your content type, or you may wish to repeat step 2 for each entity reference field, and set the new fields to fulltext so that searching for term values brings back node results.
"},{"location":"user-documentation/file-viewers/","title":"File Viewers","text":""},{"location":"user-documentation/file-viewers/#what-are-viewers","title":"What are viewers?","text":"Viewers allow site builders to display files in interactive JavaScript-based widgets, that provide functionality like zooming in/out, turning pages, playing/pausing, viewing in full screen, etc.
In Drupal, a common way to implement a viewer is through a module that provides a Drupal field formatter that interfaces with the appropriate JavaScript library. The field formatter will work with specific types of Drupal fields (e.g. file fields or image fields, some may even provide their own fields). Some viewer modules in Islandora also provide a block, that can display appropriate files based on the context.
Viewers that are known to work with Islandora include:
The simplest Drupal-y way of making a viewer appear is to configure a Media to render. You can do this by configuring a View Mode that shows the desired file field, displayed in a field formatter that invokes the desired viewer.
In the Starter Site:
Note
Formerly, this field was called \"Display Hints\". That field name has been retired in order to reduce confusion, since this uses a different mechanism. This mechanism no longer relies on Node View Modes, or EVA views. However, the basic EVA view still persists in the starter site as it is part of the Islandora Core Feature. Again, it will first look for a Service File, then fall back to the Original File.
"},{"location":"user-documentation/file-viewers/#changing-a-viewer-for-all-media-of-a-media-type","title":"Changing a Viewer for all media of a media type","text":"With the above configuration:
field_media_file, field_media_image... as appropriate), select a different field formatter and configure it how you like it. Suppose you have a new viewer available, for example, for zip files. You could either:
Either would work! The choice is yours to make. They're honestly both good.
Should you choose the latter:
Both OpenSeadragon and Mirador provide blocks that act as multi-page viewers. To configure one of these viewers:
node/%node/book-manifest-original; in the block, we enter this as node/[node:nid]/book-manifest-original. When the block is rendered on a node page, such as node/18, then the nid (18) will be passed into the view.Islandora uses Flysystem and the associated Drupal module to persist binary files to Fedora instead of keeping a copy in both Drupal and Fedora.
"},{"location":"user-documentation/flysystem/#background","title":"Background","text":"\"Flysystem is a filesystem abstraction library for PHP\" which allows applications to read from and write to a variety of data source beyond the local file system, such as an SFTP server, Amazon S3, and Zip files provided an Adapter is available to support it. Flysystem Adapters extend a single class implementing League\\Flysystem\\FilesystemInterface although some separate adapter traits for common actions and properties, such as the StreamedCopyTrait, are available.
The Drupal Flysystem module extends Flysystem to work within the Drupal filesystem structure. Drupal flysystem plugins include a Flysystem adapter (if not provided by default or in another library) and a class implementing Drupal\\flysystem\\Plugin\\FlysystemPluginInterface which instantiates the Flysystem adapter based on the Drupal site's configuration.
The Drupal Flysystem module uses flysystem stream wrappers to define filesystem descriptors which are configured in the site's settings.php file. The configurations including the filesystem prefix, adapter (driver), and any adapter-specific configurations such as API endpoints and authorization information.
Islandora implements a Flysystem adapter and a Drupal Flysystem plugin. The Flysystem adapter acts as an intermediary between the Flysystem filesystem API and Fedora, translating requests and responses between them. The adapter interacts with Fedora using an instance of the Chullo Fedora API it receives from the Drupal Flysystem plugin. The Drupal Flysystem plugin's main responsibility is to instantiate the Chullo Fedora API object with the proper authentication and pass it to the Flysystem adapter. To authenticate with Fedora the plugin adds a handler to the Chullo's Guzzle client which adds a JWT authentication header to each request.
The Fedora Flysystem adapter does not use Gemini to map the relationship between Drupal URIs and Fedora URIs, so they are indexed separately using the \"files_in_fedora\" Context which triggers the \"Index Fedora File in Gemini\" and \"Delete Fedora File in Gemini\" actions as appropriate.
"},{"location":"user-documentation/flysystem/#configuration","title":"Configuration","text":"The fedora file system is configured in the site's settings.php file. An example configuration can be seen in the islandora-playbook web server role's drupal tasks:
$settings['flysystem'] = [\n 'fedora' => [\n 'driver' => 'fedora',\n 'config' => [\n 'root' => 'http://localhost:8080/fcrepo/rest/',\n ],\n ],\n];\nhttp://localhost:8080/fcrepo/rest/test.tif and will be accessible from the Drupal URL http://localhost:8000/_flysystem/fedora/test.tif. The driver value fedora corresponds to the plugin's machine name. The config section contains all the adapter-specific configurations. In this case, the only thing configured for the site is the Fedora REST end-point. (Change this value to match your own Fedora's location, if needed.) The JWT is configured separately. Other examples of Drupal Flysystem configurations can be seen in the module's README.
Islandora is configured to have all Media use the Fedora file system by default in the islandora_core_feature. For example, the field storage uri_scheme setting for field_media_image (and the other media types) is \"fedora\". This can also be viewed in the UI on the field's \"Field settings\" page; e.g. http://localhost:8000/admin/structure/media/manage/image/fields/media.image.field_media_image/storage, look for \"Upload destination\" and see that \"Flysystem: fedora\" is selected.
However, there are methods for saving files that can explicitly set a different filesystem than the default. Migrations can explicitly set which file system a file is saved to and Islandora can emit events that also specify which file system a derivative should be saved to.
"},{"location":"user-documentation/flysystem/#derivatives","title":"Derivatives","text":"As hinted in the previous section, Islandora, by default saves derivatives to the Drupal public file system.
For example, if I upload a TIFF to a repository item as a File Media with the term \"Original File\", the \"Image Original File\" (image_original_file) Context is triggered. This fires the 'image_generate_a_service_file_from_an_original_file' action which emits an event using the 'public' scheme (file system).
To make Islandora save future derivatives to Fedora instead of to Drupal, change the corresponding action's \"File system\" setting ('scheme' in the corresponding config file) to 'fedora' instead of 'public'.
"},{"location":"user-documentation/glossary/","title":"Glossary","text":"The following glossary of terms addresses an Islandora context. When comparing new Islandora and Fedora to older versions it may also be helpful to reference the Islandora 7 Glossary.
"},{"location":"user-documentation/glossary/#alpaca","title":"Alpaca","text":"Islandora's event-driven middleware based on Apache Camel that handles communication between various components of Islandora, for instance synchronizing Drupal data with a Fedora repository and the Blazegraph triple store.
"},{"location":"user-documentation/glossary/#ansible","title":"Ansible","text":"Open source software for provisioning, configuration management and application deployment. In the context of Islandora, Ansible can be used to install and maintain the Islandora software stack more conveniently and efficiently on a server or group of servers. The configuration and installation instructions are captured in a human-readable list of tasks, called 'Playbook'. The Islandora Playbook for Ansible is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#apache","title":"Apache","text":"Refers to the Apache Software Foundation, a not-for-profit organization supporting various open source software development projects. The Islandora software stack consists of different components that are developed under the umbrella of the Apache Software Foundation, for instance Apache ActiveMQ, Apache Camel, the Apache HTTP server (webserver), Apache Karaf, Apache Solr, and Apache Tomcat.
Can in a narrower sense refer to the Apache HTTP server.
"},{"location":"user-documentation/glossary/#api","title":"API","text":"See Application Programming Interface
"},{"location":"user-documentation/glossary/#application-programming-interface","title":"Application Programming Interface","text":"Also API; a connection between computers or between computer programs. It is a type of software interface, offering a service to other pieces of software.
"},{"location":"user-documentation/glossary/#blazegraph","title":"Blazegraph","text":"Blazegraph is an open source triplestore and graph database. Islandora ships Blazegraph as part of the software stack. Metadata about Resource nodes is synchronized between the Drupal database and Blazegraph. Data in the Blazegraph triple store can be queried using SPARQL.
"},{"location":"user-documentation/glossary/#bundle","title":"Bundle","text":"A bundle is the generic name for a sub-type of a Content Entity type in Drupal. To illustrate: Node and Taxonomy Term are both names of Content Entity types, and both have sub-types (\"bundles\"). The bundles of Node are called \"Content Types\" and the bundles of Taxonomy Term are called \"Vocabularies\". Each bundle includes its own configurations of what fields are present on the bundle and how they are entered and displayed. A bundle is thus part of the configuration of your site. Some Content Entity Types, such as User, do not have bundles.
"},{"location":"user-documentation/glossary/#cantaloupe","title":"Cantaloupe","text":"Cantaloupe is an image server written in Java. It implements the IIIF Image API, which means it handles deep zooming of large images and other image manipulations. It is required to serve images to some viewers such as Mirador and OpenSeadragon.
"},{"location":"user-documentation/glossary/#checksum","title":"Checksum","text":"Checksums are a sequence of numbers and letters to check data for errors. If you know the checksum of an original file, you can use a checksum utility to confirm your copy is identical. Checksums can be used to check the Fixity of a file.
"},{"location":"user-documentation/glossary/#claw","title":"CLAW","text":"CLAW (CLAW Linked Asset WebDataFrameWork) was the development code name for the software released in June 2019 as Islandora 8, now called Islandora.
"},{"location":"user-documentation/glossary/#collection","title":"Collection","text":"A collection is a way of grouping related resources together, much like a directory on a computer. Collections can contain any number of related resource Nodes and sub-collections.
"},{"location":"user-documentation/glossary/#configuration","title":"Configuration","text":"See also: Configuration entity
Contrast: Content
In Drupal, your configuration is the total set of configuration entities that are live in your site. Configuration is usually managed through the Drupal GUI, and it can also be exported and imported. When it is active in your site, configuration lives in the Drupal database. When it is exported or serialized, configuration appears as a set of YAML (.yml) files, one file per configuration entity. Configuration can be overridden in the settings.php file.
"},{"location":"user-documentation/glossary/#configuration-entity","title":"Configuration entity","text":"See also: Configuration
Contrast: Content entity
A Drupal configuration entity (or \"config entity\") is an individual piece that makes up your site's configuration. It is usually represented as a single YAML (.yml) file, though the actual (\"live\") configuration lives in the database. A config entity usually represents the results of saving a single form in the administration interface, and may contain multiple (usually related) individual settings. Each configuration entity can be exported or imported as a \"single item\" through the Configuration Synchronization GUI, or with the Devel module's \"config editor\" can be edited individually. However, config entities are often interrelated and manual editing is usually not recommended.
"},{"location":"user-documentation/glossary/#content","title":"Content","text":"See also: Content Entity
Contrast: Configuration
In Drupal, your content is the total set of things that have been created or uploaded \"as content\" in your website. This includes all content entities - the actual nodes, media, files, taxonomy terms, etc, but does not include anything that is configuration. Content can be exported and imported, but only between sites with exactly the same configuration.
Sometimes, \"Content\" is used to refer to Nodes but not other content entities. This is the case when creating a new View and one of the options is to make a view of \"Content\".
"},{"location":"user-documentation/glossary/#content-entity","title":"Content entity","text":"See also: Content
Contrast: Configuration entity
In Drupal, content entities are the actual nodes, media, taxonomy terms, users, comments, and files that you've created on your site. For example, you may have 223 nodes, 534 media, 1000 taxonomy terms, 14 users, and 535 files in your site - those counts represent the numbers of content entities present in your site. \"Node\", \"Media\", \"Taxonomy term\" etc. are the high-level \"types\" of content entities. Some of these types have sub-types which are called bundles.
Content entities should not be confused with content types, which are bundles of nodes, and are part of a site's configuration.
"},{"location":"user-documentation/glossary/#content-model","title":"Content model","text":"Deprecated concept used in Islandora Legacy; see Islandora Model.
"},{"location":"user-documentation/glossary/#content-type","title":"Content type","text":"A type of Node. Content types are the \"bundles\" of Nodes, which are a type of Content Entity.
The standard Drupal Content types are 'Article' and 'Basic page'. Islandora Starter Site adds 'Repository Item' as a Content type, defining metadata fields typically used to describe digital resources. You can easily create your own content types.
"},{"location":"user-documentation/glossary/#context","title":"Context","text":"An \"if-this-then-that\" configuration created using the Drupal Context contrib module. Islandora extends the capabilities of Context by adding custom Conditions, custom Reactions, and by evaluating context at specific times to allow Contexts to be used for derivatives, indexing, and display.
"},{"location":"user-documentation/glossary/#crayfish","title":"Crayfish","text":"A collection of Islandora microservices. Some microservices are built specifically for use with a Fedora repository, while others are just for general use within Islandora.
"},{"location":"user-documentation/glossary/#datastream","title":"Datastream","text":"Deprecated terminology, refers to how Fedora 3/Islandora Legacy stored files as part of a resource ('object') in the Fedora repository. Replaced by Drupal Media entities, which 'wraps' Files in an intermediate structure. This allows Fields to be attached to files, for instance for storing technical metadata.
"},{"location":"user-documentation/glossary/#derivative","title":"Derivative","text":"A version of a file which is derived from an uploaded file. For example, a thumbnail generated from an uploaded image. Islandora uses microservices to generate derivatives. See the concept page for Derivatives.
"},{"location":"user-documentation/glossary/#docker","title":"Docker","text":"Docker is a platform that use OS-level virtualization to deliver software in packages called containers. Islandora uses Docker as part of ISLE, a suite of Docker containers that run the various components of Islandora.
"},{"location":"user-documentation/glossary/#drupal","title":"Drupal","text":"Drupal is an open source web content management system (CMS) written in PHP. Known for being extremely flexible and extensible, Drupal is supported by a community of over 630,000 users and developers. Drupal sites can be customized and themed in a wide variety of ways. Drupal sites must include Drupal Core and usually involve additional, Contributed code.
"},{"location":"user-documentation/glossary/#drupal-core","title":"Drupal Core","text":"The files, themes, profile, and modules included with the standard project software download.
"},{"location":"user-documentation/glossary/#drupal-roles","title":"Drupal Roles","text":"Roles are a way of assigning specific permissions to a group of users. Any user assigned to a role will have the same permissions as all other users assigned to that role. This allows you to control which users have permission to view, edit, or delete content in Drupal. Islandora provides a special role called fedoraAdmin that is required to have actions in Drupal reflected in Fedora.
"},{"location":"user-documentation/glossary/#entity","title":"Entity","text":"A Drupal term for an item of either content or configuration data. Examples include Nodes (content items), Blocks, Taxonomy terms, and definitions of content types; the first three are content entities, and the last is a configuration entity. In common usage, the term often refers to Drupal content entities like Nodes or Taxonomy terms.
"},{"location":"user-documentation/glossary/#fedora-repository-software","title":"Fedora (Repository Software)","text":"Fedora is a digital asset management architecture upon which institutional repositories, digital archives, and digital library systems might be built. Fedora is the underlying architecture for a digital repository, and is not a complete management, indexing, discovery, and delivery application.
The Fedora repository functions as the standard smart storage for Islandora.
"},{"location":"user-documentation/glossary/#ffmpeg","title":"FFmpeg","text":"FFmpeg is a cross-platform audio and video processing software. In Islandora, FFmpeg is provided by the Crayfish microservice, Homarus.
"},{"location":"user-documentation/glossary/#field","title":"Field","text":"Data of a certain type that is attached to a content entity. For instance, on a Resource Node content type, you might have fields for a title, description, display hints, subjects, and other metadata.
"},{"location":"user-documentation/glossary/#field-instance","title":"Field instance","text":""},{"location":"user-documentation/glossary/#field-type","title":"Field type","text":""},{"location":"user-documentation/glossary/#fits","title":"FITS","text":"File Information Tool Set, a set of software components for identifying, validating and extracting of technical metadata for a wide range of file formats.
"},{"location":"user-documentation/glossary/#fixity","title":"Fixity","text":"Also file fixity; digital preservation term meaning that a digital file remains unchanged ('fixed') over time. Fixity checking verifies that a file has not been corrupted or manipulated during a transfer process or while being stored. Typically, a fixity checking process computes checksums or cryptographic hashes for a file and compares the result to a reference value stored earlier. The Riprap microservice and the contributed Riprap Islandora module support fixity checking and error reporting in Islandora.
"},{"location":"user-documentation/glossary/#flysystem","title":"Flysystem","text":"Flysystem is a filesystem abstraction library for PHP. Islandora uses Flysystem to swap about different backend filesystem applications. Islandora provides a custom Flysystem adapter for Fedora.
"},{"location":"user-documentation/glossary/#glam","title":"GLAM","text":"Acronym for \"galleries, libraries, archives, and museums\".
"},{"location":"user-documentation/glossary/#greenfield","title":"Greenfield","text":"An installation without legacy constraints. Usually refers to a brand new system where users load new content, as opposed to migrating content from a previous system.
"},{"location":"user-documentation/glossary/#imagemagick","title":"Imagemagick","text":"Imagemagick is an open-source image processing library. In Islandora, Imagemagick is provided by the Crayfish Microservice, Houdini.
"},{"location":"user-documentation/glossary/#hocr","title":"hOCR","text":"hOCR is an open standard for representing OCR (Optical Character Recognition) results, including text positioning, as HTML. hOCR can be produced by Tesseract, and can be displayed as an overlay on an image by Mirador.
"},{"location":"user-documentation/glossary/#homarus","title":"Homarus","text":"Homarus is a microservice wrapper for FFMpeg. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#houdini","title":"Houdini","text":"Houdini is a microservice wrapper for Imagemagick. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#hypercube","title":"Hypercube","text":"Hypercube is a microservice wrapper for Tesseract. It is part of Crayfish.
"},{"location":"user-documentation/glossary/#iiif","title":"IIIF","text":"The International Image Interoperability Framework. Generally pronounced \"triple-eye-eff.\" A set of open standards and APIs that help archives, libraries, and museums make the most of their digitized collections with deep zoom, annotation capabilities, and more, and also the community of users and developers that support the framework.
"},{"location":"user-documentation/glossary/#iiif-manifest","title":"IIIF Manifest","text":"Defined in the IIIF Presentation API, it is a document that includes \"The overall description of the structure and properties of the digital representation of an object.\" In Islandora, it lists one or more files, in order, that can be displayed in a viewer such as Mirador or OpenSeadragon.
"},{"location":"user-documentation/glossary/#ingest","title":"Ingest","text":"To ingest an object is to add an entry for it in Islandora. This can be done through the Drupal graphical user interface or one of the Drupal APIs (REST, Migrate API). The third-party contributed software Islandora Workbench uses the Drupal REST API for convenient bulk ingest.
In the context of digital repositories, ingest refers to the process by which the repository software imports and subsequently processes an object, creating derivatives automatically, and running any other processing that is configured to occur when an object is added. This would be distinguished by software which simply stores objects after import (with or without associated files) and performs no processing. The Islandora GUI and the documentation sometimes use other terms such as 'import' or 'add resource node'. In such contexts, these terms generally refer to the ingest process.
"},{"location":"user-documentation/glossary/#islandora-8-8x-20","title":"Islandora 8 (8.x, 2.0)","text":"Islandora 8, 8.x, 2.0, and CLAW are all deprecated names for the current version of Islandora. They referred to Islandora's use of Drupal 8, and being a major shift away from Islandora Legacy (formerly known as Islandora 7 or 7.x as it runs on Drupal 7).
"},{"location":"user-documentation/glossary/#islandora-install-profile","title":"Islandora Install Profile","text":"The Islandora Install Profile (in GitHub as Islandora Install Profile Demo, is a Drupal install profile that was developed by Born Digital, an Islandora vendor. It defines an Islandora with additional modules, themes, and configurations that were not defined in the Islandora Starter Site (formerly Islandora Defaults). The Install Profile and the Starter Site share the same function (though they approach it differently) and it is not possible to use both.
"},{"location":"user-documentation/glossary/#islandora-starter-site","title":"Islandora Starter Site","text":"The Islandora Starter Site is a way to install Drupal that provides a functional Islandora \"out of the box.\" It was created from Islandora Defaults [now defunct] by discoverygarden inc, an Islandora vendor. The Islandora Install Profile and the Starter Site share the same function (though they approach it differently) and it is not possible to use both.
"},{"location":"user-documentation/glossary/#islandora-model","title":"Islandora model","text":""},{"location":"user-documentation/glossary/#islandora-playbook","title":"Islandora playbook","text":"A set of human-readable YAML files, containing instructions for automatically configuring a server environment and installing the different components of the Islandora software stack. The instructions recorded in Playbook are executed by Ansible. The Islandora Playbook for Ansible is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#isle","title":"ISLE","text":"ISLE, or ISLandora Enterprise, is a community initiative to ease the installation and maintenance of Islandora by using Docker. ISLE is one of the installation methods currently supported by the Islandora community.
"},{"location":"user-documentation/glossary/#json-ld","title":"JSON-LD","text":"JSON-LD (JavaScript Object Notation for Linked Data) is a method of encoding linked data using JSON.
"},{"location":"user-documentation/glossary/#linked-data","title":"Linked data","text":"In computing, linked data is structured data which is interlinked with other data so it becomes more useful through semantic queries. Linked data typically employs the Resource Description Framework for data modelling.
"},{"location":"user-documentation/glossary/#manifest","title":"Manifest","text":"See IIIF Manifest.
"},{"location":"user-documentation/glossary/#matomo","title":"Matomo","text":"Matomo, formerly called Piwik, is a software for tracking visits to websites. It is an open source alternative to Google Analytics and allows the generation of website usage reports.
"},{"location":"user-documentation/glossary/#media","title":"Media","text":"Media are a Drupal Content entity type, which allows to manage Media items (Files) like images, documents, slideshows, YouTube videos, tweets, Instagram photos, etc. The Media module provides a unified User Interface where editors and administrators can upload, manage, and reuse files and multimedia assets. In the context of Islandora, Media entities 'wrap' files and provide a place to store file-specific metadata.
See https://www.drupal.org/docs/8/core/modules/media/overview for more information on the Drupal foundations, and refer to https://islandora.github.io/documentation/user-documentation/media/ for how Islandora uses Media.
"},{"location":"user-documentation/glossary/#memento","title":"Memento","text":"Protocol specification that allows a web client to request an earlier/historic state web resource (if available). Fedora implements the Memento protocol to store and serve versions of content in a Fedora repository.
"},{"location":"user-documentation/glossary/#mirador","title":"Mirador","text":"Mirador is a javascript-based zoomable image Viewer. It is related to (and more fully-featured than) OpenSeadragon. It has the ability to do zooming, display multiple pages, and display positioned text (e.g. hOCR or attributions). To render an image through Mirador, it must be provided a IIIF Manifest and the images must be served through a IIIF-friendly image server such as Cantaloupe.
"},{"location":"user-documentation/glossary/#microservice","title":"Microservice","text":"A software development technique \u2014 a variant of the service-oriented architecture (SOA) structural style \u2014 that arranges an application as a collection of loosely coupled services. In a microservices' architecture, services are fine-grained and the protocols are lightweight.
"},{"location":"user-documentation/glossary/#module","title":"Module","text":"Software (usually PHP, JavaScript, and/or CSS) that extends site features and adds functionality. Drupal modules conform to a specific structure allowing them to integrate with the Drupal architecture.
"},{"location":"user-documentation/glossary/#node","title":"Node","text":"Usually refers to a piece of Drupal Content of the type 'Node'. This includes actual pages, articles, and Resource nodes. Nodes must belong to a specific node bundle, called a \"Content Type\".
"},{"location":"user-documentation/glossary/#oai-pmh","title":"OAI-PMH","text":"The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed for harvesting metadata descriptions of records in an archive so that services can be built using (aggregated) metadata from many archives. Islandora allows to publish metadata in a way conformant to OAI-PMH, acting as a so-called OAI-PMH endpoint.
"},{"location":"user-documentation/glossary/#ontology","title":"Ontology","text":"In computer science and information science, an ontology encompasses a representation, formal naming and definition of the categories, properties and relations between concepts, data and entities. In the narrower context of the Resource Description Framework (RDF), an ontology is a formal, machine-readable description of the 'vocabulary' that can be used in a knowledge graph. An RDF ontology for instance specifies classes of things or concepts (e.g. the class of all book authors) and properties of classes/class instances (e.g. an author's name, birthdate, shoe size; also the fact that an author has written something that is in the class of books).
"},{"location":"user-documentation/glossary/#open-source","title":"Open Source","text":"Open source describes a method of software development that promotes access to the end product's source code. Islandora is an open source product with an active development community, operating under the GPL license (2.0) for Drupal components and the MIT license for non-Drupal components.
"},{"location":"user-documentation/glossary/#openseadragon","title":"OpenSeadragon","text":"OpenSeadragon is javascript-based zoomable image Viewer. It has the ability to do zooming and display multiple pages. To render an image through OpenSeadragon, it must be provided in a IIIF Manifest.
"},{"location":"user-documentation/glossary/#pr","title":"PR","text":"See Pull request
"},{"location":"user-documentation/glossary/#pull-request","title":"Pull request","text":"Also PR; sometimes also known as merge requests; technical term from distributed version control systems for software code like Git. Code contributors can request that the maintainer of a code repository 'pulls' the code change into the repository after approval.
"},{"location":"user-documentation/glossary/#rdf","title":"RDF","text":"See Resource Description Framework
"},{"location":"user-documentation/glossary/#repository-item","title":"Repository Item","text":"A type of content entity that comes \"out of the box\" with the Islandora Starter Site. See also: Resource Node
"},{"location":"user-documentation/glossary/#resource-description-framework","title":"Resource Description Framework","text":"Also RDF; family of World Wide Web Consortium (W3C) specifications originally designed as a data model for metadata. It has come to be used as a general method for conceptual description or modeling of information that is implemented in web resources. The data is modelled as a set of statements, also known as triples. A collection of RDF statements intrinsically represents a directed graph. Data represented according to the RDF specifications can be serialized in different ways, for instance using JSON-LD.
"},{"location":"user-documentation/glossary/#resource-node","title":"Resource Node","text":"A Resource node is a generic Islandora term for a Drupal Node that represents a single conceptual item or object stored in an Islandora repository. It acts as a stand-in for all files and metadata associated with that item, and is the place where the item 'lives' as a visitable URI.
The term 'Resource node' is specific to Islandora. Typically, Resource nodes in an Islandora installation will use a specific Content type for the digital assets stored in the repository.
For example, a video stored in Islandora will have a Resource node, with metadata stored in Fields. Attached to the Resource node is a Media entity, which encapsulates the preservation-grade file. The Resource node may be linked to further Media, for instance for a thumbnail, web-friendly derivative, and technical metadata associated with the resource node. The Resource node may also belong to one or more collections.
"},{"location":"user-documentation/glossary/#taxonomy-term","title":"Taxonomy term","text":"A Drupal Content Entity of the type 'taxonomy term'. Taxonomy terms belong to vocabularies which define what fields are available and how they behave. Drupal generally uses terms contained in taxonomies or vocabularies to classify content (tag or category). Taxonomy terms are used in Islandora to establish locally controlled vocabularies for describing resources, for instance for standardised spellings of names or subject terms.
"},{"location":"user-documentation/glossary/#tesseract","title":"Tesseract","text":"Tesseract is an open-source OCR (Optical Character Recognition) software. It can perform OCR in multiple languages. It can produce OCR (plain text) and hOCR (HTML, which includes positional data). In Islandora, Tesseract is provided by the Crayfish Microservice, Hypercube.
"},{"location":"user-documentation/glossary/#theme","title":"Theme","text":"Software and asset files (images, CSS, PHP code, and/or templates) that determine the style and layout of the site. The Drupal project distinguishes between core and contributed themes.
"},{"location":"user-documentation/glossary/#vagrant","title":"Vagrant","text":"Vagrant is an open-source software product for building and maintaining portable virtual software development environments (virtual machines). The Islandora Playbook includes a 'vagrantfile', a set of instructions that allows users to create a local virtual machine environment which will subsequently run Ansible to execute the configuration and installation steps recorded in the Islandora Playbook.
"},{"location":"user-documentation/glossary/#vbo","title":"VBO","text":"See Views Bulk Operations.
"},{"location":"user-documentation/glossary/#view","title":"View","text":"Drupal Views let you query the database to generate lists of content, and format them as lists, tables, slideshows, maps, blocks, and many more. The Views UI module, part of Drupal Core, provides a powerful administrator interface for creating and editing views without any code. There is a large ecosystem of extension modules for Views.
Views power many of the Islandora features, including viewers, IIIF Manifests, and search.
"},{"location":"user-documentation/glossary/#view-mode","title":"View Mode","text":"A View Mode is a way that a piece of Drupal content can be rendered. View modes let you create alternate configurations for what fields get displayed, in what order, and rendered in what field formatters. View modes are created under Manage > Display Modes > View Modes, but are configured at the bundle level (after first enabling that view mode to have its own configuration). If the requested view mode does not have a custom configuration, then the \"Default\" view mode will be used.
In Views, you can choose to show \"Rendered entities\" (usually as opposed to \"Fields\"). Here, you can select which view mode to use to render the results.
"},{"location":"user-documentation/glossary/#viewer","title":"Viewer","text":"A Viewer is any tool that allows Drupal to embed, display, or play back a particular object in a web-accessible format. Viewers are typically projects unto themselves. To use a viewer within Drupal usually involves a Library containing the viewer's code, as well as a Drupal Module that makes the viewer code appear within Drupal. Usually a viewer displays a single binary file, but some viewers (e.g. Mirador and OpenSeadragon) can display an entire manifest (ordered list of files).
"},{"location":"user-documentation/glossary/#views-bulk-operations","title":"Views Bulk Operations","text":"Also called VBO; a Drupal Module for performing bulk/batch operations on Nodes selected by a View definition.
"},{"location":"user-documentation/glossary/#virtual-machine-image","title":"Virtual Machine Image","text":"The Virtual Machine Image allows you to mount a fully working version of Islandora on your local machine as a separate virtual machine.
"},{"location":"user-documentation/glossary/#vocabulary","title":"Vocabulary","text":"A Drupal configuration entity that holds taxonomy terms. The vocabulary defines what fields are available on each term and how the terms behave. Vocabularies are the \"bundles\" of taxonomy terms.
"},{"location":"user-documentation/glossary/#weight","title":"Weight","text":"Drupal field that stores an integer value on an entity, allowing to represent the relative order of the entity in relation to other entities of the same type or subtype. Used by Islandora to store the order of components in compound objects, for instance pages in paged content items (books, serials).
"},{"location":"user-documentation/glossary/#yaml","title":"YAML","text":"YAML is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. Software applications like Drupal or Ansible store configuration information in YAML files for easy transportability of a configuration.
Some definitions adapted from Wikipedia and Drupal.org
"},{"location":"user-documentation/iiif/","title":"IIIF (International Image Interoperability Framework)","text":"IIIF is a set of specifications that provides interoperability of image-based collections across platforms. What this means for a repository platform like Islandora at a general level is that image-based objects such as (still) images and paged content can be managed by Islandora but viewed in external applications, and that Islandora can bring in image-based content from elsewhere to supplement locally managed content. If this intrigues you, see the section \"Looking under the hood (and beyond)\" below.
At a practical level, because Islandora supports several of the IIIF specifications, we can:
Display thumbnails for all pages of a book or newspaper issue within image viewers
If you're not using one of our provisioning tools, you will need to:
The Islandora Starter Site uses a Context to automatically use the IIIF Presentation API with the Mirador viewer for showing paged content.
To use this Context, give your book or newspaper (or other paged content) a model of \"Paged Content\" or \"Publication Issue\". To double-check this, in the Mirador Block - Multipaged items Context, you should see those terms used in the \"Node has term\" condition (you can register more than one term there, and having one of these on your node will activate this Context). Now, when you view a paged content Islandora node, you will see service files of all of its child pages (assuming you have added some child pages to the object) in the Mirador viewer as illustrated above.
If you are using the Mirador viewer, it enables a lot of features out of the box, including the strip of thumbnails at the bottom, and there is little to configure.
"},{"location":"user-documentation/iiif/#openseadragon-viewer-optional","title":"OpenSeadragon Viewer (optional)","text":"However if you are using the OpenSeadragon viewer, a Context can be set up as above, to show the OpenSeadragon viewer. In the Starter Site, there is currently a OpenSeadragon Block - Multipaged Items Context that is not enabled. You may wish to enable this and disable the Mirador Block - Multipaged items Context.
You can change how individual or paged content images are arranged in the OpenSeadragon viewport by doing the following:
admin/config/media/openseadragonIf you want to see the raw output of the IIIF API implementations in Islandora, visit a node that is displaying the OpenSeadragon viewer (doesn't matter if it's a single image or a paged content node like a book), and tack \"manifest\" onto the end of the URL, like http://myrepo.org/node/23/manifest and hit enter. You will see the raw JSON that IIIF-compliant viewers use to render the content. To see the output for a paged content item that lists all its children, tack /book-manifest on the end of the URL.
The really neat thing is, IIIF-compliant viewers don't need to be embedded in Islandora websites. If a viewer on another website knows the URL of a IIIF manifest like the ones that Islandora can produce, that viewer can display the content described in the manifest. Some implementations of IIIF viewers that show off the potential to combine content from multiple IIIF servers include:
These two examples have nothing to do with Islandora, but illustrate the potential for IIIF to build tools that extend beyond a given repository platform.
"},{"location":"user-documentation/iiif/#resources","title":"Resources","text":"To find resources on how to customize the IIIF interface check out IIIF's \"Guides to finding and working with IIIF materials\".
There is also a list of awesome IIIF resources that includes examples, tutorials, Digital Asset Management (DAMs), Image servers, Exhibits, Annotations, discovery, community involvement, Online training courses, and more.
"},{"location":"user-documentation/jwt-authentication/","title":"JWT authentication","text":"Islandora uses JWT tokens to authenticate communication between its components. RSA private public key pair is used to sign and verify JWT tokens. The process of issuing JWT tokens using RSA private key is handled by the Drupal jwt module.
The private public RSA pair needed by JWT authentication mechanism is generated in the web server. By default, claw playbook places the keys in /opt/islandora/auth. Crayfish and Tomcat/Karaf need the public key to verify the JWT token. By default, they are put in the following locations: /var/www/html/Crayfish/public.key, /etc/tomcat8/public.key. If you are deploying Crayfish and Karaf/Tomcat components to different servers, ensure that web server public.key files are in the expected locations.
Note that the connection need to be over SSL or an encrypted channel for this communication to be secure. Otherwise, a third party can capture your token and get access to your servers.
The JWT tokens expiration time is configurable via Islandora core settings: http://localhost:8000/admin/config/islandora/core. Currently, it is recommended to set the JWT Expiry to the maximum expected time for a job, including batch jobs.
The purpose of this page is to provide a guided reading list to anyone who wants to get up to speed on the basics of linked data within the Islandora community. Those who make their way through the readings will be able to talk competently about linked data and better understand the design decisions made in Islandora. The list starts with the fundamentals of linked data (RDF, SPARQL, serializations and ontologies) and moves toward more advanced topics specific to the use cases of a Fedora 4 based digital repository system.
"},{"location":"user-documentation/linked-data/#reading-list","title":"Reading list","text":""},{"location":"user-documentation/linked-data/#basics-of-linked-data","title":"Basics of linked data","text":"This section seeks to give the reader a foundational understanding of what linked data is, why it is useful, and a very superficial understanding of how it works.
This section is all about RDF, the Resource Description Framework, which defines the way linked data is structured.
This section takes a look at SPARQL, the query language that allows you to ask linked data very specific questions. The queryable nature of linked data is one of the things that makes it so special. Try some SPARQL queries on DBpedia's endpoint to get some hands-on experience.
RDF data can be translated into many different formats. RDF/XML is the original way that RDF data was shared, but there are much more human-friendly serialization formats like Turtle which is great for beginners. JSON-LD is the easiest format for applications to use, and is the serialization format that Islandora uses internally. Make sure to check out the JSON-LD Playground for an interactive learning experience.
Ontologies and vocabularies are created by communities of people to describe things, and once created, anyone can use an ontology or vocabulary to describe their resources. This section goes over some of the more popular ontologies & vocabularies in use.
One isn't limited to the ontologies & vocabularies that already exist in the world, anyone is free to create their own. This section goes over ontologies that exist to help those trying to create their own ontologies.
Most ontologies are very specific to certain use cases, and digital repository systems are no different. This section covers ontologies that are of specific interest to users of Islandora, or any Fedora 4 based digital repository system.
In Islandora, the JSON-LD Module transforms nodes (or media, or taxonomy terms) into the RDF that is synced into Fedora and the Triplestore. It uses RDF mappings, a concept defined by the RDF Module, and exposes them through the REST API at ?_format=jsonld.
A quick overview of JSON-LD, the RDF module, and the REST API.
"},{"location":"user-documentation/linked-data/#the-json-ld-syntax","title":"The JSON-LD syntax","text":"JSON-LD is a syntax which can be used to express RDF (like Turtle, or RDF XML), that is written in JSON, because devs like JSON and it's web-friendly. The JSON-LD syntax was designed for including Linked Data within HTML of web pages (similar to microdata or RDFa). Instead of nesting the RDF predicates within existing HTML tags as RDFa does, JSON-LD lets you put a solid blob of Linked Data inside a <script> tag. JSON-LD can also function as a standalone document, which is how we're using it.
The RDF Module is part of Drupal Core, but has no official documentation. The RDF Module embeds RDFa, a form of linked data, within the Drupal-generated HTML when you load the web page for a node, media, or taxonomy term. Official line is that this will allow Google to provide \"rich snippets\" such as star-ratings, contact info, and business hours. As an example of Drupal-provided RDFa:
<h1 class=\"page-header\">\n<span property=\"schema:title\">My cat</span>\n</h1>\nproperty=\"schema:title\" is markup generated by Drupal's RDF module that identifies the value \"My cat\" as the schema.org title of this page. A node's fields (such as field_tags) and properties (such as author) can be mapped to RDF according to a bundle-specific \"mapping\" that is stored within Drupal. In Drupal8-ese, RDF mappings are configuration entities. Drupal doesn't have a good UI for editing RDF mappings, but you can create, read, and update them as YAML files using Drupal's Configuration Synchronization interface (see section below on How to Edit an RDF Mapping).."},{"location":"user-documentation/linked-data/#rest-api","title":"REST API","text":"The pattern of using ?_format= to get a different representation of content is provided by the RESTful Web Services (rest) module. It allows other services to interact with Drupal entities through HTTP requests (GET, POST, PATCH, and DELETE). Which operations are allowed, and with what formats (such as xml, json, and jsonld) is configured at admin/config/services/rest/. Note that only jsonld uses RDF mappings; the json and xml formats expose a structured object based on how Drupal sees the entity. Access to these alternate formats through the REST API corresponds to permissions on the entity, so anyone with access content permission can view the JSON-LD version of that content. This is new as of Drupal 8.2.
For more information on interacting with Drupal entities via REST requests, see An Introduction to RESTful Web Services in Drupal 8.
"},{"location":"user-documentation/linked-data/#json-ld-module","title":"JSON-LD module","text":"Using the RDF mapping configurations provided by the RDF module, the JSON-LD Module exposes the RDF-mapped entity in JSON-LD, through the REST API, at node/[nid]?_format=jsonld (for nodes; for media and terms, at media/[mid]?_format=jsonld and taxonomy/term/[tid]?_format=jsonld).
types (which maps to rdf:type - see below, under Structure of an RDF Mapping).islandora module uses this hook to trigger any \"Map URI to Predicate\" and \"Alter JSON-LD Type\" reactions that are configured in Contexts. The Islandora Starter Site provides two Contexts - \"All Media\" and \"Content\" - that configure these to occur on Media and Repository Item nodes.?_format=jsonld to be part of, or not part of, the URIs of Drupal objects. On an out-of-the-box islandora-playbook, this string is stripped, but by default on a fresh install of the jsonld module, it is not.{\n\"@graph\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/8\",\n\"@type\":[\n\"http:\\/\\/pcdm.org\\/models#Object\"\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/title\":[\n{\n\"@value\":\"lasmomias de uninpahu\",\n\"@language\":\"fa\"\n}\n],\n\"http:\\/\\/schema.org\\/author\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/en\\/user\\/1\"\n}\n],\n\"http:\\/\\/schema.org\\/dateCreated\":[\n{\n\"@value\":\"2019-06-04T14:32:05+00:00\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#dateTime\"\n}\n],\n\"http:\\/\\/schema.org\\/dateModified\":[\n{\n\"@value\":\"2019-06-04T17:02:51+00:00\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#dateTime\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/description\":[\n{\n\"@value\":\"mpermmbklmh\",\n\"@language\":\"fa\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/created\":[\n{\n\"@value\":\"2015-10-15\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#string\"\n},\n{\n\"@value\":\"2015-10-15\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#date\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/extent\":[\n{\n\"@value\":\"1 item\",\n\"@type\":\"http:\\/\\/www.w3.org\\/2001\\/XMLSchema#string\"\n}\n],\n\"http:\\/\\/pcdm.org\\/models#memberOf\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/7\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/type\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/3\"\n}\n],\n\"http:\\/\\/purl.org\\/dc\\/terms\\/subject\":[\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/27\"\n}\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/en\\/user\\/1\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Person\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/node\\/7\",\n\"@type\":[\n\"http:\\/\\/pcdm.org\\/models#Object\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/3\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Thing\"\n]\n},\n{\n\"@id\":\"http:\\/\\/future.islandora.ca\\/taxonomy\\/term\\/27\",\n\"@type\":[\n\"http:\\/\\/schema.org\\/Thing\"\n]\n}\n]\n}\nIf using the Islandora Starter Site, the RDF mappings are set in the config files with names like rdf.mapping.[...].yml. The starter site config files will override configs set by modules. If you are building a site from scratch (not using the Islandora Starter Site, there are relevant configs in the following folders:
[drupal modules directory]/islandora/modules/islandora_core_feature/config/install/ (media and taxonomy terms)[drupal modules directory]/controlled_access_terms/modules/controlled_access_terms_defaults/config/install/ (the default corporate_body, family, geo_location, person, resource_type and subject vocabularies)[drupal web root]/core/profiles/standard/config/install/ (articles, pages, comments, and tags).Once loaded by modules, configuration .yml files are not live so editing them will not change the existing configuration. However, for modules that are Features, it is possible to re-import the changed configuration files at admin/config/development/features (todo: link to further reading on Features).
Once loaded, RDF mappings can be customized for the needs of a particular site through Drupal's Configuration Synchronization UI at admin/config/development/configuration. They can be exported, modified, and re-imported one-at-a-time by choosing the \"Single Item\" option on the Export/Import tabs. You can also create new RDF mappings (e.g. for a custom content type) and load them through this interface, by copying an existing mapping and changing the appropriate values.
Contributed module for RDF Mappings
A custom module rdfui exists, and is installed-but-not-enabled on boxes provisioned by the islandora-playbook. We don't use it because it is very rudimentary and limited to the schema.org vocabulary. We have an open ticket to develop a UI to support RDF mappings to any ontology. Contributions welcome.
ldp, ebucore, pcdm, are premis are registered in islandora.module using hook_rdf_namespaces(). To register your own namespaces, you will need to create a custom module that implements that hook.Below is an example of an RDF mapping as a .yml (YAML) file. It is the RDF mapping (current at time of writing) of the Repository Item (islandora_object) bundle, provided by the Islandora Starter Site and exportable as rdf.mapping.node.islandora_object.yml).
types specifies the rdf:type of the resource or content model. field_model, a required field of Islandora objects, also gets mapped to rdf:type through an arcane back-end process.fieldMappings specifies fields attached to that bundle and their RDF property mappings. One field can be mapped to more than one RDF property. It is a simple flat list.mapping_type:: There are several mapping types which are provided out of the box. - rel - standing for relationship, expresses a relationship between two resources - property - the default, or if a relationship is not provided, expresses the relationship between a resource and some literal text.
datatype_callback: This is a custom function that transforms the output of the field. There are some provided to us by Drupal and some added by Islandora, such as: - Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value which converts dates to ISO format - Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough which converts a referenced entity to the URI on the entity (which is configurable with the link_field argument An example usage of the Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough is as follows:
field_subject:\n properties:\n - 'dcterms:subject'\n datatype_callback:\n callable: 'Drupal\\jsonld\\EntityReferenceConverter::linkFieldPassthrough'\n arguments:\n link_field: 'field_authority_link'\nfield_authority_link of that subject's taxonomy term entity."},{"location":"user-documentation/linked-data/#sample-rdf-mapping","title":"Sample RDF mapping","text":"langcode: en\nstatus: true\ndependencies:\n config:\n - node.type.islandora_object\n module:\n - node\nid: node.islandora_object\ntargetEntityType: node\nbundle: islandora_object\ntypes:\n - 'pcdm:Object'\nfieldMappings:\n field_alternative_title:\n properties:\n - 'dc:alternative'\n field_edtf_date:\n properties:\n - 'dc:date'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_edtf_date_created:\n properties:\n - 'dc:created'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_edtf_date_issued:\n properties:\n - 'dc:issued'\n datatype_callback:\n callable: 'Drupal\\controlled_access_terms\\EDTFConverter::dateIso8601Value'\n field_description:\n properties:\n - 'dc:description'\n field_extent:\n properties:\n - 'dc:extent'\n field_identifier:\n properties:\n - 'dc:identifier'\n field_member_of:\n properties:\n - 'pcdm:memberOf'\n mapping_type: rel\n field_resource_type:\n properties:\n - 'dc:type'\n mapping_type: rel\n field_rights:\n properties:\n - 'dc:rights'\n field_subject:\n properties:\n - 'dc:subject'\n mapping_type: rel\n field_weight:\n properties:\n - 'co:index'\n title:\n properties:\n - 'dc:title'\n created:\n properties:\n - 'schema:dateCreated'\n datatype_callback:\n callable: 'Drupal\\rdf\\CommonDataConverter::dateIso8601Value'\n changed:\n properties:\n - 'schema:dateModified'\n datatype_callback:\n callable: 'Drupal\\rdf\\CommonDataConverter::dateIso8601Value'\n uid:\n properties:\n - 'schema:author'\n mapping_type: rel\nDrupal 8 recognizes files (such as images, audio files, video files, etc.) but wraps each file in an intermediate structure called a \"media\" to allow us to attach fields to files. Drupal uses a media's fields to store information about the media's file, such as file size, width and height (for images), alt text (for images), creation date, and so on.
Compared to Islandora Legacy
In Islandora Legacy, this sort of technical metadata would have been stored as datastream properties or as additional metadata-specific datastreams. In Islandora, each datastream holds its technical metadata using an associated media entity.
Fedora will store the media's file as a binary resource and use the media's properties as the binary resource's description.
For example, creating a media with a file named test.jpg (in November 2019) will create a Fedora binary resource of the file accessible at /fcrepo/rest/2019-11/test.jpg with the media's fields accessible at /fcrepo/rest/2019-11/test.jpg/fcr:metadata.
Islandora Core Feature and the Islandora Starter Site make use of the media types provided by the Standard Drupal installation (Video, Audio, etc). The file extensions allowed by each media type have been configured at the Drupal level. It is possible to create your own media types, and/or to edit the allowed field types and functionality of the existing media types. However, with Islandora Starter Site, the Image media type only allows .png, .gif, .jpg or .jpeg files. Many large images such as TIFFs (.tiff files) and JP2s (.jp2) must be added in the File media type instead of Image.
"},{"location":"user-documentation/media/#media-revisions","title":"Media revisions","text":"The metadata associated with a file can be updated by clicking the Edit tab while on the media's page and clicking Save.
The Create new revision checkbox is selected by default which will prompt Fedora to make a new version of the media's metadata before updating its resource record. A message can be added to the revision which is stored in Drupal but is not currently saved in Fedora.
"},{"location":"user-documentation/media/#using-the-media-form-to-replace-an-existing-file-does-not-behave-as-expected","title":"Using the Media form to replace an existing file does not behave as expected.","text":"The media edit form allows a user to remove a file and replace it with a new one. However, because of the relationship Islandora creates between a file and its media, the effects of removing a file and uploading a new one are not intuitive.
First, the remove button removes the file's reference on the media but does not delete the file, which, if in Fedora, will remain in Fedora.
Second, because Drupal does not want users to reuse a file path uploading a file, even with the same name in the same month, Drupal will rename the file for you. This will result in a new binary resource in Fedora, rather than replacing the existing resource, so the file's URI in Fedora will change
Third, the metadata synced from the Media into Fedora at [old Fedora URI]/fcr:metadata will remain in Fedora, but the metadata in the Media will also be synced into Fedora at [new Fedora URI]/fcr:metadata. There will be no way for the metadata in Fedora describing the file at the old Fedora URI to be edited.
Removing Media with Actions
To completely delete a file and its metadata from Fedora and Drupal, run the \"Delete media and file(s) action\" after selecting the Media in the general media list (/admin/content/media). This will cause the paths to the file and its metadata in Fedora to return 404s.
Replacing Media via REST
It is possible to use Islandora's REST interface to replace Media and Files.
"},{"location":"user-documentation/media/#media-ownership","title":"Media ownership","text":"Islandora objects can have any number of media associated with them. Media store a reference to the resource node they belong to using a special field, \"Media Of\". By changing this field's value, you can change which resource node owns the media, and therefore, where it gets displayed or managed.
Compared to Islandora Legacy
The direction of the relationship between objects and datastreams is reversed when compared to Islandora 7. Generally speaking, objects are unaware of their datastreams, and it's a Drupal view that lists datastreams for an object.
"},{"location":"user-documentation/media/#media-use","title":"Media use","text":"Islandora media express their intended use with a special \"Media Use\" field, which accepts taxonomy terms from the \"Media Usage\" vocabulary. Because the Media Usage vocabulary is an ordinary Drupal vocabulary, Islandora site administrators can create additional terms, and in turn, these local terms can be used to identify media that have some custom local purpose. However, most of the default set of \"Media Use\" terms are taken from the PCDM Use Extension vocabulary:
Compared to Islandora Legacy
Terms from the Media Usage vocabulary are very similar to DSIDs in Islandora Legacy. The only difference is that a DSID is immutable, but a media's usage can be changed at any time through the media's edit form.
"},{"location":"user-documentation/media/#derivatives","title":"Derivatives","text":"Islandora generates derivatives based on the Media Use term selected for a Media and the Model of the node that owns it. All of this is configurable using Context.
By default, derivatives are generated from Media with the \"Original Files\" term selected. When an Original File is uploaded, if the node that owns it has an \"Image\" model, image derivatives are created. If it's a \"Video\", then video derivatives are generated, etc...
By default, derivatives are created with a path determined by the current year, current month, the associated resource node' identifier assigned by Drupal, and the type of derivative. For example, a resource node found at node/2 with a service file media generated in October 2019 will have the path \"2019-10/2-Service File.jpg\". Naming conventions can be configured by editing each derivative action's File path settings.
Within a node's media tab, you can see all of its media, including derivatives, listed along with their usage. For example, from the Original File, a lower quality \"Service File\" and a smaller \"Thumbnail Image\" file were generated.
For more information on how to configure derivatives, see the section on Context.
"},{"location":"user-documentation/media/#multi-file-media","title":"Multi-File Media","text":"An alternate method of using Media to store files and derivatives is known as \"multi-file\" or multifile media. This method allows a single resource node to have multiple \"Original Files\" or \"Preservation Masters\". This method was implemented by UPEI's Research Data Management project, to represent datasets that may include multiple important files that require equal preservation treatment and derivatives (such as a spreadsheet and accompanying data dictionary), while being described by a single set of descriptive metadata.
There are no multi-file media bundles that currently ship with Islandora. The following recipe describes how to set up multi-file media:
A video tutorial explaining how to configure multi-file Media is available under the following URL: https://www.youtube.com/watch?v=3U6tBvD8oJY
"},{"location":"user-documentation/metadata-harvesting/","title":"Metadata harvesting","text":""},{"location":"user-documentation/metadata-harvesting/#oai-pmh","title":"OAI-PMH","text":"The Open Archives Initiative Protocol for Metadata Harvesting, commonly referred to as OAI-PMH, is a specification for exposing repository metadata for harvesting. OAI-PMH specifies six services which can be invoked over HTTP(s). The full specification details the services:
Service URL on localhost:8000 Identify http://localhost:8000/oai/request?verb=Identify ListMetadataFormats http://localhost:8000/oai/request?verb=ListMetadataFormats ListSets http://localhost:8000/oai/request?verb=ListSets GetRecord http://localhost:8000/oai/request?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:localhost:node-1 ListIdentifiers http://localhost:8000/oai/request?verb=ListIdentifiers&metadataPrefix=oai_dc ListRecords http://localhost:8000/oai/request?verb=ListRecords&metadataPrefix=oai_dcThese services are provided by the OAI-PMH Drupal REST OAI-PMH module Drupal module. The Islandora Starter Site provides some default configuration so that repository content can be harvested. This configuration used to be part of Islandora OAI-PMH, a submodule of Islandora Defaults.
For OAI-PMH functionality, the Islandora Starter Site provides:
/admin/config/services/rest/oai-pmh) to connect this view with the REST OAI-PMH module.The REST OAI-PMH module indexes (caches) set membership, so new Items may not appear immediately in their respective sets. Indexing will happen automatically during cron runs but can be triggered manually at /admin/config/services/rest/oai-pmh/queue.
The OAI-PMH module makes use of one of two modules to provide metadata mappings: the RDF module or the Metatag module. By configuring OAI-PMH to use the RDF module (appears as \"OAI Dublin Core (RDF Mapping)\" and is enabled by default in the Islandora Starter Site), the OAI-PMH module will use the RDF mapping as configured for your content type (the same mapping that is used for Fedora and Blazegraph, e.g. rdf.mapping.node.islandora_object.yml).
However,
Field values not showing up in OAI-DC record?
If you want the value of a field to be emitted in the OAI-DC record, you must assign a Dublin Core predicate for that field in your content type's RDF mapping. If you are wondering why a field is not showing up in the OAI-DC record, the content type's RDF mapping is the first thing to check.
The REST OAI-PMH module does not support metadata formats other than OAI-DC, but it supports some alternate methods of defining mappings to OAI-DC. Consult that module's documentation for more information.
"},{"location":"user-documentation/metadata-harvesting/#creating-additional-metadata-formats","title":"Creating additional metadata formats","text":"This involves creating a new plugin.
The Drupal rest_oai_pmh module's DefaultMap plugin provides a basic model to follow for creating a plugin.
Exact implementation of your plugin will depend on your data model. The rest_oai_pmh module by default expects a flat list of fields and field values to output. This means that if your data model uses anything like Typed Relation field types, Paragraphs, or other complex nested entity modeling, you will need to add custom logic to build the values to emit via OAI-PMH for those fields.
"},{"location":"user-documentation/metadata/","title":"Metadata in Islandora","text":"Metadata vs Fields in the Starter Site
To learn about the default out-of-the-box metadata fields, see Starter Site Metadata Configuration. This page describes technical details about how metadata is handled as fields in Drupal and provides a deeper understanding of, and tools for modifying, your metadata configuration.
1-minute synopsis: In Islandora, metadata is stored in Drupal, in fields attached to entities (nodes or media). This allows us to interact with metadata (add, edit, remove, display, index in a search engine...) almost entirely using standard Drupal processes. If exporting this metadata to Fedora and/or a triplestore, the values are serialized to RDF using mappings that can be set for each bundle.
Drupal 8 Terminology
In Drupal 8, Fields can be attached to bundles (sometimes called entity sub-types -- e.g. Content types, Media types, Vocabularies) or entities (e.g. Users). For more on Fields, see \"2.3 Content Entities and Fields\" and \"6.3 Adding Basic Fields to a Content Type\" in the Official Drupal Guide.
As described in the resource nodes section, Islandora digital objects are comprised of Drupal nodes for descriptive metadata, Drupal media for technical metadata, and Drupal files for the binary objects. This section describes how Islandora uses and extends Drupal fields to manage descriptive metadata.
"},{"location":"user-documentation/metadata/#content-types","title":"Content Types","text":"In Drupal, Nodes come in different subtypes called Content Type. These let you define a type of content (\"Article\" and \"Basic Page\" are Drupal defaults and \"Repository Item\" is an Islandora specific example), the set of metadata fields that are attached to that content, and how those fields can be edited and displayed. Each Content Type is essentially a metadata profile that can be used for a piece of web content, or to describe a digital resource. You can create your own Content Types for your Islandora project or use a pre-defined one like Repository Item from the Islandora Starter Site. We will go over the metadata specific aspects of Content Types below, but for a fuller walk-through of creating a Content Type see here.
Not all Content Types in your Drupal site need be Islandora Resource Nodes. Making a Content Type a Resource Node will associate Islandora specific behaviours (such as syncing to Fedora or causing derivatives to be generated) with it. The decision to make a content an Islandora resource node is left to the discretion of the site manager. In Islandora, a \"resource node\" is usually considered a descriptive record for \"a thing\", and is conceptually similar to an \"Islandora Object\" in 7.x, i.e. a \"Fedora Object\" in Fedora 3.x and below. Read more on configuring a Content Type to be treated as a Resource Node.
"},{"location":"user-documentation/metadata/#fields","title":"Fields","text":"The administrator will define the fields that are associated with a specific Content Type. The same fields can be applied to different Content Type, but the field display and editing configurations are unique to each Content Type. The names and definitions of these fields are specific to Drupal and do not have to correspond to an outside metadata schema. You will give each field a Label, Machine Name, and a specific Field Type, like Text, Integer, EDTF, or Entity Reference (see below). Specific to the Field Type you will then define the maximum length of the field, the number of values it can contain, and what taxonomies it might link to.
Fields can be added under Administration >> Structure >> Content types >> Your Content Type's Name >> Manage fields (/admin/structure/types/your_type/fields). This tab will list all Fields, their Label, Machine Name, Field Type, and give you the option to make what edits to the definition of that field that you can.
Certain decisions must be made when fields are created, and before any content is added, because they can not be changed later. Field Type can not be changed, so you wouldn't be able to change a text field to a taxonomy field after creation. The field's machine name also can't be changed. The number of values allowed in a field or its maximum length or type of item to reference (in the case of Entity reference fields) can not be changed after content has been added. You can, however, always add new fields to a Content Type, even after content has been added.
7.x Migration Note: What About My MODS XML?
Even when using the Islandora Starter Site, there is no \"official\" metadata schema in Islandora. Where Islandora 7.x used MODS, and took advantage of its hierarchical/extensible structure, Drupal Fields are a flat structure working with distinct, individual elements. You can base your fields on those in MODS, or any other schema, but that structure is up to you. The Metadata Interest Group has developed a sample MODS-Drupal-RDF mapping, which provides a structure upon which you can build your Drupal fields. It is used by the Repository Item Content Type in the Islandora Starter Site.
You Cannot Change The Content Type Of A Node
Once a node is created, its Content Type cannot be changed. Just as you are unable to change many aspects of a Field once it has been created, once a node has been created it is now permanently of that Content Type and the fields associated with it. At that point your only option would be to create a new node of the intended content type, map the field values (programmatically or by copy-paste), and update any media or children that refer to the old node to refer to the new one.
The Islandora Starter Site provides a Repository Item Content Type that can be used as a structure to build your collection around, or it can be used as a sample to see how fields in Content Type work. It pre-defines fields, including Alternative Title and Date Issued that could be of use in many digital repositories. The full list of fields and their field types can be seen in the screenshot below.
Titles Aren't Conventionally-Configurable Fields
The field title is built-in to each Content Type by default, and can be referenced in views, templates, and indexing like other fields, but it cannot be configured like other fields. The only aspect you can change about title is its label. It has a built-in maximum length of 255 characters which cannot be changed. If your content requires longer titles we recommend you create a separate \"long_title\" field to store the full title and reserve the default title field for a display title. There is a contributed module called Node Title Length, which allows an administrator to configure the length of the title field in the core node table. However, this only works on nodes (not media or other entities) and involves meddling in a core Drupal database schema, which makes some people uneasy.
"},{"location":"user-documentation/metadata/#content-entry-formmanage-form-display","title":"Content Entry Form/Manage Form Display","text":"After creating the Fields for a Content Type you'll be able to manage the form used by content creators to create Nodes of that Content Type. On the Manage form display tab you'll be able to edit this form by arranging the order of the fields, choose what Widget will define the entry options for a field, and then set certain settings for that Widget. Fields are arranged by dragging the cross to the left of the Label. They can also be removed from the form, but not the Content Type, by dragging them to the bottom of the list under the Disabled heading. Widgets are defined by Field Type, so an Entity reference field could use auto complete, a select list, or even checkboxes, and are chosen from a drop-down list. The widget settings are accessed through the gear on the far right of a row and may allow you to set the size of an entry field, whether the field Label is displayed, or if you use placeholder text.
"},{"location":"user-documentation/metadata/#content-displaymanage-display","title":"Content Display/Manage Display","text":"The Manage display tab is where you will make decisions about how to display the metadata. Order is arranged as above, and can again be dragged to the Disabled section to hide the field from display. You can choose whether a field's label is displayed above the value, in-line, or hidden.
"},{"location":"user-documentation/metadata/#vocabularies","title":"Vocabularies","text":"See also: MIG Presentation on Taxonomies by Kristina Spurgin, 2021-07-19
In Drupal, Taxonomy Vocabularies (or simply Vocabularies) are also entity subtypes that define a set of fields and their configurations. Whereas instances of content types are called nodes, items in a vocabulary are called taxonomy terms (or simply terms). Traditionally, taxonomy terms are used to classify content in Drupal. For instance, the Article Content Type includes a field field_tags that can refer to terms in the Tags vocabulary.
There are two ways that users can interact with taxonomies: they can be \"closed,\" e.g. a fixed list to pick from in a dropdown, or \"open,\" e.g. field_tags where users can enter new terms, which are created on the fly. This is not set on the vocabulary itself, but in the configuration of the field (typically on a node). Terms within vocabularies have an ordering, and can have hierarchical structure, but do not need to.
Islandora (through the Islandora Core Feature) creates the 'Islandora Models' vocabulary which includes the terms 'Audio', 'Binary', 'Collection', 'Compound Object', 'Digital Document', 'Image', 'Newspaper', 'Page', 'Paged Content', 'Publication Issue', and 'Video'. Islandora Starter Site provides contexts that cause certain actions (e.g. derivatives to happen, or blocks to appear) based on which term is used.
The Controlled Access Terms module provides additional vocabularies:
Each of these vocabularies has its own set of fields allowing repositories to further describe them. The Repository Item Content Type has fields that can reference terms in these vocabularies. See 'Entity Reference fields' in the 'Field Types' section below.
The vocabularies provided by default are a starting point, and a repository administrator can create whatever vocabularies are desired.
Large Taxonomy Vocabularies
The Drupal Taxonomy UI is known to break down when your vocabularies get large (e.g. over 20,000 terms). Jonathan Hunt created the CCA Taxonomy Manager module for SFU to solve this problem.
"},{"location":"user-documentation/metadata/#field-types","title":"Field Types","text":"Fields are where descriptive and administrative metadata about Drupal entities is stored. There are different types of fields including boolean, datetime, entity reference, integer, string, text, and text_with_summary. These field types also have widgets (controlling how data is entered) and formatters (controlling how data is displayed). The Drupal 8 documentation on FieldTypes, FieldWidgets, and FieldFormatters includes a list of the core field types with brief definitions, along with a list of core widgets and formatters. Custom field types can be created to represent data in ways not provided by these core options.
More field types, formatters, and widgets are available in various modules. For example, the controlled_access_terms module provides two additional field types designed specifically for use with Islandora: ETDF, and Typed Relation. These and the Entity Reference field type are described in more detail below, since they are of particular interest for Islandora users.
"},{"location":"user-documentation/metadata/#authority-link","title":"Authority Link","text":"The Authority Link data type configures fields to hold two associated values:
Within Islandora, this data type is used by a metadata field in Taxonomy Vocabularies called Authority Sources to capture equivalent representations of terms from external authority sources.
Tip
The term external authority source refers to both controlled vocabularies like Art & Architecture Thesaurus or FAST as well as Name Authority Files like Library of Congress Name Authority File or VIAF.
For instance, if you are creating a term called Red squirrels within the default Taxonomy Vocabulary Subject, you may want to include the URI for Tamiasciurus from the FAST (Faceted Application of Subject Terminology) vocabulary. If you configured the field Authority Sources to list FAST (Faceted Application of Subject Terminology) as an external source authority option, you can select this source and add the associated URI (http://id.worldcat.org/fast/1142424).
"},{"location":"user-documentation/metadata/#configurations-for-authority-sources-field","title":"Configurations for Authority Sources field","text":"Each Taxonomy Vocabulary can have different external source authority options for its Authority Sources field. To configure the Authority Sources field to change these options, navigate to Home-->Administration-->Structure-->Taxonomy-->Edit Taxonomy Vocabulary Name-->Taxonomy Vocabulary Name and select \"Edit\" for the Authority Sources field. Then enter your options in the Authority Sources text box, entering one value per line in the format key|label. The key is the stored value (typically an abbreviation representing the authority source). The label will be used in displayed values and editing forms.
By default, this field is repeatable. To change this, edit the \"Field settings\" and change Allowed numbers of values from \"Unlimited\" to \"Limited\" and enter the number of allowable values. This will apply to every instance of the Authority Sources field across your Taxonomy Vocabularies. You cannot change the repeatability of Authority Sources after data has been entered in the field.
"},{"location":"user-documentation/metadata/#entity-reference","title":"Entity Reference","text":"Entity Reference fields are a special type of field built into Drupal core that creates relationships between entities. The field's configuration options include (but are not limited to):
The Repository Item content type, provided by the Islandora Starter Site, includes several entity reference fields that reference vocabularies defined by the islandora and controlled_access_terms modules.
"},{"location":"user-documentation/metadata/#configurations-for-entity-reference-field","title":"Configurations for Entity Reference field","text":"The screenshots below show how you can configure an entity reference field (in this case the Subject field on the Repository Item content type).
Tip
Note that once the type of entity to reference has been defined, and data has been created, it cannot be changed.
Storage settings for entity reference field where you set whether the field will reference content nodes or taxonomy terms:
Reference type settings for entity reference field where you select which vocabularies the autocomplete utility should query when editors are entering data:
Data Consistency
Selecting which vocabularies can be referenced from an entity reference field only affects which vocabularies will be searched when a user types into the autocomplete field in the Drupal form for adding a new item. These settings do not impose constraints on the underlying database, so it is still possible to load references to other vocabularies without being stopped or warned when ingesting data through various migration methods.
"},{"location":"user-documentation/metadata/#edtf","title":"EDTF","text":"The EDTF field type is defined in the controlled_access_terms module, and designed for recording dates in Extended Date Time Format, which is a format based off of the hyphenated form of ISO 8601 (e.g. 1991-02-03 or 1991-02-03T10:00:00), but also allows expressions of different granularity and uncertainty. The Default EDTF widget has a validator that only allows strings that conform to the EDTF standard. The Default EDTF formatter allows these date strings to be displayed in a variety of human-readable ways, including big- or little-endian, and presenting months as numbers, abbreviations, or spelling month names out in full. Close review of the EDTF Specifications is recommended when configuring this field type.
Endianness
Big-endian = year, month, day. Little-endian = day, month, year. Middle-endian = month, day, year.
Known EDTF Bug
When configuring the EDTF widget for a field in a content type, you can choose to allow date intervals (aka date ranges), but doing this prevents the widget from accepting values that include times. (The EDTF standard states that date intervals cannot contain times, but the field should be able to accept either a valid EDTF range or a valid EDTF datetime, so this is a bug.)
Example of valid inputs in a multivalued EDTF Date field (including the seasonal value 2019-22 as defined in the EDTF specification):
Example of the same EDTF dates displayed using little-endian format:
EDTF field values cannot include textual representations of dates, as shown below in this example of a valid EDTF value ('1943-05') and an invalid value ('May 1943') with the corresponding error message. Use the formatter configurations detailed further below to achieve textual display of dates.
"},{"location":"user-documentation/metadata/#configuration-for-the-default-edtf-widget","title":"Configuration for the Default EDTF Widget","text":"This configuration can be set per field by clicking the gear icon next to any field defined with EDTF field type at Administration >> Structure >> Content types >> Repository Item >> Manage form display (admin/structure/types/manage/islandora_object/form-display)
Configuration options include strictness level of date validation, allowing date intervals and allowing date sets.
"},{"location":"user-documentation/metadata/#configuration-for-the-default-edtf-formatter","title":"Configuration for the Default EDTF Formatter","text":"This configuration can be set per field by clicking the gear icon next to any field defined with EDTF field type at Administration >> Structure >> Content types >> Repository Item >> Manage display (admin/structure/types/manage/islandora_object/display)
Example of how the EDTF formatter settings can change the display of an EDTF value:
"},{"location":"user-documentation/metadata/#configuration-for-indexing-and-sorting-edtf-fields-in-search-results","title":"Configuration for indexing and sorting EDTF fields in search results","text":"By default, EDTF date values are indexed in Solr as string values. The entered value (not the displayed value) is indexed.
Solr
The Solr string data type requires the full field value to match the query in order to count as a match. This means that searching for 2014 will not retrieve a record where the recorded date value is 2014-11-02.
EDTF date fields may be configured as sort fields in your search results Views. By default, this results in a simple ordering by the literal EDTF date string.
An EDTF date field with multiple or unlimited number of allowed values may be set as a sort field. In this case, the first occurrence of the field value is used as the sorting value.
"},{"location":"user-documentation/metadata/#typed-relation","title":"Typed Relation","text":"The Typed Relation field is defined in the controlled_access_terms module, is an extension of Drupal's Entity Reference field type, and allows the user to qualify the type of relation between the resource node and other entities, such as taxonomy terms. For example, it enables the inclusion of a resource's contributor's (assuming contributor names are modelled as taxonomy terms or some other Drupal entities) as well as their roles (such as \"author\", \"illustrator\", or \"architect\") in the resource node itself. Using only Drupal's Entity Reference fields, we would need individual fields for \"author\", \"illustrator\", \"architect\", and any other roles that may need to be made available. Using a Typed Relation field, we can have one Entity Reference field for \"Contributors\" and let the user pick the affiliated role from a predefined dropdown list.
Typed relation name
The parts of a field are called properties, so 'entity reference' and 'relation type' are properties of the Typed Relation field type.
"},{"location":"user-documentation/metadata/#configurations-for-the-typed-relation-field","title":"Configurations for the Typed Relation field","text":"The Islandora Starter Site includes a Typed Relation field labelled 'Contributors' as part of the Repository Item content type, and populates the available relations from the MARC relators list. This field was formerly called \"Linked Agent\".
The list of available relations for this Contributors field is configurable at '/admin/structure/types/manage/islandora_object/fields/node.islandora_object.field_linked_agent'.
Typed relation tradeoffs
Publishers
Until Mar 2024, the Islandora Starter Site included publishers in the Contributors (field_linked_agent) field. The MIG made the decision to make publisher its own text field, in order to make it easier to separate publishers from other contributors, and to prevent clutter in the linked taxonomies. Publishers are often recorded by transcribing what is on the item, rather than formatting the name per Authority rules, so variations on a single name are expected.
Relations are defined in the format key|value, and the key is used in the RDF mapping (see below).
By default, facets can be created for typed relation fields that will facet based on the linked entity alone, not separating references based on the relationship type.
"},{"location":"user-documentation/metadata/#adding-a-new-field","title":"Adding a new field","text":"You are free to add new fields to your Islandora content type(s). After you set one up, you may want to configure the following:
rest_oai_pmh module's templates/mods.html.twig. For example, that template file includes {{ elements.publisher }} so the label in views needs to be exactly publisher. Save the View.An example of this, in terms of config changes, is visible in the changeset of the Starter Site's 1.6.0 version for the Publisher field. However, pulling such a changeset is unlikely to work smoothly and it is recommended to set these configurations manually.
"},{"location":"user-documentation/metadata/#getting-metadata-into-fedora-and-a-triple-store","title":"Getting Metadata into Fedora and a Triple-store","text":"Depending on the needs at your institution, you may or may not be using Fedora with your Islandora installation. You also may or may not be hoping to publish your metadata as RDF triples that can be queried in a triplestore. Both of these functionalities are driven by the JSON-LD module (written for Islandora), which provides a JSON-LD serialization of your content nodes, media nodes, as well as your taxonomy terms. This JSON-LD is what gets ingested by Fedora, and is also what is used to add RDF triples to the blazegraph triplestore if you choose to use that service.
The JSON-LD serialization for an entity is available by appending _format=jsonld to the entity's URL. Below is an example JSON-LD document representing the RDF serialization of a Repository item node created in a standard islandora-playbook based vagrant VM:
{\n \"@graph\":[\n {\n \"@id\":\"http://localhost:8000/node/1\",\n \"@type\":[\n \"http://pcdm.org/models#Object\"\n ],\n \"http://purl.org/dc/terms/title\":[\n {\n \"@value\":\"New York, New York. A large lobster brought in by the New England fishing boat [Fulton Fish Market]\",\n \"@language\":\"en\"\n }\n ],\n \"http://schema.org/author\":[\n {\n \"@id\":\"http://localhost:8000/user/1\"\n }\n ],\n \"http://schema.org/dateCreated\":[\n {\n \"@value\":\"2019-03-14T19:05:24+00:00\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#dateTime\"\n }\n ],\n \"http://schema.org/dateModified\":[\n {\n \"@value\":\"2019-03-14T19:20:51+00:00\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#dateTime\"\n }\n ],\n \"http://purl.org/dc/terms/date\":[\n {\n \"@value\":\"1943-05\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n },\n {\n \"@value\":\"1943-05\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#gYearMonth\"\n }\n ],\n \"http://purl.org/dc/terms/extent\":[\n {\n \"@value\":\"1 negative\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/identifier\":[\n {\n \"@value\":\"D 630714\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/type\":[\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/11\"\n }\n ],\n \"http://purl.org/dc/terms/rights\":[\n {\n \"@value\":\"No known restrictions. For information, see U.S. Farm Security Administration/Office of War Information Black & White Photographs(http://www.loc.gov/rr/print/res/071_fsab.html)\",\n \"@type\":\"http://www.w3.org/2001/XMLSchema#string\"\n }\n ],\n \"http://purl.org/dc/terms/subject\":[\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/26\"\n }\n ],\n \"http://schema.org/sameAs\":[\n {\n \"@value\":\"http://localhost:8000/node/1\"\n }\n ]\n },\n {\n \"@id\":\"http://localhost:8000/user/1\",\n \"@type\":[\n \"http://schema.org/Person\"\n ]\n },\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/11\",\n \"@type\":[\n \"http://schema.org/Thing\"\n ]\n },\n {\n \"@id\":\"http://localhost:8000/taxonomy/term/26\",\n \"@type\":[\n \"http://schema.org/Thing\"\n ]\n }\n ]\n}\nThe RDF mapping for a content type, media type, or vocabulary defines how fields in Drupal are mapped to properties in the JSON-LD serialization. The mapping defines the RDF predicates that should be used for each field. You reference Drupal fields via their Machine Name, and the RDF predicate by using the conventional syntax namespace:predicate. In this example, the dc prefix stands for http://purl.org/dc/terms/, so when concatenated the final RDF predicate is http://purl.org/dc/terms/title.
To show a small example, the RDF mapping:
title:\n properties:\n - dc:title\nwill map the Repository item's title field to http://purl.org/dc/terms/title. As a result, the node's title value appears like this in the JSON-LD output:
\"http://purl.org/dc/terms/title\":[\n {\n \"@value\":\"New York, New York. A large lobster brought in by the New England fishing boat [Fulton Fish Market]\",\n \"@language\":\"en\"\n }\n],\nTip
To set up prefixes for namespaces and see a list of available predefined namespaces, see the \"RDF Generation\" page.
"},{"location":"user-documentation/metadata/#typed-relation-fields-in-rdf","title":"Typed Relation fields in RDF","text":"Unlike other fields, which can be assigned RDF predicates in RDF Mapping YAML files, a typed relation field uses a different predicate depending on the chosen type. These predicates are assigned using the 'keys' in the key|value configuration. The key must be formatted namespace:predicate, e.g. relators:act.
Bug
The Drupal RDF module is currently limited in the complexity of graph you can generate. All fields must be mapped directly to either a literal value, or a reference to another Content Type instance, media type instance, or taxonomy term instance. It is not currently possible to create blank nodes or skolemized nodes for nesting fields under more complex structures.
"},{"location":"user-documentation/metadata/#batch-editing-metadata-in-fields","title":"Batch editing metadata in fields","text":"If you are editing multiple resources in order for them to have the same metadata value, the Views Bulk Edit module can help. Here is a video of creating a view using Views Bulk Operations to apply a subject term to multiple resources simultaneously.
For more complex changes, or when the values need to differ for each value, an export-modify-reimport method may be needed. Use a view to export CSV or other structured data (including an identifier such as a node id), modify the values as necessary, then use migrate csv or Workbench to re-import and update the values.
"},{"location":"user-documentation/metadata/#exporting-data","title":"Exporting Data","text":"One common approach for exporting your content and/or taxonomy data out of Islandora is to use Drupal's Views Data Export module. The module has extensions that can allow you to configure exports as CSV, XML, text files, and other formats based on your local needs.
"},{"location":"user-documentation/metadata/#drupal-bundle-configurations","title":"Drupal Bundle Configurations","text":"In Islandora, content models are primarily created using content types (also known as node bundles) and media bundles. Bundles are defined by YAML configuration files. To create new content models, one would create the needed content types and media bundles via UI, then export the yml files related for those bundles using Configuration Synchronization (http://localhost:8000/admin/config/development/configuration) or Features. An understanding about the structure of a bundle and various configuration files used to define it helps in creating and updating it.
Content types and media bundles can be thought of as web forms consisting of fields. Drupal provides widgets to define the behavior of a field and field storage to define how the data is stored in the database. Drupal provides various display modes to show the forms to user when they are editing (Manage form display) or viewing (Manage display).
A content model is packaged as a module for installation. All yml files are put in config/install folder of the module. Note that not all content models would contain media bundles.
The following files define the bundles themselves. It contains some metadata about the bundle and lists its dependencies.
node.type.your_content_type.yml\nmedia_entity.bundle.your_media_bundle.yml\nThe following files define the fields attached to the bundle forms. There must be one config file for each field in your bundle, except for the default drupal fields.
field.field.node.your_content_type.field_name1.yml\nfield.field.node.your_content_type.field_name2.yml\n...\nfield.field.media.your_media_bundle.field_name1.yml\nfield.field.media.your_media_bundle.field_name2.yml\nIf the new bundle contains new fields, then field storage configurations for the newly created fields would be needed as well. Note that if you reused existing fields, storage definitions should not be defined again. Storage config contains information about the number of values allowed for that field (cardinality).
field.storage.node.field_new_name3.yml\nfield.storage.media.field__new_name3.yml\nThere is a configuration file for each combination of bundle / display mode when managing form displays. Usually, form displays will have default and inline modes.
core.entity_form_display.media.your_media_bundle.default.yml\ncore.entity_form_display.media.your_media_bundle.inline.yml\n---\ncore.entity_form_display.node.your_content_type.default.yml\ncore.entity_form_display.node.your_content_type.inline.yml\nThere is a configuration file for each combination of bundle / display mode when managing displays. Usually, displays will have default and teaser modes for content types and default and content modes for media bundles.
core.entity_view_display.media.your_media_bundle.default.yml\ncore.entity_view_display.media.your_media_bundle.content.yml\n---\ncore.entity_view_display.node.your_content_type.default.yml\ncore.entity_view_display.node.your_content_type.teaser.yml\nIn addition, Islandora needs an RDF mapping to express the content in RDF and to sync to fedora. There will be one RDF mapping per bundle.
rdf.mapping.media.your_media_bundle.yml\nrdf.mapping.node.your_content_type.yml\nIslandora enables you to build full-fledged multilingual repositories leveraging the multilingual support provided by Drupal core modules. The multilingual content gets indexed into Fedora repository as well as the Triplestore (Blazegraph), and can be queried using the SPARQL endpoint. In this guide, we will describe the steps needed to set up a basic multilingual Islandora site.
"},{"location":"user-documentation/multilingual/#drupal-concepts","title":"Drupal Concepts","text":"Drupal allows you to translate user interface text, configuration text, and content. See section 2.7 of the Drupal documentation for details.
"},{"location":"user-documentation/multilingual/#islandora-configuration","title":"Islandora Configuration","text":"Islandora enables Drupal's Language and Content Translation modules by default. Drupal provides additional modules for multilingual support, for instance for translating the built-in user interface or editable interface text.
"},{"location":"user-documentation/multilingual/#adding-languages","title":"Adding Languages","text":"From the top menu, go to Configuration >> Regional and language >> Languages. Add a language.
"},{"location":"user-documentation/multilingual/#adding-language-switcher","title":"Adding Language switcher","text":"You can place the default language selector block to switch between languages. To create the language switcher block go to Structure >> Block layout. Click Place block in a region of your choice. Search for Language switcher block and click Place block.
You can place the language switcher block in different regions of the user interface. You might have to customize the theme to style the language switcher block.
"},{"location":"user-documentation/multilingual/#adding-multilingual-menus","title":"Adding Multilingual Menus","text":"From the top menu, go to Configuration >> Regional and language >> Content language and translation. Check Custom menu link under Custom language settings. Scroll down to Custom menu link section and check all the relevant fields and Save the configurations. Clear the cache (Configuration >> Development >> Performance).
From the top menu, go to Structure >> Menu. Edit \"Main navigation\" menu. Default home menu item cannot be translated due to this issue. Disable that menu item. Click Add link to create a new menu item. Provide a menu title (i.e. Home) and input <front> for the link field. Save. Right-click on the Operations beside the new menu link and click the Translate button. Translate the menu link title for the language added above and save.
Go back to home. The language switcher will enable you to switch the language/content of the menu and content.
"},{"location":"user-documentation/multilingual/#adding-a-multilingual-repository-item","title":"Adding a Multilingual Repository Item","text":"From the top menu, go to Content >> Add content >> Repository item. Provide the required fields and save the object. Click the Translate tab of the object, provide a title in the second language and fill any translatable fields (i.e description). Add the media for the object. Media objects can be translated similar to the repository item node.
Go back to home, you should be able to view content in the language selected in the language switcher.
"},{"location":"user-documentation/multilingual/#field-label-translations","title":"Field Label Translations","text":"If you need the field labels of the repository Item displayed in a different language, additional configuration is needed. The Drupal module Configuration translation module in the core needs to be enabled. Note that this will enable the module User Interface translation as well.
Each field label needs to be translated through the Drupal GUI (Configuration >> Regional and Language: User interface translation). Alternatively, you can import existing translations, or translations generated with an external translation editor (for example Gtranslator). Go to Interface translation import (Configuration >> Regional and Language: User interface translation >> Import tab). Set the Treat imported strings as custom translations option, select the import file, the target language, and click import. Clear the cache to see the changes. An example second language display is shown below.
Islandora indexes the multilingual metadata values in Fedora as RDF literals using language tags. An example representation is shown below.
"},{"location":"user-documentation/multilingual/#triplestore-representation","title":"Triplestore Representation","text":"Islandora indexes the multilingual metadata values in Blazegraph as RDF literals using language tags. An example representation is shown below.
You can query the result and filter the results by a specific language. For example, to get all titles and filter by language, the following query can be used:
PREFIX dcterm: <http://purl.org/dc/terms/>\nselect ?s ?o {\n ?s dcterm:title ?o\nFILTER (lang(?o) = 'ta')\n}\nPaged content, such as books, periodicals, photographs with the front and back, etcetera, can use the membership structure provided by Islandora, namely, field_member_of. This involves creating a resource node for the root record (i.e. the entire book or the photograph) and child resource nodes for each sub-component (e.g. \"Page 1\", \"Page 2\", etc., or \"recto\" and \"verso\") with their corresponding media. Each \"child\" resource node contains a reference to their \"parent\" resource node using the field_member_of property.
Child v. Member
Islandora uses the \"child\" and \"member\" descriptor for resource nodes that store a reference to another resource node in the \"Member Of\" field interchangeably. Administrators will more often see the \"member\" terminology more often while front-end users will usually see \"child\" terminology.
Similar to the collection view showing members of a collection, Islandora provides taxonomy terms Paged Content and Page in the Islandora Model vocabulary, that can be used to trigger paged content behaviors. Islandora Starter Site provides a Context for Paged Content resource nodes to trigger displaying a Mirador viewer showing the child resource nodes' service files.
To enable this, use Paged Content for the Islandora Model of the parent resource node and Page for the Islandora Model of the child resource node. After the child resource nodes have \"service file\" media (either generated via the built-in derivative creation or added manually), they will be included in the Mirador paginated viewer displayed on the parent resource node's page. (See the IIIF Documentation for more details on controlling the viewer's behavior.)
However, repository managers can use any method they like, such as a Views-based list of teasers, a gallery, or a slide-show to display child resource nodes.
"},{"location":"user-documentation/paged-content/#ordering-weight","title":"Ordering (weight)","text":"By default, child resource nodes are un-ordered. To order the subcomponents of a paged content resource node, Islandora provides a Weight field to store an integer value on child resource nodes. Children resource nodes with smaller weight values will float to the top and come before child resource nodes with heavier weight values.
Weight values do not need to be sequential, just ordered from smallest to largest. For example, the first child resource node can have a value of 10 and the next could have a value of 20 and they will be ordered accordingly. Should a new child be added with the weight value of 15 it will automatically be sorted after the child with the weight value 10 and before the child with the weight value 20.
Child resource nodes can be reordered using a drag-and-drop interface by clicking on the Re-order Children button on the Children tab of the parent resource node.
Re-ordering children resource nodes on this page and clicking Save will cause each child resource node's weight value to be updated using sequential values.
Why not Drupal's book or weight modules?
Drupal provides the book module in core for creating multi-level ordered content, such as books and manuals. However, this module stores structure and pagination separately from the nodes making serializing these relationships as RDF we can provide to Fedora more difficult than simply using field_member_of with an RDF mapping. Support for the book module may be added in the future.
Drupal also has a weight module that provides a weight field and a drag-and-drop reordering user-interface. However, this module requires users to set a specified range of values which includes their negative corresponding value. E.g. a range setting of '20' will require all children to have a value between '-20' to '20'. This presumes a repository manager can predict how many pages the largest paged content item in their repository will be beforehand. Also, these weight values are serialized into RDF using the Collections Ontology 'index' predicate which assumes positive integer values which cannot be guaranteed using the weight module.
"},{"location":"user-documentation/paged-content/#adding-children","title":"Adding children","text":"Islandora provides an interface on the Children tab to either Batch upload children or Add Child. Both methods will result in new resource nodes that are member of the current node. It is also possible to create child nodes separately, then edit their Member Of field to point to the desired parent.
"},{"location":"user-documentation/paged-content/#batch-upload-children","title":"Batch upload children","text":"If you want to add a number of pages to this node, for example, the Batch Upload Children button may suit your needs. It allows you to upload multiple files, after selecting a content type for the child nodes and the media type and media use for the media that will hold the uploaded files. If selecting a node with the Model (field_model) field, it will also let you select a model from the Islandora Models vocabulary. These settings will apply to all nodes and media created for this batch.
This method does not create full metadata for the child nodes. It uses the filename as the node title. It also does not accept zip files; individual files must be uploaded. For more methods of bulk uploading content, see Islandora Workbench and Migrate Islandora CSV
"},{"location":"user-documentation/searching/","title":"Setup and Configure Search","text":"Islandora comes with the Drupal 8 Search API and SOLR modules enabled with a corresponding SOLR instance. This guide gives an overview to the setup provided by the islandora-playbook. Much more detail is available in the Search API documentation. Another helpful resource is Adam Fuch's \"Drupal 8 Custom Site Search with Search API\" (https://www.electriccitizen.com, 2018-01-10; last accessed 2019-03-08).
"},{"location":"user-documentation/searching/#indexing-islandora-with-solr","title":"Indexing Islandora with SOLR","text":"To access the search indexing settings, log in as an administrator and navigate to '/admin/config/search/search-api' or click Configuration and then Search API.
"},{"location":"user-documentation/searching/#solr-server-configuration","title":"SOLR Server Configuration","text":"On the Search API page, Use the Solr Server link to view the SOLR server's configuration and the Default Solr content index link to view or update the index settings. In most cases, where the site was built using Ansible, the Solr Server settings that were configured during installation should be left alone.
"},{"location":"user-documentation/searching/#solr-index-configuration","title":"SOLR Index Configuration","text":"The Default Solr content index user interface is divided into four tabs: View, Edit, Fields, and Processors.
"},{"location":"user-documentation/searching/#view-tab","title":"View Tab","text":"The 'View' tab gives an overview of the index and its status:
The View tab also provides links to some common actions. Start Indexing Now allows you to start an indexing job for a specified number of items (default is 'all'). You can also specify how many items should be indexed in each batch (default is '50'). The other links allow a repository manager to queue all objects for reindexing, clear the index, or rebuild tracking information.
"},{"location":"user-documentation/searching/#edit-tab","title":"Edit Tab","text":"The Edit tab allows repository managers to configure how the index works as a whole, including the Index name, the data sources \u2014 entity types \u2014 it can index (including which specific content types or taxonomies will be indexed), which server it is connected to, and other SOLR-specific options.
Content (types) is the only data source enabled by default. Selecting Taxonomy term will enable searching taxonomies which is recommended if the repository uses taxonomies for subjects or other discovery points. Once the data sources are enabled a configuration box for each of them will appear in a section just below the list of data sources. This allows repository managers to select which content types (or taxonomy vocabularies) will be included in the index. By default, all the content types, and vocabularies if the taxonomy data source is enabled, are indexed.
Defaults
The defaults assume a repository is adding content using the web interface. If a repository manager plans on bulk-loading content they should disable the Index items immediately option in the expandable Index Options box and increase the 'Cron batch size' option.
"},{"location":"user-documentation/searching/#fields-tab","title":"Fields Tab","text":"The Fields tab allows repository managers to select which fields will be indexed. The default set of fields enabled come from a standard Drupal installation and do not reflect the fields Islandora adds for 'Repository Item'. Repository managers need to add the fields necessary for their Islandora instance.
To add a field, click the + Add fields button. A shadow-box will appear with a list of the fields available for the index.
Some fields, such as the Body ('body') field provided by Drupal, have multiple properties which can be completely different values or variations on the same value. Click on the plus-sign next to the field to show the properties available to index. In most cases repository managers can ignore the properties list and click the Add button by the field to index the default property ('value'). Only select a different field property if you understand how it will impact user searching. Entity reference fields, such as Tags ('field_tags'), allow you to select fields or their properties from the referenced entity for indexing, such as a referenced taxonomy term's name field.
Once the fields are added they can be configured further on the Fields tab, although the label, machine name, and type usually don't need to be changed. The 'Type' dropdown has several Full-text processing options available, which may be of interest. Each is described in the expandable Data Types box at the bottom of the page. The Boost setting allows repository managers to increase the weight of particular fields when calculating search relevancy.
"},{"location":"user-documentation/searching/#processors-tab","title":"Processors Tab","text":"The Processors tab allows repository managers to adjust how data and search queries are processed to adjust results. The defaults are acceptable in most cases.
"},{"location":"user-documentation/searching/#indexing-edtf-dates","title":"Indexing EDTF Dates","text":"EDTF date fields can only be successfully indexed directly as strings. However, there is a way to include a custom \"edtf year\" field containing the facets-friendly year (or year ranges) from one or more EDTF fields. This must first be enabled by checking \"EDTF Year\" under the Processors tab, then can be added on the Field tab (it'll be called \"EDTF Creation Date Year (edtf_year)\"), and finally can be configured as a facet. See \"How should this be tested\" on this pull request for instructions on setting it up.
"},{"location":"user-documentation/searching/#searching-islandora","title":"Searching Islandora","text":"Searching using Search API in Drupal is done using Drupal Views. The Islandora Starter Site comes with a search page pre-configured (accessible at '/solr-search/content'). To edit the search page, navigate to '/admin/structure/views/view/solr_search_content'.
Repository managers may want to change the URL used to access the page, add it to the site navigation, or add a search box. In the 'Page Settings' box in the middle of the page, click on the existing path to open a shadow-box with an edit field. Change the URL as desired, for example, to 'search' and click Apply. Then, click the No menu link just below it to open the menu settings shadow-box. Selecting 'Normal menu entry' will allow a repository manager to add a menu link text, description, and place it within the site menu tree (the default, <Main navigation> works for most sites). A search box can be added by expanding the Advanced options and changing the Exposed form in block setting and then use the Block Layout interface (found at '/admin/structure/block') to place the block where desired. After making changes to the View's settings, click the Save button to ensure the changes are not lost.
Islandora's Repository Items are displayed in the search results as a fully rendered entity by default. Repository managers can choose which view mode should be used for each search datasource by clicking the Settings link next to the Show: setting under the Format section of the search view configuration page (shown in a red box in the screenshot below). The Teaser and Search result highlighting input are the two most likely options. Alternatively, repository managers can select specific fields to display instead by clicking the Rendered Entity link and changing it to Fields and then choosing which fields will be displayed in the Fields section underneath.
Thumbnails
thumbnails will not immediately be available using the Fields display option without more advanced configurations.
"},{"location":"user-documentation/starter-site-metadata-configuration/","title":"Islandora Starter Site Metadata Configuration","text":""},{"location":"user-documentation/starter-site-metadata-configuration/#introduction","title":"Introduction","text":"As described in Metadata In Islandora, in Islandora metadata is stored in Drupal, in fields attached to entities. Provided by the Islandora Starter Site, the \u201cRepository Item\u201d content type contains a set of default fields to describe the digital objects an Islandora repository might contain. This set of fields is based on MODS fields commonly used in Islandora 7, and is not intended to be \"the standard\" metadata profile, rather, a starting point for institutions designing their own repository.
This page presents the primary descriptive and administrative metadata fields found in the Islandora Starter site. We will define each field and give its basic configuration. Most of these configurations can be customized after installation, and new fields can be added as per the needs of an individual institution. Fields are grouped in standard MODS order.
Further information on the metadata configuration can be found in a Google Spreadsheet Islandora Starter Site Metadata Configuration (Google Sheets) which parallels the information on this page. It goes into further detail about the configurations connected to each field, provides information on taxonomies and mappings, and can be filtered or sorted in a variety of ways. The document, or the spreadsheet above, can be copied and used as a basis for planning your own configuration customizations as you work on your Islandora site.
"},{"location":"user-documentation/starter-site-metadata-configuration/#administrativesystem-fields","title":"Administrative/System Fields","text":"Title The name given to the resource. Title is a system field and as such its configurations can not be adjusted. It is the only field that is required by the system. Machine Name title Drupal Field Type Text (plain) Required yes Maximum Length 255 (character length is set by system but can be changed with a contrib module, see below) Repeatable no CSL Citation Mapping title RDF Mapping dcterms:title XPath MODS mods/titleInfo/title Transformation To create a single string out of the subelements of <titleInfo>, we suggest to use the <titleInfo> section of the LOC MODS-DC transform https://www.loc.gov/standards/mods/v3/MODS3-5_DC_XSLT1-0.xsl. In words, it says to:It is worth noting that the subtitle, partNumber, and partName elements are technically repeatable, but this transform only uses the first value encountered. If you have repeating subelements, information will be lost. Drupal Module Integration The Title Length module allows you to extend the length of the title to 500 characters (possibly more). The Views Natural Sort module allows you to sort a view by titles while skipping a configurable list of non-sorting characters (\"A\", \"The\", \"L'\", etc.). It's not as precise as nonSort, but does most of the job. Alternatives Paragraphs are a way to model a multi-part title (nonSort, title, subtitle, partName, partNumber, etc). Combine paragraphs with (automatic entity label? automatic nodetitles?) to not have to enter your title information twice. (Who to contact with experience about this?) Member of This item's parent item in Islandora. Usually this will be a collection, book (\"Paged Content\"),or compound object. Machine Name field_member_of Drupal Field Type Entity reference Required no (will default to empty if no value is entered) Repeatable yes Create Referenced Entities no (can only be connected to existing entities) Facet Member Of Alternatives You could arrange your content with Entity Reference Hierarchy, which is a very scalable way of representing large hierarchy trees. However, you'd have to re-work a number of hard-coded elements in the islandora module. Model The internal-to-Islandora category of the resource. Affects how the item is displayed/viewed.
This field is actionable by Islandora and was designed to trigger derivatives and view modes through the use of Contexts. It is a controlled list and new values should only be added when required (i.e. new behaviors are made available and these values are created to trigger them). Machine Name field_model Drupal Field Type Entity reference Required yes Repeatable no Taxonomies Islandora Models (pre-populated/controlled)
This field does not allow creating names of entities on the fly. First create a term in either the person, family, or corporate body taxonomies, then link it here.
Field name changed from \"Linked Agent\" 09/23. Machine Name field_linked_agent Drupal Field Type Typed Relation [link to https://islandora.github.io/documentation/user-documentation/metadata/#typed-relation] Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Person Create Referenced Entities no Relators Starter Site includes 269 terms from the MARC Relators list. Further terms from the list, or custom terms, can be added in the configuration for this field. Publisher has been removed from this list to encourage use of the Publisher field. CSL Citation Mapping author ; contributor ; editor Facet Creators and Contributors RDF Mapping dcterms:contributor XPath For MODS mods/name/namePart + role/roleTerm > relator
mods/name[@type=\"personal\"]/namePart > \"Person\" taxonomy
mods/name[@type=\"corporate\"]/namePart > \"Corporate Body\" taxonomy
mods/name[@type=\"family\"]/namePart > \"Family\" taxonomy Alternatives Create custom facets using the \"Typed Relation filtered by type\" Search API processor. This would allow you to separate out, for instance, creator|author|photographer from the other types of relators as a Search API field/facet."},{"location":"user-documentation/starter-site-metadata-configuration/#type-and-genre","title":"Type and Genre","text":"Type The general nature or genre of the content of the resource. To describe the digital or physical format of the resource, use Form instead. In contrast with Model, which is system-actionable, this field is designed to record the Type purely as a metadata element. Machine Name field_resource_type Drupal Field Type Entity reference [link to https://islandora.github.io/documentation/user-documentation/metadata/#entity-reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable no Taxonomies Resource Types (pre-populated with hierarchical LOC Resource Type Scheme terms) Create Referenced Entities no Facet Resource Type RDF Mapping dcterms:type XPath For MODS mods/typeOfResource, mods/typeOfResource[@collection=yes] Alternatives You may be required to use the Dublin Core types (for compatibility/harvesting); in that case you can replace the current Type vocabulary contents with the desired values. Genre A term or terms that designate a category characterizing a particular style, form, or content, such as artistic, musical, literary composition, etc. Machine Name field_genre Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Genre Create Referenced Entities yes CSL Citation Mapping genre RDF Mapping dcterms:type XPath For MODS mods/genre; mods/subject/genre"},{"location":"user-documentation/starter-site-metadata-configuration/#origin-information","title":"Origin Information","text":"Place Published Plain text field to describe the place of publication in full or transcribed from an item \\ \\ See also the entity reference field Place Published Country. Machine Name field_place_published Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes CSL Citation Mapping publisher-place RDF Mapping relators:pup XPath For MODS mods/originInfo/place/placeTerm [@type = 'text'] OR [not(@type)] Country of Publication Entity reference field to singularly describe the country (or jurisdiction) of publication. Connected to the unpopulated Country taxonomy by default, which can be used for terms or codes from MARC Code List for Countries or elsewhere. Disabled, by default, on Metadata Display and Metadata Form. Machine Name field_place_published_country Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable no Taxonomies Country Create Referenced Entities no RDF Mapping relators:pup XPath For MODS mods/originInfo/place/placeTerm[@type=\"code\"] Publisher New field added 04/2024. Previously the Contributors/Linked agent field (with a relator value of Publisher) was used for this content. Machine Name field_publisher Drupal Field Type Text (plain) Required no Maximum Length 500 Repeatable yes CSL Citation Mapping publisher RDF Mapping XPath For MODS mods/originInfo/publisher"},{"location":"user-documentation/starter-site-metadata-configuration/#date-fields","title":"Date FieldsDate IssuedDate CreatedDate (Unspecified)Copyright DateDate ValidDate CapturedDate ModifiedEdition StatementMode of IssuanceFrequency","text":"
All date fields use the Drupal EDTF field type and must follow EDTF formatting. For more information on this format and how it works with Islandora see https://islandora.github.io/documentation/user-documentation/metadata/#edtf
Date of formal issuance of the resource. This includes publication dates. Machine Name field_edtf_date_issued Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes CSL Citation Mapping issued Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:issued XPath For MODS mods/originInfo/dateIssued Date of creation of the resource Machine Name field_edtf_date_created Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:created XPath For MODS mods/originInfo/dateCreated A date without a type or relationship to the resource specified Machine Name field_edtf_date Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:date XPath For MODS mods/originInfo/dateOther Date of copyright of the resource. Machine Name field_copyright_date Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes Facet Year (string converted to a simple year value for field_edtf_year and then used for Year facet. A range is converted to multiple years.) RDF Mapping dcterms:dateCopyrighted XPath For MODS mods/originInfo/copyrightDate Date (often a range) of validity of a resource. Machine Name field_date_valid Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping dcterms:valid XPath For MODS mods/originInfo/dateValid The date on which the resource was digitized or a subsequent snapshot was taken. Machine Name field_date_captured Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping premis:creation XPath For MODS mods/originInfo/dateCaptured Date on which the original resource being represented in Islandora was changed. Typically modification dates of digital representations of the resource stored in Islandora will be recorded on the relevant Media instead of here. This field is not populated automatically by any Drupal functionality. Machine Name field_date_modified Drupal Field Type EDTF Required no Maximum Length 128 Repeatable yes RDF Mapping dcterms:modified XPath For MODS mods/originInfo/dateModified Information identifying the version of the resource. Machine Name field_edition Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping rdau:P60329 XPath For MODS mods/originInfo/edition A term that designates how the resource is issued. MODS standards limit the values of this field to monographic, single unit, multipart monograph, continuing, serial, and integrating resource, but there are no such limitations in Islandora and the \u201cIssuance Mode\u201d vocabulary is empty on initial installation. Machine Name field_mode_of_issuance Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Issuance Mode Create Referenced Entities yes RDF Mapping rdau:P60051 XPath For MODS mods/originInfo/issuance The publication frequency, in textual form. Can be based on MARC Frequency terms. Machine Name field_frequency Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Frequency Create Referenced Entities yes RDF Mapping rdau:P60538 XPath For MODS mods/originInfo/frequency"},{"location":"user-documentation/starter-site-metadata-configuration/#language","title":"Language","text":"Language Language of the resource content. Can be based on a languages vocabulary such as ISO 639-2.MARC Code List for Languages.This field is not connected to Drupal system language fields or related to Drupal translation functionality. Machine Name field_language Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Language Create Referenced Entities yes CSL Citation Mapping language RDF Mapping dcterms:language XPath For MODS mods/language/languageTerm[@type = 'text'] or [@type = 'code'] mapped to text per iso639-2b"},{"location":"user-documentation/starter-site-metadata-configuration/#physical-description","title":"Physical Description","text":"Form The physical format of the original resource being described. If the resource is a physical object, the physical form is recorded here. If the resource is born-digital, the original digital form is recorded here. Details of the formats of resource representations stored in Islandora should be recorded on the relevant Media files. Machine Name field_physical_form Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Physical Form Create Referenced Entities yes Facet Physical Form RDF Mapping dcterms:format XPath For MODS mods/physicalDescription/form Extent The size or duration of the resource in its original form. Extent of representations of the resource stored in Islandora should be recorded on the relevant Media files. Machine Name field_extent Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:extent XPath For MODS mods/physicalDescription/extent"},{"location":"user-documentation/starter-site-metadata-configuration/#descriptions","title":"Description(s)","text":"Description Description of the resource. Description may include but is not limited to: an abstract, a table of contents (alternately its own field), a graphical representation, or a free-text account of the resource. Machine Name field_description Drupal Field Type Text (plain, long) Required no Repeatable no CSL Citation Mapping abstract RDF Mapping dcterms:description XPath For MODS mods/abstract (xpath could also map to Abstract; do not map same values to both) Abstract Supplementary to the Description field, for including descriptions that more fit the definition of Abstract as it refers to ETDs, journal articles, or other scholarly publications. Machine Name field_abstract Drupal Field Type Text (formatted, long) Required no Repeatable no RDF Mapping dcterms:abstract XPath For MODS mods/abstract (xpath could also map to Description; do not map same values to both) Table of Contents Supplemental to Description, for breaking out specific Table of Contents information separate from a summary description of the object. Allows formatting. Machine Name field_table_of_contents Drupal Field Type Text (formatted, long) Required no Repeatable no RDF Mapping dcterms:tableOfContents XPath For MODS mods/tableOfContents Note General textual information relating to a resource, not described in other fields. Machine Name field_note Drupal Field Type Text (formatted, long) Required no Repeatable yes RDF Mapping skos:note XPath For MODS mods/note (with [@displayLabel] or [@type] prepended with ': ' to the note)"},{"location":"user-documentation/starter-site-metadata-configuration/#subjects","title":"Subject(s)","text":"Subject General subjects which may be constructed from topical, geographic, temporal, and genre subdivisions. If you wish to manage these types of subjects separately, use the more specific Subject fields below.
This field can be used if you are storing composed subject strings rather than component headings/subdivisions in their own fields. Machine Name field_subject_general Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Geographic Location; Person; Subject Create Referenced Entities yes Store New Items In Subject Facet Subject RDF Mapping dcterms:subject XPath For MODS mods/subject Subject (Topical) For use when using separate fields to record topical, geographic, name, and temporal terms. Topical subject terms would be recorded here. Machine Name field_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Subject Create Referenced Entities yes RDF Mapping dcterms:subject XPath For MODS mods/subject/topic Subject (Geographic) For use when using separate fields to record topical, geographic, name, and temporal terms. Geographic subject terms would be recorded here. Machine Name field_geographic_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Geographic Location Create Referenced Entities yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/geographic Subject (Name) For use when using separate fields to record topical, geographic, name, and temporal terms. Names of individuals, organizations, or families would be recorded here. Machine Name field_subjects_name Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Corporate Body; Family; Person Create Referenced Entities yes Store New Items In Person Facet Subject (name) RDF Mapping dcterms:subject XPath For MODS mods/subject/name/namePart Subject (Temporal) For use when using separate fields to record topical, geographic, name, and temporal terms. Temporal subject terms would be recorded here. Machine Name field_temporal_subject Drupal Field Type Entity reference Required no Maximum Length 255 (system limit for Taxonomy terms) Repeatable yes Taxonomies Temporal Subjects Create Referenced Entities yes CSL Citation Mapping Facet Temporal Subject RDF Mapping dcterms:temporal XPath For MODS mods/subject/temporal"},{"location":"user-documentation/starter-site-metadata-configuration/#coordinates","title":"Coordinates","text":"Coordinates Must be formatted either in decimal (51.47879) or sexagesimal format (51\u00b0 28' 43.644\") Machine Name field_coordinates Drupal Field Type Geolocation Required no Repeatable yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/cartographics/coordinates Coordinates (Text) A plain text field meant as an alternative to the Coordinates field, for values that do not meet the requirements of the Drupal geolocation field type. Machine Name field_coordinates_text Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:spatial XPath For MODS mods/subject/cartographics/coordinates"},{"location":"user-documentation/starter-site-metadata-configuration/#classification","title":"Classification","text":"Dewey Classification Classification based on the Dewey Decimal system. Machine Name field_dewey_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification[@authority=\"ddc\"] Library of Congress Classification Classification based on the Library of Congress Classification system. Machine Name field_lcc_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification[@authority=\"lcc\"] Classification (Other) Classification based on any system that does not have a dedicated field. Machine Name field_classification Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dc11:subject XPath For MODS mods/classification"},{"location":"user-documentation/starter-site-metadata-configuration/#identifiers","title":"Identifier(s)","text":"Identifier A unique standard number or code that distinctively identifies a resource. Machine Name field_identifier Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:identifier XPath For MODS mods/identifier[not(@type)] ISBN An International Standard Book Number (ISBN) identifier. Machine Name field_isbn Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dbpedia:isbn XPath For MODS mods/identifier[@type=\"isbn\"] OCLC Number An identifier assigned by an OCLC system, such as WorldCat. Machine Name field_oclc_number Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dbpedia:oclc XPath For MODS mods/identifier[@type=\"oclc\"] Local Identifier Identifier from any locally managed system. Machine Name field_local_identifier Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable yes RDF Mapping dcterms:identifier XPath For MODS mods/identifier[@type=\"local\"] PID PID of this object in Islandora 7.x (applies to migrated objects only)
Disabled, by default, in Metadata Display. Machine Name field_pid Drupal Field Type Text (plain) Required no Maximum Length 255 Repeatable no"},{"location":"user-documentation/starter-site-metadata-configuration/#access-conditions","title":"Access Conditions","text":"Rights Free text field for information about restrictions imposed on, and/or rights to, access to a resource, particularly a digital resource. It is not connected to any access control functionality within Drupal. Machine Name field_rights Drupal Field Type Text (formatted, long) Required no Repeatable yes RDF Mapping dc11:rights XPath For MODS mods/accessCondition"},{"location":"user-documentation/transcripts/","title":"Transcripts","text":"
WEBVTT transcripts, captions, or subtitles may be displayed along with audio and video media. Transcripts must be added to the media entity that will be playing, using a special field type (\"Media track\"). When the media is configured to use the \"Audio with Captions\" and \"Video with Captions\" field formatters, tracks in \"Media track\" fields will be available.
"},{"location":"user-documentation/transcripts/#media-track-field-type","title":"Media track field type","text":"The Islandora module provides a field type, Media Track, that accepts .vtt files. Once a file has been uploaded, additional options relevant to media tracks become available to configure.
"},{"location":"user-documentation/transcripts/#using-transcripts-with-the-islandora-starter-site","title":"Using Transcripts with the Islandora Starter Site","text":"In the Islandora Starter Site, Audio and Video media are pre-configured to work with tracks.
Sandbox
On the public sandbox, or other sites using the Islandora Install Profile Demo, you will first need to make the \"Track\" field visible in the media form, at Structure > Media > Audio|Video > Manage Display.
"},{"location":"user-documentation/transcripts/#video","title":"Video","text":"Selecting the correct media
Note that if you add the transcript file to the Original File, but you have configured the site to play the Service File, then you will not see the transcript.
Languages
While you can add subtitles in different languages, you may only choose from the site's installed languages.
Types
The five options: captions; subtitles; descriptions; chapters; and metadata come from the HTML standard's <track> element. As per their definitions, captions and subtitles will be displayed as optional text over the video, available through the usual [cc] icon in the viewer controls. Descriptions, chapters, and metadata will not be displayed as they are intended for programmatic use.
If you want to upload large (> 1GB) files, you'll need to tune the following settings in your PHP config (the following assumes Apache and a config file php.ini):
upload_max_filesize \u2013 The maximum allowed upload file size.post_max_size \u2013 The maximum allowed POST data size.max_input_time \u2013 Maximum allowed input time.max_execution_time \u2013 The maximum allowed time the scripts are allowed to run.default_socket_timeout - Default timeout (in seconds) for socket based streams.However, large file transfer over HTTP still has a host of issues once you properly configure your server. Uploads are not resumable and subject to connectivity issues. If you really want to upload large files, you should consider some alternatives such as
If loading large (e.g. range 30-45 GB) files into Fedora, you may need to change the fcrepo.session.timeout property, which defaults to 3 minutes (180,000 ms). Documentation is on the Properties page on the Fedora wiki.
If using FITS, you may need to change the following in /var/lib/tomcat9/webapps/fits/WEB-INF/classes/fits-service.properties:
# Maximum allowable size of uploaded file\nmax.upload.file.size.MB=2000\n# Maximum size of HTTP Request object. Must be equal to or larger than the value for max.upload.file.size.MB\nmax.request.size.MB=2000\n# Maximum size of an uploaded file kept in memory. Otherwise temporarily persisted to disk.\nmax.in.memory.file.size.MB=4\nA URL alias is an alternate URL pattern that resolves to a Drupal entity, such as a node, media, taxonomy term, or user. For example, you could set \"/welcome\" as an alias for \"/node/1\". Aliases are part of Drupal Core and can be extended by contrib modules. One such module is Pathauto, which enables automatic alias generation based on patterns, and the patterns may involve \"Tokens\" (such as [node:title]).
This page will attempt to cover the Islandora Starter Site's use of aliases and what we consider to be best practices. A full description of creating and managing aliases is out of scope.
"},{"location":"user-documentation/url-aliases/#best-practices-with-url-aliases","title":"Best practices with URL aliases","text":"While every site may choose to set up their aliases differently, we cannot prescibe a universal setup.
A common \"nice-to-have\" is the presence of the slug /islandora/ in the URL which identifies the content as \"Islandora\".
A potential \"best practice\" is that if your site uses persistent identifiers such as DOIs or Handles, that those identifiers make up part of the URL alias.
"},{"location":"user-documentation/url-aliases/#use-of-url-aliases-in-islandora-starter-site","title":"Use of URL aliases in Islandora Starter Site","text":"The Islandora Starter Site includes the Pathauto module, which we consider that most sites will want to use in some way. However its default configuration should not be interpreted as prescriptive. You are encouraged to use persistent identifiers if you have them!
The default Pathauto pattern for Repository Items is /islandora/[node:title] with the pathauto configuration trimming the alias at 100 characters.
Sites migrating from Islandora Legacy may wish for their objects to still be available through their old URLs, with the pattern /islandora/object/[PID].
Options for doing this include:
field_pid with the legacy PID, and using Pathauto to create URL aliases of the pattern /islandora/object/[node:field_pid]. However, you will need to set up something for new objects that don't have Legacy PIDs.field_pid. Drupal 8 supports various web analytics integrations such as Google Analytics and Matomo. For privacy reasons, Islandora integrates Matomo web analytics platforms by default.
The Matomo server is installed here: http://localhost:8000/matomo. The default configurations can be found in http://localhost:8000/admin/config/system/matomo. You can log in to the Matomo server using username admin and password islandora. The dashboard will look like this:
To see page views, login to Matomo and go to Behaviour >> Pages.
"},{"location":"user-documentation/usage-stats/#further-reading","title":"Further Reading","text":"This Using Islandora section is aimed at site administrators and repository managers who need to understand and configure their Islandora repositories. It will go in-depth on how Islandora allows you to use the various features of Drupal to construct and display repository items. For easy readability, we have divided this documentation into five major sections, organized by functionality.
Islandora, like Drupal, provides tools to create a site, but does not force you to conform to any specific site structure, organization, or navigation. There is a hope that we can provide something useful out of the box, while also allowing the full suite of Drupal configuration options. This out-of-the-box configuration is the Islandora Demo module.
As you read this documentation, it is recommended to be familiar with the basics of Drupal, including content types, fields, users, and views. The Official Drupal User Guide and the Community Guide to Drupal are a good place to start.
"},{"location":"user-documentation/user-intro/#modeling-content","title":"Modeling content","text":"In Islandora, we often want to store (and manage, and preserve) sets of metadata with binaries (digital files). Previously in Islandora Legacy, these components were referenced together as a single \"object.\" In Islandora, metadata and their binaries are now represented as multiple interconnected entities.
Metadata is stored in nodes (a.k.a. content).
Binary files are stored in are media, which are wrapper entities to help manage files.
Metadata values can be stored as taxonomy terms, which let you reuse the same value in multiple places.
For users familiar with Islandora Legacy, the relationship between Fedora and Islandora has greatly changed in the current version of Islandora.
Islandora Legacy inherited its object model from Fedora 3.x. In Legacy, Fedora stored all properties and content associated with an object - not only its owner, dc.title, status, PID, and status - but also any content files such as OBJ, DC, MODS, and RELS-EXT. In Islandora Legacy, Fedora acted as the authoritative, primary source for all aspects of an object. Fedora 3.x was not an optional component of an Islandora Legacy repository, instead it served as the primary datastore.
In Islandora, using Fedora is optional. That's right, optional. Drupal, and not Fedora, is the primary source of all aspects of an Islandora object, and, with some variations, Drupal, not Fedora, is the primary datastore in an Islandora repository. If Fedora is present in an Islandora repository, content in it is a tightly synchronized copy of object properties and files managed by Drupal.
Even though Fedora is optional in Islandora, most repositories will use it since it provides its own set of services that are worth taking advantage of, such as:
In Islandora repositories that use Fedora, all properties about Drupal nodes are mirrored in Fedora as RDF properties. But, even if an Islandora instance does not use Fedora, Drupal can provide an object's properties as RDF (again, Drupal is the primary source of data in Islandora). In addition, the Drupal media associated with Islandora objects are persisted to Fedora, although exactly which media is configurable within the Islandora admin interface. Just as Drupal out of the box has a public and private filesystem, Islandora adds a third filesystem to Drupal called, not surprisingly, \"fedora\", and it is to this filesystem that media are persisted. We will provide more information about Fedora's role in an Islandora repository in the metadata and media sections.
"},{"location":"user-documentation/user-intro/#architecture","title":"Architecture","text":""},{"location":"user-documentation/user-intro/#conceptual-diagram","title":"Conceptual diagram","text":"Many users of Islandora may be familiar with the metaphorical diagram of Islandora Legacy as a cheeseburger, which provides a memorable approximation of how the different parts of the software stack interact in a vertically-integrated, relatively customizable fashion (ie, Drupal, Solr, Islandora, and Fedora are stable layers, and the \"toppings\" stand in for Solution Packs and other utilities that can be added or removed to customize Islandora):
For a similar conceptual approach to Islandora, we present it as a bento box: a very modular platform, in which each piece may be removed and replaced with something different, without disrupting other parts of the stack:
For a true diagram of how the various parts of Islandora interact, please see the full Architecture Diagram.
"},{"location":"user-documentation/users/","title":"Managing Users","text":""},{"location":"user-documentation/users/#overview","title":"Overview","text":"Anyone who visits your Drupal site is a user. There are three different default users in Drupal:
Islandora adds an additional default role:
Additional user roles can be created and assigned customized permissions, as described below.
"},{"location":"user-documentation/users/#before-you-start","title":"Before you start","text":"Warning
If you are writing to Fedora, your username must not contain spaces.
"},{"location":"user-documentation/users/#how-to-add-a-user","title":"How to Add a User","text":"To review/edit the permission for each role, in the People page click the Permissions tab in the set of tabs above the Add user button.
"},{"location":"user-documentation/users/#how-to-create-a-new-user-role","title":"How to Create a New User Role","text":"Islandora Quick Lessons
Learn more with this video on how to Add a User.
"},{"location":"user-documentation/users/#video-walk-through-users-roles-and-permissions","title":"Video Walk-through: Users, Roles, and Permissions","text":"Click the image below to open the Users, Roles, and Permissions video tutorial on the Islandora YouTube channel.
See more videos from the Drupal 101 series here.
"},{"location":"user-documentation/users/#further-reading","title":"Further Reading","text":"For more information on managing users in Drupal visit the section Managing User Accounts of Drupal.org.
"},{"location":"user-documentation/versioning/","title":"Versioning","text":"As a user of an Islandora repository, you may be wondering - Is this content being versioned? Could I restore from a previous version if I needed to? Can I see a list of versions for an object? The answer to these questions is two-fold, and largely yes. The architecture of Islandora provides users with a Drupal implementation and a Fedora implementation which are connected in Islandora.
Islandora Software Versioning
Looking for information about versions of Islandora itself? The latest Islandora follows semantic versioning. Previously, Islandora's versions were tied to the version of Drupal and numbered in order of release, such as Islandora 6.x-13.1 or Islandora 7.x-1.13. More information.
"},{"location":"user-documentation/versioning/#drupal-revisioning","title":"Drupal Revisioning","text":"Drupal provides a concept of revisions which allows you to track the differences between multiple versions of your content and revert to older ones. The list of revisions for a node are available at http://localhost:8000/node/1/revisions. There are Drupal docs on revisioning. Media objects are also versioned in Drupal but there is not a UI component for this - see related issue.
"},{"location":"user-documentation/versioning/#fedora-and-memento","title":"Fedora and Memento","text":"Fedora implements the Memento specification for versioning resources, which is a time-based HTTP framework. Fedora provides documentation as well as an API implementation.
"},{"location":"user-documentation/versioning/#basic-data-flow","title":"Basic Data Flow","text":"Islandora Quick Lessons are a series of short videos demonstrating how to do common tasks in Islandora.
New videos are added to the playlist regularly.
"},{"location":"user-documentation/video-docs/#general-information","title":"General Information","text":"The following recipe details how to connect Islandora with Alexa, using custom Alexa skills and the Drupal Alexa module. The potential applications are broad: - Send Citations, metadata, whatever we want to the user if they have set up their email - Creating a collaborative exhibit - Play audio and video objects and read transcripts - Respond to user search queries (like how many objects match the subject in the repositories) - Answer specific questions about the object (\u201cInvocation Name, when was this recorded?\u201d) - Interact with other applications or modules (got a print ordering system? Want to add event calendar items to your exhibit?) - Be accessed via web page, Alexa device, or phone app
Note
This recipe has not been extensively tested.
"},{"location":"user-documentation/recipes/alexa-search/#ingredients","title":"Ingredients","text":"composer require \"drupal/alexa\"