Skip to content

Latest commit

 

History

History
537 lines (383 loc) · 17.9 KB

README.template.md

File metadata and controls

537 lines (383 loc) · 17.9 KB

SITE_NAME

LICENSE

Introduction

This is the development and production infrastructure for INSTITUTION's SITE_NAME.

Requirements

Docker Compose

There are a number of docker-compose.yml files provided by this repository:

File Description
docker-compose.yml Defines all development & production services.
docker-compose.darwin.yml MacOS platform specific customizations to allow access to host SSH_AGENT. For development use only.
docker-compose.linux.yml Linux platform specific customizations to allow access to host SSH_AGENT. For development use only.
docker-compose.override.yml Customizations for local development environment.
docker-compose.registry.yml Used for creating a local registry for testing multi-arch builds, etc. Can typically be ignored.

Override

This git repository does not track docker-compose.override.yml which will be included in all docker compose commands you invoke.

Two platform dependent templates that allow for access to the hosts SSH agent are provided for development environments.

Simply copy the appropriate* docker-compose.PLATFORM.yml file into docker-compose.override.yml, on your development machine.

Any additional changes that are for your local / development environment can then be added to docker-compose.override.yml. Because they're not tracked...

Unfortunately getting OpenSSH working is complicated on Windows machines, so in that cause it is left as an exercise to the reader.

Building

You can build your development environment locally using docker compose:

docker compose --profile dev build --pull

This builds the Drupal image so that it contains the Starter Site files that you downloaded locally earlier.

Pulling Docker Images

The Docker Compose file provided requires that you pull the non-buildable islandora images with the following command:

docker compose --profile dev pull --ignore-buildable --ignore-pull-failures

Running / Stoping / Destroying

You must specify a profile - either dev or prod - to use the docker compose files provided by this repository. Running/stopping/destroying for each profile is outlined below.

Development Profile

Use the dev profile when bring up your local/development environment:

docker compose --profile dev up -d

After all containers are "Started", you must wait several minutes for the Islandora site to install. When completed, you can see the following in the output from the drupal-dev container, with the following command:

docker compose logs -f drupal-dev
#####################
# Install Completed #
#####################

For all accounts in the development profile the username and password are set to the following:

Credentials Value
Username admin
Password password

If you have the domain in your .env set to islandora.dev (default), you can access all the services at the following URLs.

Service URL
Drupal https://islandora.dev
IDE https://ide.islandora.dev
ActiveMQ https://activemq.islandora.dev
Blazegraph https://blazegraph.islandora.dev/bigdata/
Cantaloupe https://islandora.dev/cantaloupe
Fedora https://fcrepo.islandora.dev/fcrepo/rest/
Solr https://solr.islandora.dev
Traefik https://traefik.islandora.dev

To stop your local/development environment:

docker compose --profile dev down

To destroy all data from your local/development environment:

docker compose --profile dev down -v

Production Profile

Use the prod profile when bring up your production environment:

docker compose --profile prod up -d

To stop your production environment:

docker compose --profile prod down

N.B. You shouldn't really ever run the following on your production server. This is just when testing the differences for production environment on your local machine.

To destroy all data from your production environment:

docker compose --profile prod down -v

Pushing Docker Images

Pushing requires setting up either a Local Registry, or a Remote Registry. Though you may want to use both concurrently.

Additionally the command to build & push changes if you need multi-platform support, i.e. if you need to be able to run on ARM (Apple M1, etc) as well as x86 (Intel / AMD) CPUs.

Local Registry

To test multi-platform builds locally requires setting up a local registry.

This can be done with the assistance of isle-builder repository.

N.B. Alternatively you can push directly to a remote registry like DockerHub, though that can be slow as it needs to upload your image over the network.

Now you can perform the build locally by pushing to the local registry:

REPOSITORY=islandora.io docker buildx bake --pull --builder isle-builder --push

N.B. If you do not override REPOSITORY environment variable, the value provided by .env is used, which will typically be the remote registry you intended to use.

Remote Registry

First you must choose a Docker image registry provider such as DockerHub.

Assuming your are logged into your remote repository, i.e. you've done docker login with the appropriate arguments and credentials for your chosen remote Docker image repository.

You must then replace the following line in .env to match the repository you have created with your chosen registry provider:

# The Docker image repository, to push/pull custom images from.
# islandora.io redirects to localhost.
REPOSITORY=islandora.io

If you do not need to build multi-platform images, you can then push to the remote repository using docker compose:

docker compose --profile dev push drupal-dev

If you do need produce multi-platform images, you'll need to setup a builder which is covered under the Local Registry section.

docker buildx bake --pull --builder isle-builder --push

N.B. In this example REPOSITORY is not overridden, so the value provided by .env is used.

Development

UID

Use the following bash snippet to generate the ./certs/UID file.

printf '%s' "$(id -u)" > ./certs/UID

This is used on container startup to make sure bind mounted files are owned by the same user as the host machine.

N.B. Alternatively this file is generated when you run generate-certs.sh

Development Certificates

If you have mkcert properly installed you can simply run generate-certs.sh to generate development certificates, otherwise follow the manual steps outlined below.

Before we can start a local instance of the site we must generate certificates for local development. This varies a bit across platforms, please refer to the mkcert documentation to ensure your setup is correct for your host platform, and you have the appropriate dependencies installed.

These certificates are only used for local development production is setup to use certificates automatically generated by lets-encrypt.

Create Certificate Authority

N.B. This only has to be done once per host, and is only required for local development. On a production server you should be using actual certificates which will be documented in a later section.

Windows

You must do this using cmd.exe as an Administrator as WSL does not have access to Windows trust store and is not able to install certificates.

  1. Generate and install rootCA files:
mkcert.exe -install

OSX and Linux

  1. Generate and install rootCA files:
mkcert -install

Copy Certificate Authority files

The previous step generate two rootCA files which you must copy into this repositories certs folder.

Windows

Using cmd.exe although no longer as an administrator.

  1. Determine the location of the rootCA files:
mkcert.exe -CAROOT
  1. Copy the certificates into the certs folder (from the root of your repository):
set CAROOT="VALUE FROM STEP #1"
copy %CAROOT%\rootCA-key.pem certs
copy %CAROOT%\rootCA.pem certs

N.B. Firefox does not work with these certificates on Windows. So you must use either Chrome or Edge on Windows.

OSX and Linux

cp $(mkcert -CAROOT)/* certs/

Create Development Certificates

Windows

Using cmd.exe although no longer as an administrator.

  1. Create site certificates (from the root of your repository):
mkcert.exe -cert-file certs\cert.pem -key-file certs\privkey.pem "*.islandora.dev" "islandora.dev" "*.islandora.io" "islandora.io" "*.islandora.info" "islandora.info" "localhost" "127.0.0.1" "::1"

OSX and Linux

  1. Create site certificates (from the root of your repository):
mkcert \
  -cert-file certs/cert.pem \
  -key-file certs/privkey.pem \
  "*.islandora.dev" \
  "islandora.dev" \
  "*.islandora.io" \
  "islandora.io" \
  "*.islandora.info" \
  "islandora.info" \
  "localhost" \
  "127.0.0.1" \
  "::1"

Upgrading Isle Docker Images

First, read the release notes for the versions between your current version and your target version, as manual steps beyond what is listed here will likely be required.

Edit .env and replace the following line with the desired new version tag:

# The version of the isle-buildkit images to use.
ISLANDORA_TAG=x.x.x

Then you can pull the latest images as described previously.

Of course make backups before deploying to production and test thoroughly.

Drupal Development

For local development via the development profile, an IDE is provided which can also support the use of PHPStorm.

There are a number of bind mounted directories and changes made in the following files & folders will persist in this Git repository.

  • /var/www/drupal/assets
  • /var/www/drupal/composer.json
  • /var/www/drupal/composer.lock
  • /var/www/drupal/config
  • /var/www/drupal/web/modules/custom
  • /var/www/drupal/web/themes/custom

Other changes such as to the vendor folder or installed modules are not persisted in Git. This is by design as these changes should be managed via composer and baked into the Drupal Docker image for production.

Changes made to composer.json and composer.lock will require you to rebuild the Drupal Docker image, see building for how.

N.B. None of the above directories are bind mounted in production as development in a production environment is not supported. The production site should be fairly locked down, and only permit changes to content and not configuration.

Pushing code changes

The IDE configured for local development receives its identity from the host system using ssh-agent. This is how you are able to git push from within the IDE container to a remote git origin (e.g. GitHub). Depending on whether the host is Mac OS or Linux the docker compose overrides can be seen in the respective override file on how the host identity is propagated into the IDE container.

Warning

If ssh-agent is not configured properly on your host, code changes made in the IDE may be lost

For linux host systems such as Debian or Ubuntu 24+ an environment variable SSH_AUTH_SOCK (set by ssh-agent) needs to be available in order to pass the host system's SSH identity into the IDE container. If this environment variable is not set (which can be checked on the host by seeing if echo $SSH_AUTH_SOCK prints output) it will need to be configured on your host system in order to allow the IDE to git push code changes to remote git origins. One possible solution to set this is to add this to your shell's login profile. e.g.

echo 'eval $(ssh-agent -s)' >> ~/.bashrc

After ensuring SSH_AUTH_SOCK is set properly, you will need to restart the codeserver service to pickup the changes. Then you should be able to push changes made in your IDE to a remote git origin.

Production

Running in production makes use of the production profile, which requires either manually provided secrets, or generating secrets. It also requires a properly configured DNS record as described in the following sections.

Generate secrets

To be able to run the production profile of the docker-compose.yml file the referenced secrets and JWT public/private key pair must be created. There are inline instructions for generating each secret in docker-compose.yml.

Alternatively you can use the generate-secrets.sh bash script to generate them all quickly.

N.B. The script will not overwrite existing secret files so it's safe to run repeatedly.

Production Domain

The .env has a variable DOMAIN which should be set to the production site's domain.

# The domain at which your production site is hosted.
DOMAIN=xxx.xxx

Automated Certificate Generation

Traefik has support for acme (Automatic Certificate Management Environment). This is what is used to generate certificates in a production environment.

This is configured to use a HTTP based challenge and requires that the following A Records be set in your production sites DNS Records. Where DOMAIN is replaced with the production sites domain.

  • ${DOMAIN}
  • fcrepo.${DOMAIN}

Each of the above values should be set to the IP address of your production server.

Additionally be sure to update the default email to that of your sites administrator:

# The email to use for admin users and Lets Encrypt.
[email protected]

N.B. This is required to property generate certificates automatically!

Setup as a systemd Service

Most Linux distributions use systemd for process management, on your production server you can use the following unit file with systemd.

N.B. Replace the User, Group, and WorkingDirectory lines as appropriate.

[Unit]
Description= Islandora
PartOf=docker.service
After=docker.service

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/opt/SITE_NAME
ExecStart=/usr/bin/docker compose --profile prod up
ExecStop=/usr/bin/docker compose --profile prod down

[Install]
WantedBy=multi-user.target

SELinux Considerations

If you using a system with SELinux enabled, you will need to set the appropriate labels on the generated secrets files for Docker to be allowed to mount them into containers.

sudo chcon -R -t container_file_t secrets/*

Troubleshooting

Windows

There are many services, which can listen port 80 on windows.

Luckily you can detect and stop them all running simple console command:

NET stop HTTP

Additionally you may wish to stop this service on startup, see the windows documentation for more details.