GitLab module for Puppet

Table of Contents

1. Overview
2. Module Description - What the module does and why it is useful
3. Setup - The basics of getting started with GitLab
   • What GitLab affects
   • Setup requirements
   • Beginning with GitLab
4. Usage - Configuration options and additional functionality
5. Reference - An under-the-hood peek at what the module is doing and how
6. Limitations - OS compatibility, etc.
7. Development - Guide for contributing to the module

Overview

This Puppet module installs and manages GitLab. It makes use of the provided Omnibus packages and the packagecloud package repositories.

Please note: The module vshn/gitlab has been deprecated and is now available under Vox Pupuli puppet/gitlab.

Module Description

The module installs the GitLab package from the provided repositories and creates the configuration file which is then used by gitlab-ctl reconfigure to configure all the services. Fun fact: This really uses Chef to configure all the services.

Supported are Debian based (Ubuntu, Debian) and RedHat based (CentOS, RHEL) operating systems.

Beaker acceptance tests are run in Travis for supported versions of CentOS and Ubuntu.

This module is designed to support the most recent versions of the gitlab-omnibus package (both ce and ee). GitLab will support and release patches for the last 3 releases. This module can typically support the most recent major version, as well as the previous major version, but is currently only tested in the gitlab-supported versions of the module.

If you find configurations or features in gitlab-omnibus that are not supported by this module, please open an issue or submit a pull request.

Current Support Status

gitlab-omnibus version	support of gitlab.rb configurations
11.x	Mostly implemented, supported configs are stable
10.x	All configs implemented and stable

For older versions of GitLab, you may find an older version of this module to work better for you, as this module changes over time to support the valid configuration of versions of the gitlab-omnibus supported by the gitlab engineering team. The oldest versions of this puppet module were designed to support gitlab-omnibus 7.10, and may be unstable even then.

Setup

What GitLab affects

• Package repository (APT or YUM)
• Omnibus gitlab package, typically gitlab-ce or gitlab-ee
• Configuration file /etc/gitlab/gitlab.rb
• System service gitlab-runsvdir
• GitLab configuration using gitlab-ctl reconfigure

Setup Requirements

Have a look at the official download page for the required prerequisits (f.e. Postfix). This module doesn't handle them, that's the job of the specific modules.

It requires only the puppetlabs/apt module when using it under a Debian based OS and the parameter manage_package_repo is not false. Furthermore the stdlib module is required.

At least on RedHat based OS versions, it's required that Puppet is configured with the stringify_facts setting set to false (Puppet < 4.0), otherwise the $::os fact used in install.pp doesn't work as expected.

Beginning with GitLab

Just include the class and specify at least external_url. If external_url is not specified it will default to the FQDN fact of the system.

class { 'gitlab':
  external_url => 'http://gitlab.mydomain.tld',
}

The module also supports Hiera, here comes an example:

gitlab::external_url: 'http://gitlab.mydomain.tld'
gitlab::gitlab_rails:
  time_zone: 'UTC'
  gitlab_email_enabled: false
  gitlab_default_theme: 4
  gitlab_email_display_name: 'GitLab'
gitlab::sidekiq:
  shutdown_timeout: 5

If one wants to install GitLab Enterprise Edition, just define the parameter manage_upstream_edition with the value ee:

class { 'gitlab':
  external_url => 'http://gitlab.mydomain.tld',
  manage_upstream_edition      => 'ee',
}

Note: This works only for GitLab version 7.11 and greater. See this blog entry: GitLab 7.11 released with Two-factor Authentication and a publicly viewable Enterprise Edition

Usage

The main class (init.pp) exposes the configuration sections from the gitlab.rb configuration file as hashes. So if there are any parameter changes in future versions of GitLab, the module should support them right out of the box. Only if there would be bigger changes to sections, the module would need some updates.

All possible parameters for gitlab.rb can be found here: gitlab.rb.template

Some examples:

class { 'gitlab':
  external_url => 'http://gitlab.mydomain.tld',
  gitlab_rails => {
    'webhook_timeout' => 10,
    'gitlab_default_theme' => 2,
  },
  logging      => {
    'svlogd_size' => '200 * 1024 * 1024',
  },
}

Service management

GitLab Omnibus is designed to manage it's own services internally. The gitlab-runsvdir service isn't a typical service that you would manage with puppet, it is a monitoring service for the other services gitlab will create based on your selected configuration. Starting, stopping and restarting the gitlab-runsvdir service should only be done by gitlab-ctl commands. Service restart is also handled implicitly during installation and upgrades, and does not normally need to be triggered by puppet.

If you find yourself needing to modify this behavior, you can set service_manage => true to have puppet ensure the service is running.

Setting service_provider_restart => true will cause puppet to trigger a gitlab-ctl restart command to be issued following any configuration change managed by puppet.

Package & Repository Configuration

Repository Resource Configuration

This module allows you a great range of options when configuring the repository and package sources on your host. By default, the gitlab repository will be configured to use the upstream source from packagecloud. However, if you wish to use a different repository source, you can provide your own yumrepo, apt or any other package/repository configuration you wish.

This module does this by iterating through configurations provided to gitlab::omnibus_package_repository::repository_configuration. You can provide any number of repository resource types and configurations you want, as long as the dependent modules are installed on your basemodulepath.

This approach provides the following advantages:

• means any and all parameters supported by your repository manager module are inherently supported by the gitlab module
• you aren't required to use a version of a dependency we specify, supporting a wide range of versions for modules like apt
• you can easily add more required repositories and packages as needed by your infrastructure, and ensure ordering is managed within the gitlab module before any GitLab related packages are installed

In order to provide your own repository configurations, you are required to set manage_upstream_edition => disabled, and provide a hash of repository resource type configurations in the following format:

gitlab::repository_configuration:
  repository_resource_type: #ex... 'apt::source` or `apt::pin` or `yumrepo`
    repository_resource_title:
      repository_resource_attribute1: 'value'
      repository_resource_attribute2: 'value'

Examples/defaults for yumrepo can be found at data/RedHat.yaml, and for apt at data/Debian.yaml.

You could also do things like:

• add an additional repository at the same level as internal_mirror_of_gitlab_official_ce (for example if you wanted to use your own package nginx instead of the one provided in omnibus-gitlab)
• add any other high level resource types from the apt module at the level of apt:source. (apt::pin, apt::key, etc...)

Each unique resource provided to the repository_configuration setup:

• gets tagged with gitlab_omnibus_package_resource
• gets the before => Class['gitlab::install'] metaparameter.

You can use these tags to further customize ordering within your own catalogs.

Selecting Version, edition, and package name

The package_ensure parameter is used to control which version of the package installed. It expects either a version string, or one of the ensure values for the Package resource type. Default is installed. This value works with the package_name parameter to install the correct package.

If you are using upstream package source, the package name automatically switches between gitlab-ce and gitlab-ee depending on the value you have provided to manage_upstream_edition. If manage_upstream_edition is set to disabled, you will need to provide the appropriate value to package_name yourself.

This approach of package management has the following advantages:

• more easily adaptable if GitLab changes package naming based on editions (won't require you to install new puppet-gitlab module if you're not ready)
• allows you to install custom built packages for gitlab-omnibus that have different package name on your host

Custom Repository & Package configuration example

As an expanded example of repository and package configuration, let's assume you're:

• using a private mirror of the upstream GitLab‚ package channel
• hosted inside your organizations firewall
• installing gitlab-omnibus enterprise edition

class { 'gitlab':
  external_url => 'http://gitlab.mydomain.tld',
  manage_upstream_edition => 'disabled',
  package_name => 'gitlab-ee',
  repository_configuration => {
    'apt::source' => {
      'internal_mirror_of_gitlab_official_ce' => {
        'comment' => 'Internal mirror of upstream GitLab package repository',
        'location' => 'https://my.internal.url/repository/packages.gitlab.com/gitlab/gitlab-ce/debian',
        'key' => {
          'id' => 'F6403F6544A38863DAA0B6E03F01618A51312F3F',
          'source' => 'https://my.internal.url/repository/package.gitlab.com/gpg.key'
        }
      },
    }
  }
}

GitLab secrets

Note: gitlab::secrets parameter was removed in v3.0.0. See: Issues#213 - Remove support for setting content of gitlab-secrets.json

When using HA role application_role, make sure to add the appropriate shared secrets to your gitlab_rails and gitlab_shell hashes to ensure front-end nodes are configured to access all backend data-sources and repositories. If you receive 500 errors on your HA setup, this is one of the primary causes.

LDAP configuration example

Here is an example how to configure LDAP using Hiera:

gitlab::gitlab_rails:
  ldap_enabled: true
  ldap_servers:
    myldapserver:
      label: 'Company LDAP'
      host: 'ldap.company.tld'
      port: 389
      uid: 'uid'
      method: 'plain' # "tls" or "ssl" or "plain"
      bind_dn: 'MYBINDDN'
      password: 'MYBINDPW'
      active_directory: false
      allow_username_or_email_login: false
      block_auto_created_users: false
      base: 'MYBASEDN'
      group_base: 'MYGROUPBASE'
      user_filter: ''

NGINX Configuration

Configuration of the embedded NGINX instance is handled by the /etc/gitlab/gitlab.rb file. Details on available configuration options are available at [https://docs.gitlab.com/omnibus/settings/nginx.html][NGINX settings]. Options listed there can be passed in to the nginx parameter as a hash. For example, to enable redirection from HTTP to HTTPS:

class { 'gitlab':
  external_url => 'https://gitlab.mydomain.tld',
  nginx        => {
    redirect_http_to_https => true,
  },
}

Similarly, the certificate and key location can be configured as follows:

class { 'gitlab':
  external_url => 'https://gitlab.mydomain.tld',
  nginx        => {
    ssl_certificate     => '/etc/gitlab/ssl/gitlab.example.com.crt',
    ssl_certificate_key => '/etc/gitlab/ssl/gitlab.example.com.key'
  },
}

Skip Auto Reconfigure (formerly Skip Auto Migrations)

In order to achieve Zero Downtime Upgrades of your GitLab instance, GitLab will need to skip the post-install step of the omnibus package that automatically calls gitlab-ctl reconfigure for you. In GitLab < 10.5, GitLab check for the presence of a file at /etc/gitlab/skip-auto-migrations. As of GitLab 10.6, this is deprecated, and you are warned to use /etc/gitlab/skip-auto-reconfigure going forward.

Both of these are currently supported in this module, and you should be aware of which option is right for you based on the version of GitLab Omnibus you are running. You will be presented with a deprecation notice in you puppet client if using the deprecated form.

# use 'absent' or 'present' for the skip_auto_reconfigure param
class { 'gitlab':
  skip_auto_reconfigure => 'present'
}

# use true/false for the skip_auto_migrations param
class { 'gitlab':
  skip_auto_migrations => true
}

GitLab Custom Hooks

Manage custom hook files within a GitLab project. Custom hooks can be created as a pre-receive, post-receive, or update hook. It's possible to create different custom hook types for the same project - one each for pre-receive, post-receive and update.

gitlab::custom_hook { 'my_custom_hook':
  namespace       => 'my_group',
  project         => 'my_project',
  type            => 'post-receive',
  source          => 'puppet:///modules/my_module/post-receive',
}

or via hiera

gitlab::custom_hooks:
  my_custom_hook:
    namespace: my_group
    project: my_project
    type: post-receive
    source: 'puppet:///modules/my_module/post-receive'

Since GitLab Shell 4.1.0 and GitLab 8.15 Chained hooks are supported. You can create global hooks which will run for each repository on your server. Global hooks can be created as a pre-receive, post-receive, or update hook.

gitlab::global_hook { 'my_custom_hook':
  type            => 'post-receive',
  source          => 'puppet:///modules/my_module/post-receive',
}

or via hiera

gitlab::global_hooks:
  my_custom_hook:
    type: post-receive
    source: 'puppet:///modules/my_module/post-receive'

Gitlab System Hooks

A file hook will run on each event so it's up to you to filter events or projects within a file hook code. You can have as many file hooks as you want. Each file hook will be triggered by GitLab asynchronously in case of an event. For a list of events see the system hooks documentation.

gitlab::system_hook { 'my_custom_hook':
  source          => 'puppet:///modules/my_module/file-hook',
}

or via hiera

gitlab::system_hooks:
  my_custom_hook:
    source: 'puppet:///modules/my_module/file-hook'

Fast Lookup of SSH keys

GitLab instances with a large number of users may notice slowdowns when making initial connections for ssh operations. GitLab has created a feature that allows authorized ssh keys to be stored in the db (instead of the authorized_keys file for the git user)

You can enable this feature in GitLab using the store_git_keys_in_db parameter, or by enabling gitlab-sshd as it is configured to use fast lookup automatically.

Please note, while you can manage gitlab-sshd (Gitlab's standalone SSH server) with this module, you can not manage openssh and the sshd service as it is outside the scope of the module. You will need to configure the AuthorizedKeysCommand for the git user in sshd.server yourself. Instructions for this are provided by GitLab at Fast lookup of authorized SSH keys in the databasse

Setting up GitLab HA

pgbouncer Authentication

For use in HA configurations, or when using postgres replication in a single-node setup, this module supports automated configuration of pgbouncer authentication. To set this up, set pgpass_file_ensure => 'present' and provide a valid value for pgbouncer_password.

class {'gitlab':
  pgpass_file_ensure => 'present',
  pgbouncer_password => 'YourPassword'
}

By default, this creates a file at /home/gitlab-consul/.pgpass, which gitlab uses to authenticate to the pgbouncer database as the gitlab-consul database user. This does not refer to the gitlab-consul system user. The location of the .pgpass file can be changed based on how you manage homedirs or based on your utilization of NFS. This location should be set to be the home directory you have configured for the gitlab-consul system user.

class {'gitlab':
  pgpass_file_location => '/homedir/for/gitlab-consul-system-user/.pgpass'
}

Tasks

The GitLab module has a task that allows a user to upgrade the pgsql database GitLab uses if upgrading from version 9.2.18, which is required to upgrade GitLab past 10. When running the tasks on the command line, you will need to use the --sudo, --run-as-root, and --tty flags to execute the commands as needed for your environment.

Please refer to to the PE documentation or Bolt documentation on how to execute a task.

Development

1. Fork on Github
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Make sure your PR passes the Rspec tests.

Contributors

Have a look at Github contributors to see a list of all the awesome contributors to this Puppet module. <3 This module was created and maintained by VSHN AG until the end of 2017. It was then donated to Voxpupuli so that a broader community is able to maintain the module.‚