diff --git a/Readme.md b/Readme.md index ced7e91..ff78bfb 100644 --- a/Readme.md +++ b/Readme.md @@ -8,22 +8,9 @@ Currently supported: This is achieved by using Gitlabs "System Hooks" (not to be confused with server hooks or file hooks) that perform `HTTP POST` requests and are triggered by various Gitlab system events. You can see a complete list on Gitlabs [System Hooks page](https://docs.gitlab.com/ee/administration/system_hooks.html). -#### To create a system hook: - -> You must have Administrative access to the Gitlab instance you wish to configure the System Hook. - -1. Open the intended Gitlab instance you want to trigger events on. -2. On the left sidebar, expand the top-most chevron. -3. Select **Admin Area**. -4. On the left sidebar, select **System Hooks**. -5. Provide the **URL** and **Secret Token**. -6. Select the checkbox next to each optional **Trigger** you want to enable. -7. Select **Enable SSL verification**, if desired. -8. Select **Add system hook**. - -> You can easily debug your system hook receiver by going to **Admin Area -> System Hooks > Edit**, On the bottom of the screen there are “**Recent Deliveries**”. You can go to “View details” on each one of the requests in order to get additional debugging information— you can check the “Request headers” & “Response headers” of your request, the “Response body” that came back from your receiver and the JSON itself. +## Configuration -## Environment Variables +### Environment Variables Before running the webhook, you need to set the following environment variables: * `GITLAB_URL`: Required. The URL of your Gitlab instance (e.g., https://your.gitlab.domain). * `GITLAB_ACCESS_TOKEN`: Required. Gitlab access token with admin privileges. @@ -31,7 +18,7 @@ Before running the webhook, you need to set the following environment variables: * `CONFIG_FILE_PATH`: Optional. The path to the configuration file (`config.yml`) that defines the injection rules for new projects. (Default value: `'config.yml'`) * `LISTEN_PORT`: Optional: The port on which the webhook will listen for incoming requests. (Default value: `3002`) -## Configuration +### `Config.yaml` The config.yml file defines the injection rules for new projects. It consists of an array of objects, each representing a specific rule for project injection. Below is an explanation of the properties used in the configuration: #### `regex` (required) @@ -65,6 +52,8 @@ For example, to add two groups with different access levels to a project: access: 30 # Rest of the properties... ``` +#### `commits` +Commit files that you'd like to be injected into the project. You then define the `message` and `paths` for that commit. #### `message` (optional) This property defines the default commit message that will be used when creating commits for new projects. If not specified, a default placeholder commit message will be used. For example: @@ -78,16 +67,16 @@ This property defines the default commit message that will be used when creating This property specifies the files to be injected into the project. It should be an array of objects, where each object has a **source** and a **target** property. The **source** property specifies the path to the file that will be injected, and the **target** property specifies the path where the file will be placed in the new project. For example: ```yaml - regex: "example-project" - paths: - - source: "includes/.gitignore" - target: ".gitignore" - - source: "includes/README.md" - target: "README.md" + commit: + paths: + - source: "includes/.gitignore" + target: ".gitignore" + - source: "includes/README.md" + target: "README.md" # Rest of the properties... ``` -### Complete Example `config.yml` - +#### Complete Example `config.yml` ```yaml - regex: "^documentation/" groups: @@ -95,39 +84,75 @@ This property specifies the files to be injected into the project. It should be access: 30 - name: developers access: 10 - message: "This is a commit for documentation repositories..." - paths: [] + commit: + message: "This is a commit for documentation repositories..." + paths: [] - regex: "^game/" groups: - name: managers access: 30 - message: "This is a commit for games repositories..." - paths: - - source: "includes/games/.gitignore" - target: ".gitignore" + commit: + message: "This is a commit for games repositories..." + paths: + - source: "includes/games/.gitignore" + target: ".gitignore" - regex: ".*" # Default regex - message: "This is a commit for default repositories..." - paths: - - source: "includes/default/.gitignore" - target: ".gitignore" - - source: "includes/default/.gitlab-ci.yml" - target: ".gitlab-ci.yml" - - source: "includes/default/README.md" - target: "README.md" + commit: + message: "This is a commit for default repositories..." + paths: + - source: "includes/default/.gitignore" + target: ".gitignore" + - source: "includes/default/.gitlab-ci.yml" + target: ".gitlab-ci.yml" + - source: "includes/default/README.md" + target: "README.md" ``` -### GitLab API Tokens -Go to profile/personal_access_tokens on your GitLab instance. +## Usage + +### Endpoints +| Endpoint | Type | Purpose | +|---------- |------ |-------------------------------------------------- | +| / | POST | Verfiy the server is working | +| /hook | GET | Trigger the action on server | +| /config | POST | Get a printout of the current configuration yaml | + +### Gitlab Instance -To create a Personal Access Token, give it a name, and check the api box under Scopes. -Then click the Create personal access token button. +#### GitLab API Tokens +To authorise the hook within Gitlabs API's, you need to pass a Personal Access Tokens. + +1. Open the intended Gitlab instance you want to trigger events on. +2. Log in with the intended account you wish to generate the Personal Access Tokens for. +3. Select **Profile**. +4. On the left sidebar, select **Personal Access Tokens**. +5. Provide the **Name**. +6. Select the checkbox next to each optional **Scope** you want to enable. +3. Click the **Create** personal access token button. > Note: You may want to do this step with a unique account that doesn't belong to an actual user. This prevents problems if the token holder's user leaves your org and their account has to be deleted. +#### To create a system hook: +> You must have Administrative access to the Gitlab instance you wish to configure the System Hook. + +The **URL** should be: `yourdomain.tld/hook/` -## Usage with Docker +1. Open the intended Gitlab instance you want to trigger events on. +2. On the left sidebar, expand the top-most chevron. +3. Select **Admin Area**. +4. On the left sidebar, select **System Hooks**. +5. Provide the **URL** and **Secret Token**. +6. Select the checkbox next to each optional **Trigger** you want to enable. +7. Select **Enable SSL verification**, if desired. +8. Select **Add system hook**. + +> You can easily debug your system hook receiver by going to **Admin Area -> System Hooks > Edit**, On the bottom of the screen there are “**Recent Deliveries**”. You can go to “View details” on each one of the requests in order to get additional debugging information— you can check the “Request headers” & “Response headers” of your request, the “Response body” that came back from your receiver and the JSON itself. + +### Docker + +#### Usage with Docker If you prefer to run the webhook in a Docker container, you can use the following command: ```bash @@ -143,7 +168,7 @@ docker run -d \ ghcr.io/keiranlovett/gitlab-server-hook:main ``` -## Usage without Docker +#### Usage without Docker If you want to run the webhook without Docker, follow these steps: 1. Make sure you have [Node.js](https://nodejs.org) installed. @@ -153,11 +178,9 @@ If you want to run the webhook without Docker, follow these steps: 5. Add a system hook in Gitlab that points to the webhook's URL to receive project creation events. 6. Create a new project in Gitlab, and it will automatically be shared with the specified default group. - - # TODO / Contributing -- [] Support for adding specific `Members` to a project. -- [] Enfore a project naming convention. -- [] Custom logging support??? -- Pulling or changing data in LDAP servers. +- [ ] Support for adding specific `Members` to a project. +- [ ] Enforce a project naming convention. +- [ ] Custom logging support??? +- [ ] Pulling or changing data in LDAP servers. diff --git a/app.js b/app.js index 11f3051..85ddd08 100644 --- a/app.js +++ b/app.js @@ -9,18 +9,10 @@ let express = require('express'), path = require('path'), yaml = require('js-yaml'); - const injectiblesFilePath = '../config/'; -const officialAccessLevels = { - 0: 'No access', - 5: 'Minimal access', - 10: 'Guest', - 20: 'Reporter', - 30: 'Developer', - 40: 'Maintainer', - 50: 'Owner', -}; +// Import the plugin functions +const projectCreateEvent = require("./events/project_create"); const url = process.env.GITLAB_URL, accessToken = process.env.GITLAB_ACCESS_TOKEN, @@ -28,6 +20,14 @@ const url = process.env.GITLAB_URL, configFilePath = process.env.CONFIG_FILE_PATH || 'config.yml', port = process.env.LISTEN_PORT || 3002; + +// Map event names to corresponding plugin functions +const eventPlugins = { + project_create: projectCreateEvent +}; + + + const { Gitlab } = require('@gitbeaker/rest'); const api = new Gitlab({ @@ -43,25 +43,28 @@ let projectConfigs, verifyEnvironment(); watchConfigFile(); +// Export a function to get the loaded project configurations +module.exports.getProjectConfigs = function () { + return projectConfigs; +}; + // Functions -function loadConfig(callback) { +function loadConfig() { try { const fullConfigPath = path.join(injectiblesFilePath, configFilePath); const configFile = fs.readFileSync(fullConfigPath, 'utf8'); - projectConfigs = yaml.load(configFile, { schema: yaml.JSON_SCHEMA }); - projectConfigs.forEach((gic) => { + const loadedConfigs = yaml.load(configFile, { schema: yaml.JSON_SCHEMA }); + loadedConfigs.forEach((gic) => { if (gic.regex) { gic.regex = new RegExp(gic.regex); // Convert the regex string to a regular expression } }); - console.log('Configuration file loaded successfully.'); - if (typeof callback === 'function') { - callback(); // Invoke the callback function if provided - } - // Perform the sanity check after loading the configuration - validateAccessLevels(); + // Update the projectConfigs with the new loaded configurations + projectConfigs = loadedConfigs; + + console.log('Configuration file loaded successfully.'); } catch (err) { console.error('Error loading configuration file:', err); } @@ -76,17 +79,20 @@ function watchConfigFile() { // If the server promise exists, it means the server is running, so stop it and reload the configuration serverPromise.then(() => { stopWebserver(() => { - loadConfig(startWebserver); + loadConfig(); // Simply reload the configuration without checking the return value + startWebserver(); }); }); } else { // If the server promise does not exist, the server is not running, so simply reload the configuration and start the server - loadConfig(startWebserver); + loadConfig(); // Simply reload the configuration without checking the return value + startWebserver(); } } }); } + function verifyEnvironment() { if (typeof url == 'undefined') { console.log('GITLAB_URL not set. Please provide the url of the instance you wish to connect to.'); @@ -135,29 +141,39 @@ function startWebserver() { res.send('Hello, please give me a POST!\n'); }); + // Route to handle the '/config' endpoint + webserver.get('/config', function (req, res) { + // Generate the human-friendly output + const formattedOutput = projectConfigs.map((config, index) => { + return ` + Project Config ${index + 1}: + Regex: ${config.regex} + Commit Message: ${config.commit.message} + Groups: ${JSON.stringify(config.groups)} + Commit Paths: ${JSON.stringify(config.commit.paths)} + `; + }).join('\n'); + + // Send the formatted output back to the client + res.send(`
${formattedOutput}
`); + }); + webserver.post('/', function (req, res) { const eventName = req.body['event_name']; - if (eventName === 'project_create') { - /* - Example request: - - { - "project_id" : 1, - "owner_email" : "example@gitlabhq.com", - "owner_name" : "Someone", - "name" : "Ruby", - "path" : "ruby", - "event_name" : "project_create" - } - */ - handleProjectCreateEvent(req, res); + const eventData = req.body; + + // Check if the event has a corresponding plugin + if (eventPlugins[eventName]) { + const pluginFunction = eventPlugins[eventName]; + pluginFunction(eventData, projectConfigs, api, defaultBranch, () => { + res.send('Project Create Event handled successfully.'); + }); + } else { - console.log(`Unknown event_name: ${eventName}`); - res.status(400).send({ message: `Unknown event_name: ${eventName}` }); + console.log(`No plugin found for event: ${eventName}`); } }); - // Start the server and save the promise serverPromise = new Promise((resolve) => { server = webserver.listen(port, () => { @@ -181,167 +197,11 @@ function stopWebserver(callback) { } } -// Add error handling for api.Commits.create -async function createCommit(projectId, projectNamespace, defaultBranch, actions) { - try { - - let commitMessage = ''; - for (const config of projectConfigs) { - if (config.regex && config.regex.test(projectNamespace)) { - commitMessage = config.message || 'Placeholder commit message....'; - break; - } - } - - await api.Commits.create(projectId, defaultBranch, commitMessage, actions); - console.log('Commit created successfully.'); - } catch (err) { - console.log(`Error creating commit: ${err.message}`); - } -} - - -// Function to handle project_create event -async function handleProjectCreateEvent(req, res) { - console.log(`New Project "${req.body['name']}" created.`); - - try { - const projectId = req.body['project_id']; - const projectName = req.body['path_with_namespace']; - const projectConfig = projectConfigs.find( - (gic) => gic.regex && gic.regex.test(projectName) - ); - - if (projectConfig) { - await addCommitToProject(projectId, projectName, projectConfig); - await addGroupsToProject(projectId, projectConfig); - } else { - console.log('No projectConfig found for the project.'); - } - - } catch (err) { - console.log(`Something unexpected went wrong! Error: ${err}`); - res.status(500).send({ message: 'Something unexpected went wrong!' }); - } -} - - -// Function to add groups to the project with the specified access levels -async function addGroupsToProject(projectId, projectConfig) { - try { - // Check if the projectConfig has any groups specified - if (projectConfig.groups && projectConfig.groups.length > 0) { - for (const groupInfo of projectConfig.groups) { - try { - // Check if the groupInfo object has both name and access properties - if (!groupInfo.name || !groupInfo.access) { - console.log('Error: Invalid groupInfo object. It must have both "name" and "access" properties.'); - continue; // Skip this iteration and move to the next groupInfo - } - - const groupName = groupInfo.name; - const accessLevel = groupInfo.access; - // First, get the group details using the name - const group = await api.Groups.show(groupName); - - // Then, add the group to the project with the specified access level - await api.Projects.share(projectId, group.id, accessLevel); - - console.log(`Group "${groupName}" added to the project with access level: ${accessLevel}.`); - } catch (err) { - console.log(`Error adding group "${groupInfo.name}" to project: ${err.message}`); - } - } - } - } catch (err) { - console.log(`Error adding groups to project: ${err.message}`); - } -} - -// Function to add groups to the project with the specified access levels -async function addCommitToProject(projectId, projectNamespace, projectConfig) { - try { - - if (projectConfig.paths.length === 0) { - console.log(`Terminating: No content to inject for files matching regex: ${projectConfig.regex && projectConfig.regex.toString()}`); - return; - } else { - console.log(`Accepting: Content to inject for files matching regex: ${projectConfig.regex && projectConfig.regex.toString()}`); - // Log all the files included in the projectConfig - for (const pathConfig of projectConfig.paths) { - console.log(`Source: ${pathConfig.source}, Target: ${pathConfig.target}`); - } - } - - // Read files and create actions using the new function - const data = await Promise.all(projectConfig.paths.map((p) => readFileAsync(p.source, 'utf8'))); - const actions = await determineActions(projectId, defaultBranch, projectConfig.paths, data); - - console.log('Actions:', actions); - - await createCommit(projectId, projectNamespace, defaultBranch, actions); - - } catch (err) { - console.log(`Error commits to the project: ${err.message}`); - } -} - - - -// Function to determine the action based on file existence -async function determineActions(projectId, branch, paths, contentData) { - const actions = []; - for (let index = 0; index < contentData.length; index++) { - const content = contentData[index]; - const targetPath = paths[index].target; - const fileExists = await getFileExists(projectId, targetPath, branch); - const action = fileExists ? 'update' : 'create'; - actions.push({ - action, - file_path: targetPath, - content, - }); - } - return actions; -} - -async function readFileAsync(file, encoding) { - const fullSourcePath = path.join(injectiblesFilePath, file); - return new Promise((resolve, reject) => { - fs.readFile(fullSourcePath, encoding, (err, data) => { - if (err) { - reject(err); - } else { - resolve(data); - } - }); - }); -} - -async function getFileExists(project_id, file, branch) { - let fileExists = false; - try { - await api.RepositoryFiles.show(project_id, file, branch); - fileExists = true; - } catch(e) { - console.log(`Error reading file: ${e.message}`); - } - return fileExists; +// Functions for handling project_create event +function getProjectConfig(projectName) { + // Load and return the project configuration based on the projectName + const projectConfig = projectConfigs.find( + (gic) => gic.regex && gic.regex.test(projectName) + ); + return projectConfig; } - -function validateAccessLevels() { - if (!projectConfigs) { - return; - } - - projectConfigs.forEach((gic) => { - if (Array.isArray(gic.groups)) { - for (const groupInfo of gic.groups) { - const { name, access } = groupInfo; - if (typeof access !== 'number' || !(access in officialAccessLevels)) { - console.warn(`Warning: Group "${name}" has an invalid or unsupported access level (${access}).`); - } - } - } - }); -} \ No newline at end of file diff --git a/events/project_create.js b/events/project_create.js new file mode 100644 index 0000000..f4ff459 --- /dev/null +++ b/events/project_create.js @@ -0,0 +1,160 @@ +let path = require('path'), + fs = require('fs'); + +const officialAccessLevels = { + 0: 'No access', + 5: 'Minimal access', + 10: 'Guest', + 20: 'Reporter', + 30: 'Developer', + 40: 'Maintainer', + 50: 'Owner', +}; + +const REQUEST_DELAY = 5000; +const injectiblesFilePath = '../config/'; + +module.exports = function (eventData, projectConfigs, api, defaultBranch, injectiblesFilePath, callback) { + const projectId = eventData.project_id; + const projectName = eventData.path_with_namespace; + const projectConfig = projectConfigs.find((gic) => gic.regex && gic.regex.test(projectName)); + + console.log(`New Project created: "${eventData.name}" at namespace "${projectName}" with id: "${projectId}"`); + + // Define inner functions that have access to the passed variables + async function addGroupsToProject() { + // Perform the sanity check after loading the configuration + validateAccessLevels(projectConfig); + + try { + // Check if the projectConfig has any groups specified + if (projectConfig.groups && projectConfig.groups.length > 0) { + for (const groupInfo of projectConfig.groups) { + try { + // Check if the groupInfo object has both name and access properties + if (!groupInfo.name || !groupInfo.access) { + console.log('Error: Invalid groupInfo object. It must have both "name" and "access" properties.'); + continue; // Skip this iteration and move to the next groupInfo + } + + const groupName = groupInfo.name; + const accessLevel = groupInfo.access; + // First, get the group details using the name + const group = await api.Groups.show(groupName); + + // Then, add the group to the project with the specified access level + await api.Projects.share(projectId, group.id, accessLevel); + + console.log(`Group "${groupName}" added to the project with access level: ${accessLevel}.`); + } catch (err) { + console.log(`Error adding group "${groupInfo.name}" to project: ${err.message}`); + } + } + } + } catch (err) { + console.log(`Error adding groups to project: ${err.message}`); + } + } + + async function addCommitToProject() { + try { + if (!projectConfig) { + console.log(`No projectConfig found for the project "${projectName}".`); + return; + } + + if (projectConfig.commit.paths.length === 0) { + console.log(`Terminating: No content to inject for files matching regex: ${projectConfig.regex && projectConfig.regex.toString()}`); + return; + } else { + console.log(`Accepting: Content to inject for files matching regex: ${projectConfig.regex && projectConfig.regex.toString()}`); + // Log all the files included in the projectConfig + for (const pathConfig of projectConfig.commit.paths) { + console.log(`Source: ${pathConfig.source}, Target: ${pathConfig.target}`); + } + } + + // Read files and create actions using the new function + const data = await Promise.all(projectConfig.commit.paths.map((p) => readFileAsync(p.source, 'utf8'))); + const actions = await determineActions(projectConfig.commit.paths, data); + + console.log('Actions:', actions); + + setTimeout(() => { + createCommit(actions, projectConfig); + }, REQUEST_DELAY) + } catch (err) { + console.log(`Error commits to the project: ${err.message}`); + } + } + + async function createCommit(actions, projectConfig) { + try { + let commitMessage = projectConfig.commit.message || 'Placeholder commit message....'; + + await api.Commits.create(projectId, defaultBranch, commitMessage, actions); + console.log('Commit created successfully.'); + } catch (err) { + console.log(`Error creating commit: ${err.message}`); + } + } + + async function determineActions(paths, contentData) { + const actions = []; + for (let index = 0; index < contentData.length; index++) { + const content = contentData[index]; + const targetPath = paths[index].target; + const fileExists = await getFileExists(targetPath); + const action = fileExists ? 'update' : 'create'; + actions.push({ + action, + file_path: targetPath, + content, + }); + } + return actions; + } + + async function readFileAsync(file, encoding) { + const fullSourcePath = path.join(injectiblesFilePath, file); + return new Promise((resolve, reject) => { + fs.readFile(fullSourcePath, encoding, (err, data) => { + if (err) { + reject(err); + } else { + resolve(data); + } + }); + }); + } + + async function getFileExists(file) { + let fileExists = false; + try { + await api.RepositoryFiles.show(projectId, file, defaultBranch); + fileExists = true; + } catch(e) { + console.log(`Error reading file: ${e.body} ${e.message}`); + } + return fileExists; + } + + function validateAccessLevels(projectConfig) { + projectConfig.groups.forEach((groupInfo) => { + const { name, access } = groupInfo; + if (typeof access !== 'number' || !(access in officialAccessLevels)) { + console.warn(`Warning: Group "${name}" has an invalid or unsupported access level (${access}).`); + } + }); + } + + // Call the inner functions + addCommitToProject(); + addGroupsToProject(); + + // Call the provided callback at the end of the module + if (typeof callback === 'function') { + callback(); + } + +}; diff --git a/package-lock.json b/package-lock.json index dcc1a55..a32ea9e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,10 +1,11 @@ { - "name": "Github-Hook", + "name": "@keiranlovett/Gitlab-Server-Hook", "version": "0.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { + "name": "@keiranlovett/Gitlab-Server-Hook", "version": "0.0.0", "dependencies": { "@gitbeaker/rest": "^39.9.0",