Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compatible: Add sync_docs.yaml #220

Merged
merged 68 commits into from
Aug 16, 2024
Merged
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
fd58087
First commit
a-velasco Aug 5, 2024
8821fbc
Misc PR review fixes
a-velasco Aug 7, 2024
71f818c
Removing run schedule since there are no longer API constraints
a-velasco Aug 7, 2024
330982e
Using GITHUB_TOKEN instead of PA TOKEN
a-velasco Aug 7, 2024
a2e0463
Update .github/workflows/_sync_docs_v2.yaml
a-velasco Aug 8, 2024
6b8e227
Update .github/workflows/_sync_docs_v2.yaml
a-velasco Aug 8, 2024
0561751
Update .github/workflows/_sync_docs_v2.yaml
a-velasco Aug 8, 2024
ac42f67
Update .github/workflows/_sync_docs_v2.md
a-velasco Aug 8, 2024
ee2866b
Update python/cli/data_platform_workflows_cli/sync_docs_v2.py
a-velasco Aug 8, 2024
c92536f
Update python/cli/data_platform_workflows_cli/sync_docs_v2.py
a-velasco Aug 8, 2024
3c43657
Update .github/workflows/_sync_docs_v2.yaml
a-velasco Aug 8, 2024
64f0a61
Small edits
a-velasco Aug 8, 2024
17e4327
Replaced original sync_docs with v2 and removed experimental file pre…
a-velasco Aug 8, 2024
39cd6c0
Merge branch 'download-discourse-topics' of github.com:canonical/data…
a-velasco Aug 8, 2024
bd3c59c
Update sync_docs.md
a-velasco Aug 8, 2024
0d196b5
Update README.md
a-velasco Aug 8, 2024
908c137
Update sync_docs.yaml
a-velasco Aug 8, 2024
67f50a4
Update .github/workflows/sync_docs.md
a-velasco Aug 9, 2024
ee6932f
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 9, 2024
5731b97
Update .github/workflows/sync_docs.yaml
a-velasco Aug 9, 2024
9f46bd4
Update .github/workflows/sync_docs.yaml
a-velasco Aug 9, 2024
d645e02
Update README.md
a-velasco Aug 9, 2024
ca1164a
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 9, 2024
6460585
Formatted with isort
a-velasco Aug 9, 2024
d05d5ff
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 9, 2024
5316d89
Update sync_docs.md
a-velasco Aug 9, 2024
aa3c9ed
Update sync_docs.md
a-velasco Aug 9, 2024
7a8e201
Update sync_docs.md
a-velasco Aug 9, 2024
eafdffd
Update sync_docs.py
a-velasco Aug 9, 2024
917485d
Reformatted with black
a-velasco Aug 9, 2024
785629c
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 9, 2024
d9d979a
Update sync_docs.md
a-velasco Aug 9, 2024
e051a00
Update sync_docs.yaml
a-velasco Aug 12, 2024
69c58fe
Update sync_docs.yaml
a-velasco Aug 12, 2024
e3cd611
test
a-velasco Aug 12, 2024
3225cd9
Update sync_docs.yaml
a-velasco Aug 12, 2024
c369e5c
Update sync_docs.yaml
a-velasco Aug 12, 2024
c7e6000
Add logging
carlcsaposs-canonical Aug 12, 2024
46f8dbc
Fixed bug where first line of navtable was getting automatically filt…
a-velasco Aug 12, 2024
4e33b34
Update .github/workflows/sync_docs.yaml
a-velasco Aug 12, 2024
6cf1cdd
temp (debug)
a-velasco Aug 12, 2024
7bdf969
temp (debug)
a-velasco Aug 12, 2024
2211711
temp (debug)
a-velasco Aug 12, 2024
ac7d9dc
Update sync_docs.yaml
a-velasco Aug 12, 2024
fd0a2ed
Update sync_docs.yaml
a-velasco Aug 12, 2024
ba261b1
Added overview topic download and support for valid non-diataxis topics
a-velasco Aug 12, 2024
3a81e0c
Update .github/workflows/sync_docs.yaml
a-velasco Aug 12, 2024
f7c2c03
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 12, 2024
2adc0db
Moved download logic into Topic class
a-velasco Aug 12, 2024
670c897
Merge branch 'download-discourse-topics' of github.com:canonical/data…
a-velasco Aug 12, 2024
663ffc0
Update sync_docs.md
a-velasco Aug 12, 2024
0bde819
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 12, 2024
9b77eac
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 12, 2024
53944e4
Rename topic download function
a-velasco Aug 12, 2024
80b8347
Some rephrasing
a-velasco Aug 12, 2024
9c0d0f2
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 12, 2024
c6d1a5f
Update python/cli/data_platform_workflows_cli/sync_docs.py
a-velasco Aug 12, 2024
fdeaf62
Update sync_docs.md
a-velasco Aug 12, 2024
71981b5
Formatting
a-velasco Aug 12, 2024
982fce9
Update .github/workflows/sync_docs.md
a-velasco Aug 13, 2024
f121d08
Update .github/workflows/sync_docs.md
a-velasco Aug 13, 2024
d1b71fb
Update sync_docs.md
a-velasco Aug 13, 2024
46b5238
Update .github/workflows/sync_docs.md
a-velasco Aug 13, 2024
d33bea5
Update .github/workflows/sync_docs.md
a-velasco Aug 13, 2024
dccdd38
Update .github/workflows/sync_docs.md
a-velasco Aug 13, 2024
d048543
Update sync_docs.md
a-velasco Aug 13, 2024
e885889
Update sync_docs.md
a-velasco Aug 13, 2024
5a80297
Fixed bad indentation in yaml template
a-velasco Aug 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 0 additions & 74 deletions .github/workflows/_sync_docs.md

This file was deleted.

31 changes: 0 additions & 31 deletions .github/workflows/_sync_docs.yaml

This file was deleted.

95 changes: 95 additions & 0 deletions .github/workflows/sync_docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
Workflow file: [sync_docs.yaml](sync_docs.yaml)

## Usage
Add `.yaml` file to `.github/workflows/`

```yaml
# Copyright 2024 Canonical Ltd.
# See LICENSE file for licensing details.
name: Sync Discourse docs

on:
workflow_dispatch:
schedule:
- cron: '53 0 * * *' # Daily at 00:53 UTC

jobs:
sync-docs:
name: Sync docs from Discourse
uses: canonical/data-platform-workflows/.github/workflows/[email protected]
a-velasco marked this conversation as resolved.
Show resolved Hide resolved
with:
reviewers: canonical/data-platform-technical-authors,octocat
permissions:
contents: write # Needed to push branch & tag
pull-requests: write # Needed to create PR
```

## Behavior

Downloads all Discourse topics in the charm's Charmhub documentation to `/docs/` directory in the charm's repository.

>[!NOTE]
> Any content in the `/docs/` directory that is not part of this workflow will get removed.

The topics are determined by the navigation table in the charm's overview page - i.e. the page linked in the `metadata.yaml` `docs:` field.

When the workflow is triggered, it downloads all Discourse topics in their latest state and compares them to the `/docs/` directory in the default branch (e.g. `main`).
* If the contents match, GitHub is up to date with Discourse. Nothing happens.
* If the contents do not match, Discourse is ahead of GitHub. The workflow opens or updates a PR for the `sync-docs` branch with the diff between Discourse and GitHub.

Each Discourse topic is downloaded to either: `/docs/`, `/docs/tutorial/`, `/docs/how-to/`, `/docs/reference/`, or `/docs/explanation/` depending on their slug prefix. Sub-categories will not create an additional directory.

```
# Example navtable:

| Level | Path | Navlink |
|-------|----------------|---------------------------|
| 1 | t-landing | [Tutorial](/t/123) |
| 2 | t-introduction | [Introduction](/t/124) |
| 1 | h-landing | [How To]() |
| 2 | h-set-up | [Set up](/t/125) |
| 3 | h-deploy-lxd | [Deploy on LXD](/t/126) |
| 2 | h-monitoring | [Monitoring]() |
| 3 | h-alert | [Add alert rules](/t/127) |

# Expected /docs output:

/docs/overview.md
/docs/tutorial/t-landing.md
/docs/tutorial/t-introduction.md
/docs/how-to/h-set-up.md
/docs/how-to/h-deploy-lxd.md
/docs/how-to/h-alert.md
```

## Documentation requirements

**Requirement**: The overview topic must contain a navigation table with the columns "Level", "Path", and "Navlink"

**Requirement**: The navigation table of the overview topic must be wrapped by the `[details]` HTML/markdown element as follows:

```
# Navigation

[details=Navigation]

<navigation table goes here>

[/details]

```

> [!NOTE]
> Make sure to leave a white space before and after the `[details]` and `[/details]` lines. Otherwise, Discourse/Charmhub may not parse them correctly.

**Requirement**: Links to topics in the `Navlink` column must be formatted as `[Text](/t/<topic_id>)`.

They **must not** be formatted as `[Text](/t/<some-additional-slug-text>/<topic_id>)`.

### Examples
The following documentation sets fulfill the above requirements and have been tested:
* [PostgreSQL VM](https://discourse.charmhub.io/t/charmed-postgresql-documentation/9710)
* [PostgreSQL K8s](https://discourse.charmhub.io/t/charmed-postgresql-k8s-documentation/9307)
* [PgBouncer VM](https://discourse.charmhub.io/t/pgbouncer-documentation/12133)
* [PgBouncer K8s](https://discourse.charmhub.io/t/pgbouncer-k8s-documentation/12132)
* [MySQL Router K8s](https://discourse.charmhub.io/t/mysql-router-k8s-documentation/12130)
58 changes: 58 additions & 0 deletions .github/workflows/sync_docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
on:
workflow_call:
inputs:
reviewers:
description: Comma separated list of GitHub usernames to request to review pull request (e.g. "canonical/data-platform-engineers,octocat")
required: false
type: string

jobs:
sync-docs:
name: Sync docs from Discourse
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Get workflow version
id: workflow-version
uses: canonical/get-workflow-version-action@v1
with:
repository-name: canonical/data-platform-workflows
file-name: sync_docs.yaml
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install CLI
run: pipx install git+https://github.com/canonical/data-platform-workflows@'${{ steps.workflow-version.outputs.sha }}'#subdirectory=python/cli
- name: Checkout
uses: actions/checkout@v4
- name: Update Discourse docs
run: sync-docs
- name: Push `sync-docs` branch & create pull request
run: |
git checkout -b sync-docs
git add docs/
# Check if changes staged
if git diff --cached --quiet
then
echo 'No changes to docs/ from default git branch'
# Delete branch on GitHub if it exists
# (GitHub will automatically close PR if it exists)
if git ls-remote --exit-code origin sync-docs
then
git push origin --delete sync-docs
fi
exit 0
fi
git config user.name "GitHub Actions"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git commit -m "Sync docs from Discourse"
git push origin sync-docs -f
# Create pull request
# Capture output in variable so that step fails if `gh pr list` exits with non-zero code
prs=$(gh pr list --head sync-docs --state open --json number)
if [[ $prs != "[]" ]]
then
echo Open pull request already exists
exit 0
fi
gh pr create --head sync-docs --title "Sync docs from Discourse" --body "Sync charm docs from https://discourse.charmhub.io" --reviewer '${{ inputs.reviewers }}'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
| [release_charm.yaml](.github/workflows/release_charm.md) | Release charm to Charmhub |
| [update_bundle.yaml](.github/workflows/update_bundle.md) | Update charm revisions in bundle |
| [sync_issue_to_jira.yaml](.github/workflows/sync_issue_to_jira.md) | Sync GitHub issues to Jira issues |
| [_sync_docs.yaml](.github/workflows/_sync_docs.md) | **Experimental** Sync Discourse documentation to GitHub |
| [sync_docs.yaml](.github/workflows/sync_docs.md) | Sync Discourse documentation to GitHub |

### Version
Recommendation: pin the latest version (e.g. `v1.0.0`) and use [Renovate](https://docs.renovatebot.com/) to stay up-to-date.
Expand Down Expand Up @@ -72,4 +72,4 @@ Workflows that begin with one underscore (e.g. `_foo.yaml`) are internal and are
Workflows that begin with two underscores (e.g. `__foo.yaml`) are for this repository only. They may only be (triggered by an event on this repository or) called by workflows in this repository that begin with two underscores.

## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md)
See [CONTRIBUTING.md](CONTRIBUTING.md)
Loading