Human-readable specification of Tracing Policy API

[christian-2]

This commit includes initial markdown documentation for the Tracing Policy API, which was generated by OpenAPI Generator. It is intended as 4th "tile" in the reference section of the Tetragon online documentation.

fixes #2074

[netlify]

✅ Deploy Preview for tetragon ready!

Name	Link
🔨 Latest commit	bc4bde0
🔍 Latest deploy log	https://app.netlify.com/sites/tetragon/deploys/6606b23f7d0c3d0008a7c7f4
😎 Deploy Preview	https://deploy-preview-2152--tetragon.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[christian-2]

I see several errors in the deploy preview and will update this PR accordingly. Please defer a review until that's fixed.

[mtardy]

I see several errors in the deploy preview and will update this PR accordingly. Please defer a review until that's fixed.

I converted to draft

[christian-2]

I've looked a bit further into this and it seems to me that the Markdown is somehow compiled by Netlify into its own app format. (I wasn't aware of Netlify's role when I set out on this task.) While the Markdown looks legit to me, not everything seems to translate and hence the rendering differs from what I had expected. Also perhaps additional metadata (between --- brackets) would be needed to have the app include the additional Markdown files under Apis and Models as well.

Current assessment: perhaps this needs attention from somebody with more Netlify experience than I can muster.

[mtardy]

I've looked a bit further into this and it seems to me that the Markdown is somehow compiled by Netlify into its own app format. (I wasn't aware of Netlify's role when I set out on this task.) While the Markdown looks legit to me, not everything seems to translate and hence the rendering differs from what I had expected. Also perhaps additional metadata (between --- brackets) would be needed to have the app include the additional Markdown files under Apis and Models as well.

Current assessment: perhaps this needs attention from somebody with more Netlify experience than I can muster.

mmh to my understanding (and hopefully I should get it because I set up the whole docs thingie) Netlify should not change anything to the website, everything is compiled statically using Hugo and the Docsy theme. You should have exactly the same result locally, using make docs than with Netlify because it will just compile everything with Hugo and then serve the static files. If there's something different then I have missed something. And btw we don't use Netlify for https://tetragon.io, we just use it for preview.

[christian-2]

@mtardy I've by now gained a better understanding of how Hugo, Docsy and Netlify serve combined in the context of Tetragon and as a result have given this PR another try.

One can now navigate between the generated web pages on the deploy preview site (see Documentation / Reference / Tracing Policy API). Could you guys perhaps have a look at your convenience to assess whether it makes sense to pursue this further? There is quite a lot of detailed documentation gleaned from the source code (perhaps it's too much).

If the overall content seems fitting, the final goal IMO should be to automate the entire pipeline for generating it as part of make docs. (See also my comments in the latest version of the commit message.)

[mtardy]

@mtardy I've by now gained a better understanding of how Hugo, Docsy and Netlify serve combined in the context of Tetragon and as a result have given this PR another try.

One can now navigate between the generated web pages on the deploy preview site (see Documentation / Reference / Tracing Policy API). Could you guys perhaps have a look at your convenience to assess whether it makes sense to pursue this further? There is quite a lot of detailed documentation gleaned from the source code (perhaps it's too much).

If the overall content seems fitting, the final goal IMO should be to automate the entire pipeline for generating it as part of make docs. (See also my comments in the latest version of the commit message.)

could you maybe put everything on the same page so it's searchable? Plus you have a tons of pages without frontmatter that are messing up with the generation here. That could solve both issues.

[christian-2]

@mtardy re single page: OpenAPI Generator, the tool I've been using, apparently has three relevant (documentation) generators: html, html2, and markdown. I've applied all three of them; preliminary output can be viewed here. html and html2 put everything on a single page. However, the current PR employs the markdown generator and FWIK it will always generate multiple pages. The employed generator could be switched of course.

re Hugo front matter: I am currently inserting front matter only on the single page markdown page tracing-policy-api.md, but I could extend this to all other generated pages as well. What particular front matter (which fields) would you prefer?

[christian-2]

@mtardy since you suggested putting everything on the same page I have switched generators from markdown to html. The generation is by now also self-contained and occurs as part of make -C docs.

The are two new stages in docs/Dockerfile.hugo. Stage openapi-spec obtains the OpenAPI specification for Tetragon from an embedded Kubernetes cluster. Subsequent stage openapi-documentation employs OpenAPI Generator (its generator html) to derive the HTML page from that specification.

I had to change docs/Makefile such that it builds the Docker image hugo in the parent directory of Dockerfile.hugo. This includes Tetragon's CRDs (in pkg/k8s/apis/cilium.io/client/crds/v1alpha1), which serve as input to stage openapi-spec, in the build context. I also had to remove enableGitInfo = true from docs/hugo.toml, for otherwise make -C docs fails. (I don't recall the exact point where it does, but could presumably reproduce it, if necessary.)

OpenAPI Generator turned out to be quite a "Java monster" (it apparent serves OpenAPI code generation primarily, not OpenAPI documentation generation alone), and one could contemplate replacing it with another OpenAPI documentation generator in stage openapi-documentation, with stage openapi-spec remaining unaffected.

For now, I have included the generated HTML page docs/content/en/docs/reference/tracing-policy-api.html (1.2M) by copying it manually from the container hugo to this location. Perhaps there should be a special target in some Makefile that is dedicated to keeping it up to date in that location (with logic that currently sits in the two new stages of Dockerfile.hugo). Since I know little about the full build pipeline and the intended dependencies between targets, I have left this task open for now.