Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rust): revamp the OpenAPI specification #1564

Merged
merged 7 commits into from
Oct 22, 2024
Merged

fix(rust): revamp the OpenAPI specification #1564

merged 7 commits into from
Oct 22, 2024

Conversation

imobachgs
Copy link
Contributor

@imobachgs imobachgs commented Aug 30, 2024

The agama-web-server docs command generates the OpenAPI specification of Agama's HTTP API. However, the resulting specification is badly broken 😢. This pull request aims to make the specification usable.

Refactoring

Apart from being incomplete, the previous implementation had a few problems.

  • The code relies on OpenApi proc-macro. This macro requires the data types to implement the ToSchema trait, and due to the orphan rule, we cannot implement it for third-party types (e.g., IpAddr, IpInet, etc.).
  • We did not find a way to add the description of the common APIs (e.g., progress or issues) under each "scope".

For those reasons, we decided to use the builder API, which is pretty flexible. We also took the chance to split the generation into smaller pieces, making it easier to maintain.

Packaging the OpenAPI documentation

Instead of generating the OpenAPI spec at runtime, we can do it at build time and put the resulting JSON files in a separate package (agama-openapi).

Generating the schema.d.ts

You can generate the API in two simple steps:

$ cargo run -p agama-server --bin agama-web-server openapi > openapi.json
$ npx @hey-api/openapi-ts -i openapi.json -o src/client -c axios

To do

  • Merging specifications is not the right approach because it does not detect conflicts (e.g., network and storage devices). Perhaps it might be a good idea to generate a separate specification for each scope (e.g., network).
  • Add third-party types descriptions.
  • Add validation to CI.

Future work

  • Add the DASD API.
  • Add the common interfaces (e.g., progress, issues, etc.)

@imobachgs imobachgs marked this pull request as ready for review August 30, 2024 12:43
@imobachgs imobachgs marked this pull request as draft August 30, 2024 13:36
@mvidner
Copy link
Contributor

mvidner commented Sep 2, 2024

I love schemas in general 😄 , they enable so many useful things...
And I've been wondering, even before this PR, what our use case for OpenAPI is.

IIUC, the output, saved as schema.d.ts (generated from the CLI output), will let us delete existing client JavaScript/TypeScript code?

@imobachgs
Copy link
Contributor Author

imobachgs commented Sep 2, 2024

I love schemas in general 😄 , they enable so many useful things... And I've been wondering, even before this PR, what our use case for OpenAPI is.

IMHO there are two use cases:

  1. generate the types for internal consumption,
  2. allow 3rd parties to use the specification for their own clients

IIUC, the output, saved as schema.d.ts (generated from the CLI output), will let us delete existing client JavaScript/TypeScript code?

The schema.d.ts was basically an experiment. Instead of generating the clients (which look pretty complex for our use case), I would like to generate only the types. But I am open to talking with the team about it.

Actually, I do not think it would be that hard to build our own generator for our use case.

@imobachgs
Copy link
Contributor Author

imobachgs commented Sep 4, 2024

If you want to give it a try, Hey API seems to be a nice alternative:

npx @hey-api/openapi-ts -i openapi.json -o client -c @hey-api/axios

imobachgs added a commit that referenced this pull request Sep 5, 2024
Trello: https://trello.com/c/nV7mjrf9/

This is the first step towards adapting the storage code to use queries.
As this part of the code is rather complex, we decided to move as many
components as possible to TypeScript to have a kind of security net.

## What is included

* Add a set of types from from our OpenAPI docs (see #1564) in
`api/storage/types`.
* Add a set of storage UI types in `types/storage`. We should unify the
types, but let's do it later.
* Define a new `api/storage` module moving big chunks of the
`StorageClient` code.
* Define a set of queries under `queries/storage`.
* Adapt the `ProposalPage` component to use queries. It retrieves a lot
of information that injects to nested components.
* Convert `ProposalPage` and `InstallationDeviceField` to TypeScript.

## What is missing

* As you may see, the front end is not building. That problems (and
other subtle bugs) will be fixed as we adapt the rest of storage
components to TypeScript. So a bunch of additional pull requests will
follow.
* Additionally, we are removing all those skeletons from the UI. We will
use a unified mechanism.
- name: Generate and validate the OpenAPI specification
run: |
cargo xtask openapi
openapi-spec-validator out/openapi/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great that it is validated 👍 , but does it also find missing piece?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if it is mentioned and not defined, it is found here (most of the problems were like that). About completely forgotten stuff, we might need to try something different (but I do not know what).

@imobachgs imobachgs marked this pull request as ready for review October 22, 2024 09:11
@imobachgs imobachgs merged commit 653ef75 into master Oct 22, 2024
5 checks passed
@imobachgs imobachgs deleted the fix-openapi branch October 22, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants