Output profiles - changelog

[tibulca]

Currently, oasdiff outputs results in json, yaml, text, and HTML format, all representing a detailed view of the differences between 2 openapi specs, and there is also the summary output.
The recently introduced -check-breaking added a new way of presenting (breaking) changes, in a format that can be very useful for generating a changelog. Example: API PATCH /api/databaseUsers/{username} removed the request property 'customField1' [request-property-removed]

For implementing changelog generation by comparing 2 specs, we're thinking about extending oasdiff by adding a new -profile (output profile: default, summary or changelog) flag and implementing one of these options:

1. [preferred] implement checkers (INFO level) for most common changelog entries (e.g. endpoint added, deprecated, removed, field added in the request, field type changed etc). Sample PR that started with this approach: Output profiles tibulca/oasdiff#1
   Pros: relies on the new approach of describing changes (human readable & includes localization), can flag changelog entries as breaking/non-breaking
   Cons: because of the complexity of the diff report, it is hard to write checkers for all scenarios, but even more important, not trivial to know which diff cases are covered and which not. Still, it's a good option if we're going to do it in an incremental way, by keep adding checkers as we find the need for them.
2. implement a changelogReport similar to the existing report.go (use for text output).
   Pros: simpler approach, with a clear understanding of what's covered and what's not in the changelog (so over time, easier to contribute with improvements)
   Cons: harder to integrate with the breaking changes code (the checkers) in order to differentiate between breaking and non-breaking changelog entries. Harder to implement localization (context has to be passed to the leafs and processed there in order to generate the final localized string) compared to the way it is done in the checkers.

I'm trying to find out if you are interested in having this changelog feature and if yes, which option aligns more with your long-term plans.

Thanks,
Ciprian

[tibulca]

Changes we consider will be included in the changelog:

1. API lifecycle updates: release, deprecate, sunset (remove)
2. request parameter updates, such as:
   • Add or remove a parameter
   • Update to a parameter type (path / query / header)
   • Required flag change
   • Param schema change:
     • Type
     • Default value
     • Format
     • Pattern, minimum, maximum, minLength, maxLenght
     • Enum updates: add, remove items
3. Request body updates:
   • Required flag change
   • Media type:
     • Add / remove
     • Schema updates (includes nested schemas):
       • Schema type change
       • Add, remove field
       • Update required field list (add / remove)
       • Change field type
       • Change readOnly, writeOnly flags
       • Default field value
       • Field format
       • Field pattern, minimum, maximum, minLength, maxLenght, nullable
       • Enum updates: add, remove items
       • AnyOf, OneOf, AllOf
       • Discriminator updates:
         • Add / delete / changed (without a description of the change)
4. Response updates:
   • Status code updates: added, removed
   • Content type updates:
     • Add, remove
     • Schema updates: same as schema updates from request body.
   • operationId updates
   • Resource tag updates
   • Resource security updates
     • Type
     • Config updates: flow, authorizationUrl, tokenUrl, scopes

Changes we consider will not be included in the changelog:

1. Description, examples, summary, external docs, OpenAPI extensions, encoding - for all change categories (resource, param, response etc.)
2. Global API tags
3. API usage info: license, description, title, terms of service
4. Servers

[reuvenharrison]

Hi @tibulca,
Your suggestion sounds interesting.
Before discussing technical/implementation details, I'd like to clarify the use-case.
Who is the user and what problem does the change-log solve for this user?
For example, in the case of Breaking-Changes, I would argue that the user is the developer or a manager that wants to ensure business continuity.

Thanks,
Reuven

[tibulca]

Hi @reuvenharrison,

The target is any user of the API (including programmatic users, not only devs / TLs / managers), and at high-level, these are some goals we're trying to achieve:

• clearly communicates new features, changes in the API contracts, and bugfixes
• help users to understand the upgrade path.
• know when endpoints are deprecated and marked for removal.

We did some investigations about other API changelogs and some generators. We found that the concise output format of the new checkers is precisely what we want for the changelog too. Example: removed the required property 'name' from the response with the '200' status. Using the checkers will also ensure that we can correctly tag every detected change as breaking / non-breaking.

[reuvenharrison]

Thanks @tibulca.
This sounds like a promising feature that can be valuable to many OpenAPI users.
I am going to sketch a document that describes the personas, the business problem(s), the business process(es) and the tools (like change log) that we would like to offer.
I welcome you to be part of the design team and hopefully we will get some others to join too.

[blva]

Hi. I’ll be working together with @tibulca on this feature. We’re eager to get started w/ a technical proposal, perhaps a PoC PR for you and your team to review @reuvenharrison. Would that be a good start? Thanks.

[reuvenharrison]

@blva, @tibulca and anyone else who is interested,
Here is an overview of the business problem and the possible solutions, including a change log.
Your feedback is welcome.
https://docs.google.com/document/d/1deIFfzROEy2ayAvSZXtCW1q3QQs_FqbMER56NNxO6eM/edit?usp=sharing

[reuvenharrison]

Hi @blva and @tibulca,
Let's go ahead with your suggestion to extend the -check-breaking checkers with an INFO level to provide a change log.
Thanks,
Reuven