Removing the exclude-elements option from changelog and breaking commands

[reuvenharrison]

Hi oasdiff users,
Version v1.10.13 removed support for the --exclude-elements flag for changelog and breaking commands.
I'd like to share the background for this change as some of you have reported it as a problem.

The trigger for this change is PR 515 which added diff support for OpenAPI Extensions which are used for AWS gateways and a few other use cases.
As part of this enhancement, I wanted to maintain backwards compatibility to allow users to exclude OAS Extension changes from the diff output, as was the previous behavior, so I added this option: exclude-elements=extensions.

While this flag makes sense with oasdiff diff, it has a side-effect on oasdiff changelog and oasdiff breaking.
When running oasdiff changelog or oasdiff breaking with exclude-elements=extensions the checks that depend on OAS Extensions break:

1. API deprecation checks which depend on the x-sunset
2. Stability-level checks that depend on x-stability-level
3. Extensible enum checks which depend on x-extensible-enum

I think this is a bad design because it requires the user to understand an internal implementation detail: how changelog and breaking-changes rely on diff.
Moreover, I realized that excluding elements doesn’t change the output of any changelog or breaking-change checks except for disabling the checks that rely on OAS extensions. In other words, there is no use-case that requires to use this flag with changelog or breaking-change.
I concluded that that this flag should be removed from the changelog and breaking commands.

I hope this clarifies things and I am looking forward for your feedback.
Regards,
Reuven

[anikitin]

HI @reuvenharrison , I made some experiments after reading your post, and here are my observations.

At the initial phase of oasdiff exploration I played with different commands and tried to figure out what report format would be the most convenient for our users. And, choosing between "diff" and "changelog" I opted for the latter one. I was under the impression that both commands actually report about the same changes but in different formats. Now I understand it is not the case and, you are right, "changelog" (probably) ignores the changes which do not affect the API contract. In this case "--exclude-elements" definitely does not make any difference for this command.

So my overall feeling is that oasdiff documentation lacks the clarity defining real differences between "changelog" and "diff".
And also it is not clear in describing what options work for what commands.

I also think that "breaking" command could be excessive, I would better add some parameter to "diff" to define output severity threshold
(for example, the famous Stoplight Spectral OpenAPI linter has the option "-D, --display-only-failures only output results equal to or greater than --fail-severity" - see https://meta.stoplight.io/docs/spectral/9ffa04e052cc1-spectral-cli).

So after realizing the difference between "changelog" and "diff" I am OK to stop using --exclude-elements for my use case.
At the same time, I feel that for some "changelog" users seeing the INFO-level changes in textual elements could be helpful in changelog as well.

[reuvenharrison]

Thanks for your feedback @anikitin

Perhaps we can add a paragraph in the readme to explain the difference between 'diff' and 'changelog'.

Meanwhile the options for each command can be seen by running 'oasdiff help'.

Regarding the redundancy of 'changelog' and 'breaking' commands, I think it makes sense to combine them.