Should we consider recommending drf-spectacular?

[carltongibson]

The built-in schema generation is OK to get started, but OpenAPI is a big beast, and it would need a lot more time than it's had, or is ever likely to have, in order to fill-in all the edge-cases.

Meanwhile over at drf-spectacular @tfranzel is doing an amazing job.

• Tfranzel/Drf-Spectacular: Sane and Flexible OpenAPI 3 Schema Generation for Django REST Framework.
• Drf-Spectacular — Drf-Spectacular Documentation

With an INSTALLED_APPS, It hooks into schema generation by providing an AutoSchema subclass (exactly as intended)...

REST_FRAMEWORK = {
    # YOUR SETTINGS
    'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
}

... and then goes on to provide the numerous customisation points that we're never going to match in core.

I think we should (consider to) make drf-spectacular the recommended way to go.

To begin a note at the top of the schema (and Documenting your API) docs — saying Hey, head-over here!

Then, the whole rest_framework.schemas package was structured to be swappable. There's just the schema attribute on APIView, the base inspector and generator stubs in rest_framework.schemas, and the DEFAULT_SCHEMA_CLASS — I think that's it — the rest could easily be extracted to a third-party package, so usable with minimal settings/import changes, and then deprecated.

Doing so would allow a focus on the viable package in this space, and significantly reduce the maintenance overhead on DRF, in an area where it's never quite had the resource needed to make it sing.

What do you think?

[n2ygk]

@carltongibson I took a quick look and have some concerns around the ability to extend DRF to add Security Schemes and Security Objects which I've addressed in PR #7516 (and you liked 2 years ago) so that I can use those extended schemes to add OAuth2 security schemes and objects in django-oauth-toolkit's downstream extensions of DRF's BaseAuthentication and BasePermission classes.

I imagine drf-spectacular could also leverage these hooks.

Here's an example of how I've had to (poorly) work around this gap.

Given the long wait and this PR getting bumped from 3.12 to 3.13 to 3.14 I was losing hope that this would ever happen. I suppose I could kludge up DOT to have configurable DRF schemas that it extends. That would be pretty ugly but sadly it's sounding like that's DRF's direction. Say it ain't so!

[carltongibson]

I commented on #7516 — but in general I think additional hooks should be added (in e.g. drf-spectacular, but anywhere) using decorators and such, rather than expanding the public API on DRF objects.

There was similar discussion on django-filter — it's much better that the schema-generating code keeps hold of it's own logic, rather than adding (likely empty or minimal) hooks in a separate project that then requires cross-project release coordination.

In the end the runtime object may end up with the same methods, but the API documentation, testing, evolution and maintenance remains coherently in the Schema concerned package.

I hope that makes sense.

[n2ygk]

Unfortunately not. Can you point me to an example of how this might work? I just don't understand.

[carltongibson]

Well here for example...

https://github.com/tfranzel/drf-spectacular/blob/master/drf_spectacular/authentication.py

DRF-spectacular is defining classes used for authentication types that encapsulate the security schema data, rather than DRF having a hook for that itself.

[n2ygk]

Oh, OK. I missed that. Still looks like a significant amount of rework for DJA to use drf-spectacular.

[carltongibson]

Sure. That's hard to say without looking at it. ...

You'd be free to keep using the current implementation. You could even extend it as you wished in a fork. (It won't stop working.)

[tfranzel]

First of all, thank you very much for your kind words. It has been a lot of work that started simply out of desperation for a usable solution. Now I'm just enjoying that fact that it's used quite successfully in the wild. You could say I already was a pro user before, but writing this tool made me learn details about Django and DRF that I neither knew existed nor that I really ever wanted to know honestly.

@carltongibson regarding a recommendation. Very much appreciated and I also think it is a good time. drf-spectacular has been very stable and I consider it mature at this point in time. We solved hundreds of issues and most new legitimate bugs are super esoteric edge cases that rarely anyone ever hits. The steady decline in bug reporting in conjuction with the amount of users tells me that we reached a nice plateau of stability. That also means the API is not expected to change in any significant way anymore (and has not for quite some time now).

regarding deprecating parts of the DRF code: It is not a issue for us. Do as you see fit. We have very little hard dependencies left. As you said, we depend on the mechanics that were provided to us. I also want to stress that those mechanics have way more hidden complexity behind them than people realize. It's a good and stable API and as long as that is not gonna change in a significant way, we are good! Code wise, we probably have zero lines dependencies in AutoSchema, a few lines in EndpointEnumerator, and some lines in SchemaGenerator. Pulling those few functions into our codebase is trivial, we would just need the heads-up on what and when.

Side-mark: what about core-api and "The Browsable API"? Is this still adding value to anyone? It was nice to have when nothing else was available, but nowadays using OpenAPI/SwaggerUI/drf-spectacular makes those old approaches seem quite antiquated with very little added benefit. Personally, I do love SwaggerUI! It's a fantastic well-maintained tool with a strong community. DRF is never gonna match this, so why keep reinventing a subpar wheel. I absolutely agree with you @carltongibson that it's better for DRF to focus on the core and stand on the shoulders of giants when it comes to those auxiliary topics.

Regarding the discussion where the schema code should live. Well, it's the old framework or library question. Here are some thought on the nitty gritty:

• DESIGN & RESPONSIBLITY & KNOWLEDGE: From a design perspective, the schema code should probably live in the library (e.g. django-filter), but in reality it is not that clear cut. Yes, maintainers should know how their code is supposed to behave, but that does not mean they know how to properly bake that into a OpenAPI specification.
• TOOLING: Some libraries require significantly more logic than DRF offers at the moment. drf-spectacular does offer those amenities, but it would create a dependency. For example, deriving model field specs is complicated and package maintainers will not write hundreds of complicated lines for that alone. I suppose maintainers will not even bother without those toolings available to them. Also writing those schema methods correctly is not an easy feat. Writing the auth definition for the bearer auth took me like 5 attempts (with the help of a handful of people) to finally get right. It's harder than it looks.
• COMMUNITY SUPPORT: Also some of the heavily used libraries are not that well maintained, un-maintained, or the maintainer is very defensive when it comes to changing things (looking at you @carltongibson 😄). Relying on their cooperation alone would make this whole thing dead on arrival. drf-spectacular would have never gained traction, if I have had to convince 20 different maintainers on why we need some particular change time and again. Realistically, this is just never going to happen in a satisfactory way. The drf-spectacular extension system was designed to make this a flexible two way street for both package maintainers and users with special needs. This really made the difference imho.

@n2ygk regarding Django OAuth Toolkit (don't know what DJA stands for):

https://github.com/tfranzel/drf-spectacular/blob/master/drf_spectacular/contrib/django_oauth_toolkit.py

We do ship with that support out-of-the-box and we do make use of what we can with BaseAuthentication and BasePermission.

Essentially, we did the same thing as you, but made it so that it's more pluggable and easier to work with. Please let me know if there is some gap for you there. I realized some time ago that this whole thing works so ridiculously automagically that people sometimes stumble over that simplicity on first contact.

This also is a good example for the above TOOLING and KNOWLEDGE points. Afaik this extension works quite well for us. Let me know if there is anything missing for you. Sry for the effort you put into that PR, but maybe we can cherry-pick some aspects if we missed something on our side.

Please let me know what you think @carltongibson @n2ygk! It's a big interconnected set of issue and writing down all my learnings from the past 2 years is quite hard, but I'll happily expand where needed.

[tfranzel]

awesome @tomchristie! I will be unavailable the next 2 weeks but after that I can give it shot.

The Browsable API I think is still hugely beneficial yup.

Okay that is a bit surprising to me. When I started using drf-yasg (and later spectacular), I never opened The Browsable API again. In comparison it is quite limited, but maybe you guys use it differently than me.

[tfranzel]

Sorry that this took so long. Here is a PR for changing the doc #8773. Feedback welcome. I was a bit unsure how to include this in a sensible way without changing too much.

Some more points:

core-api isn't adding any value, no. We probably want to deprecate this page of documentation at some point?

I'm only vaguely familiar with this, but looking at https://www.coreapi.org it seems that coreapi can also be used with an OpenAPI schema. So this page is does not seem to be completely superceded. Any mention of the core-api generator would be though.

Regarding @carltongibson point on factoring out the deprecated parts into a separate package. I can prepare a PR for that, but the details seem a bit hairy.

1. We would need something like encode/djangorestframework-legacy-schema as an official repo/package to make this work I suppose.
2. Then do some "facade imports" that make sure users are not broken by changed imports for now.
3. Should the affected tests move too?
4. This would also mean that DRF would gain an additional dependency as long as this is supposed to stay transparent.

Let me know how you would prefer to have this, and I'll cook up another PR.

[tomchristie]

I'm only vaguely familiar with this, but looking at https://www.coreapi.org/ it seems that coreapi can also be used with an OpenAPI schema. So this page is does not seem to be completely superceded. Any mention of the core-api generator would be though.

I think the initial action on this particular point is dropping mentions of coreapi from the REST framework docs.

[tfranzel]

@tomchristie will do. I found 2 remaining mentions in pagination and filtering. Also, do you also want the deprecated and hidden coreapi docs removed? They are still built but are not hyperlinked anywhere except for the 3.10 release notes. Fixing that GH link to a commit instead of blob/master would do the trick.

I will prepare another PR with your input.

[tomchristie]

Also, do you also want the deprecated and hidden coreapi docs removed? They are still built but are not hyperlinked anywhere except for the 3.10 release notes. Fixing that GH link to a commit instead of blob/master would do the trick.

I think we can do that now, yup.