Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

autodoc: signature of __new__ overwrites signature of __init__ #13257

Open
hrittich opened this issue Jan 21, 2025 · 5 comments
Open

autodoc: signature of __new__ overwrites signature of __init__ #13257

hrittich opened this issue Jan 21, 2025 · 5 comments

Comments

@hrittich
Copy link

Describe the bug

The __new__ method of a class is expected to accept the same arguments as the __init__ method. Since a __new__ method for a class also has to work for all derived classes, the __new__ method usually has the signature:

__new__(cls, *args, **kwargs)

Nevertheless, when constructing the class, the arguments of the __init__ method must be provided.

When using autodoc with autodoc_class_signature = 'mixed' for a class with an __init__ and a __new__ method, the signature of the __new__ method is show in the class signature. However, I would expect that the function signature of __init__ is shown.

How to Reproduce

An MWE repository is provided here: hrittich/sphinx-signature-bug-mwe.

conf.py

project = 'MWE Package'
copyright = '2025, N.A.'
author = 'N.A.'

# -- General configuration ---------------------------------------------------

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.napoleon'
]

templates_path = ['_templates']
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# -- Autodoc configuration ---------------------------------------------------

autodoc_class_signature = 'mixed'

# -- Options for HTML output -------------------------------------------------

html_theme = 'alabaster'
html_static_path = ['_static']

index.rst

MWE Package documentation
=========================

.. toctree::
   :maxdepth: 2
   :caption: Contents:


.. automodule:: mwe_package
   :members:

mwe_package.py

"""MWE Package."""

class Foo:
    """The Foo class.

    Args:
        x (int): X.
        y (int): Y.
    """

    def __init__(self, x: int, y: int):
        print(f'x={x}, y={y}')

    def __new__(cls, *args, **kwargs):
        print(f'NEW: args={args}, kwargs={kwargs}')
        return super().__new__(cls)

Renders the documentation as

class mwe_package.Foo(*args, **kwargs)
    The Foo class.

instead of

class mwe_package.Foo(x: int, y: int)
    The Foo class.

Environment Information

Platform:              linux; (Linux-6.1.0-30-amd64-x86_64-with-glibc2.36)
Python version:        3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0])
Python implementation: CPython
Sphinx version:        8.1.3
Docutils version:      0.21.2
Jinja2 version:        3.1.5
Pygments version:      2.19.1

Sphinx extensions

extensions = ['sphinx.ext.autodoc']

Additional context

No response

@electric-coder
Copy link

electric-coder commented Jan 21, 2025

The way autodoc_class_signature is implemented with only 2 options either __init__ or __new__ had to be chosen. The two possible workarounds are:

  1. Writing the desired signature explicitly into the .rst that brings the usual maintainability problem.
  2. Implementing a custom autodoc-process-signature event handler in conf.py to override how specific signatures are supposed to be handled.

Both options are viable and already introduce lots of flexibility, with the first one being especially beginner friendly.

You're proposing to use __init__ rather than __new__ when it's available (and your argument makes sense in that __init__ will tend to be more specific), but there'll be other users complaining of the opposite. Also notice that autodoc_class_signature is a autodoc-wide config, so choosing one or the other neglects that users may want to choose depending on what class they're documenting. The only solution that would satisfy everyone is propagating the choice of constructor as an option down to the .. autoclass:: directive (and that's what the two workarounds are for).

@hrittich
Copy link
Author

hrittich commented Jan 21, 2025

I am using autosummary as well, so overwriting inside an .rst file is not really an option.

I have looked at autodoc-process-signature and it is not immediately clear to me, how to obtain the functionality that I want. Having an easy way of influencing this behavior, e.g., by a conf.py option would be appreciated.

I think, having this option especially makes sense, when you say that there is no right way for all users.

@electric-coder
Copy link

electric-coder commented Jan 21, 2025

when you say that there is no right way for all users.

I usually start out by being skeptical of reports in the Sphinx repo but you do have a strong case here.

I am using autosummary as well, so overwriting inside an .rst file is not really an option.

This kind of extra difficulty in customizing small problems is probably what leads a lot of devs to switch to the more complicated autodoc.

I have looked at autodoc-process-signature and it is not immediately clear to me

I had forgotten about this workaround:

  1. I'm not sure if overriding the signature in the first line of the docstring is a valid workaround for autosummary - here's a good example.

If that doesn't work, I'm also not sure if the autodoc-process-signature event handler interoperates with autosummary (likely it's autodoc specific, but I don't know) a good example is given in this post but to get it to work it's highly advisable to build the docs with a debugger, so here's a good example of howto configure the PyCharm debugger to build Sphinx. If you've made it this far, you understand why a lot of folks prefer migrating to autodoc somewhere around this point.

@hrittich
Copy link
Author

Thank you for the workaround suggestions. I will have a look at them. For the time being I have switched to:

autodoc_class_signature = "separated"

This kind of extra difficulty in customizing small problems is probably what leads a lot of devs to switch to the more complicated autodoc.

Yeah. I can see that. I have thought about that a few time, but this decision is not entirely up to me.

I wanted to provide a bit more context. I encountered the issue when creating a derived class from the SciPy LinearOperator class, which is the documented way of implementing the linear operator interface. The LinearOperator defines a __new__ method and an __init__ method, while my derived class defines only defines an __init__ method, and in this case Sphinx shows the signature of the __new__ method of the parent class, instead of the signature of the __init__ method of the derived class. Hence, maybe giving precedence to non-inherited methods would also be an option to improve the issue.

@electric-coder
Copy link

electric-coder commented Jan 22, 2025

I encountered the issue when creating a derived class from the SciPy (...) The LinearOperator defines a new method (...) while my derived class defines only defines an __init__ method

These kind of inheritance problems are known, you can search the Sphinx repo for previous reports. This isn't exactly a bug since the inheritance and documentation mechanisms are working normally as expected.

Hence, maybe giving precedence to non-inherited methods would also be an option to improve the issue.

This does make a lot of sense, but I'm not an autosummary user so I'll abstain from making a judgement if the change should be done. Good report! (I was hoping workaround n#3 in my previous post would have worked out for you).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants