Should protocols allow narrower *args?

[max-muoto]

I'm working on some TypeShed improvements, and noticed one difference in MyPy/Pyright that's making something I'm working on trickier. Take this example:

from typing import Protocol, Any

class MyProtocol(Protocol):
    def my_callable(self, *args: Any) -> None:
        ...
    
    
class MyClass:
    def my_callable(self, arg_1: Any, arg_2: Any) -> None:
        return None
        
        
def foo(proto: MyProtocol) -> None:
    ...
    
foo(MyClass())

Pyright Playground

MyPy Playground

In the case that a protocol accepts arbitrary positional arguments, Pyright required the class implementing the protocol actually type out *args, as opposed to being able to lay out specific types for those positional args, or also have them be keyword arguments.

Curious if this is intentional, or perhaps could be considered a bug? Personally, I'm a bit surprised this would raise a warning.

[erictraut]

This is a bug in mypy. In this example, it MyClass is not compatible with MyProtocol. It's not type safe to allow this, and it doesn't follow from the standardized subtyping rules for callables.

Consider if my_callable were implemented as follows:

def foo(proto: MyProtocol) -> None:
    proto.my_callable(1, 2, 3, 4)

Your program would now crash.

[max-muoto]

Is there a way in Pyright you can allow an implemented protocol to accept any variable amount of arguments? It doesn't sound like it then, which would make sense, but just wanted to confirm. One strategy I saw attempted (which doesn't work) was something like this:

class MyProtocol(Protocol):
   my_callable: Callable[..., object]

[erictraut]

I'm not sure what you mean. A class (implementation) that is compatible with a protocol must accept all of the same combinations of arguments that the protocol accepts. If the protocol accepts an arbitrary number of arguments (i.e. *args), then an implementation of that protocol must also accept an arbitrary number of arguments. Otherwise it doesn't implement the protocol.

You can create a protocol that is parameterized by a TypeVarTuple. Perhaps that helps you with your use case?

from typing import Protocol

class MyProtocol[*Ts](Protocol):
    def my_callable(self, *args: *tuple[*Ts]) -> None: ...

class MyClass:
    def my_callable(self, arg_1: int, arg_2: int) -> None:
        return None

def foo(proto: MyProtocol[int, int]) -> None:
    proto.my_callable(1, 2)

foo(MyClass())

[max-muoto]

Thanks! In this case, I think was mainly hoping to find a nice way around the fact that I was fine with potential runtime errors, in the same sense that annotating a Callable with ... is sort of a free for all.

Curious, it definitely falls under the same category I think we're talking about, where you'll possible encounter a runtime error due to the lack of specific typing info. But would something like Unpack[tuple[T, ...]] work in regards to protocols?

from __future__ import annotations


from typing import Any, Protocol, TypeVar, TypeVarTuple, Unpack

T = TypeVar("T", covariant=True)
Ts = TypeVarTuple("Ts")


class Unpackable(Protocol[T, Unpack[Ts]]):
    def unpack(self, *args: Unpack[Ts]) -> None:
        ...

    def retrieve(self) -> T:
        ...


# Is it even possible to provide an implementation that suffices this?
# If so, I can't seem to figure out what that would be.
def foo(func: Unpackable[T, T, Unpack[tuple[T, ...]]]):
    ...


class UnpackableImpl:
    def unpack(self, arg_1: int, arg_2: int) -> None:
        ...

    def retrieve(self) -> int:
        ...


foo(UnpackableImpl()) # UnpackableImpl" is incompatible with protocol "Unpackable[int, int, *tuple[int, ...]]"

As let's say you had concrete implementations, then this would be fine:

from __future__ import annotations


from typing import Any, Generic, Protocol, TypeVar, TypeVarTuple, Unpack

T = TypeVar("T", covariant=True)
Ts = TypeVarTuple("Ts")


class Unpackable(Generic[T, Unpack[Ts]]):
    def unpack(self, *args: Unpack[Ts]) -> None:
        ...

    def retrieve(self) -> T:
        ...


def foo(func: Unpackable[T, T, Unpack[tuple[T, ...]]]):
    ...


z: Unpackable[int, int] = Unpackable()
foo(z)

[max-muoto]

I basically want to have stubs where it's overloaded so that the first overload enforces you only pass in one argument, but for the second overload it's any n arguments as long as its consistent within the implemented class itself. Obviously this isn't the most type-safe, but in this case it's basically the most accurate the stubs are going to get.