WIP: Add type annotations to base spec

[edgarrmondragon]

See #625 (comment)

[martindurant]

Thanks for giving this a start. Perhaps you already appreciate that this process will not be easy for the package, and spec might have been the hardest place to start (since all these methods are overridden in various duck ways in the backend implementations).

[mon]

Preface: Just a random user.

This is a really good start (I love me some types), though I feel like it could do with some minor improvements

• a few object annotations that could probably be narrowed to a Protocol
• Inconsistent typing for inferred kwargs (explicitly specified recursive: bool = False vs implied on_error="omit")
• open could do with an overload to differentiate binary and text streams

...however, I feel like it could just devolve into a bunch of minor nitpicks, whereas this already provides lots of value as-is.

What's the strategy towards a merge here? Grab a bunch of popular downstream consumers and ensure these don't cause conflicts with any overloaded duck typing happening there?

[martindurant]

@mon : as with my initial comments, I am happy to see typing leak slowly into fsspec, but the duck-type design of the (old!) code means that it will be hard to have it everywhere and certainly very hard to get mypy happy throughout the codebase, including downstream implementations.

It gets complicated fast, because of issues like people demanding support for Path instances where originally only strings were allowed. The end-user API is based on convenience, but library developers using fsspec want strict definitions - requirements that are hard to simultaneously meet.

What's the strategy towards a merge here? Grab a bunch of popular downstream consumers and ensure these don't cause conflicts with any overloaded duck typing happening there?

It shouldn't be possible to break code with types alone, right?
However, I need people more experiences with type-ifying to review the code. It is not for my convenience, but for others, so it will not be high on my priority list.

[mon]

It shouldn't be possible to break code with types alone, right?

At runtime, there will be no effect, unless you type annotate forward/circular references. But these are found as soon as the module is imported, so shouldn't ever be a problem for you, only the person adding the types (and easily fixed by gating the import behind an if TYPE_CHECKING branch).

My worry was there might just be some downstream projects that, for example, extend the valid argument types for some functions, and making them specifically typed (instead of the default Unknown) will cause type errors from tools like Pylance. Personally I don't think that's an issue, and imperfect types is better than none.