diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml
index e37e799705..44dbc60140 100644
--- a/.github/workflows/run-tests.yml
+++ b/.github/workflows/run-tests.yml
@@ -37,6 +37,7 @@ jobs:
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: ${{ matrix.python-version }}
+ allow-prereleases: true
cache: 'pip'
- name: Install dependencies
run: |
diff --git a/HISTORY.md b/HISTORY.md
index d536440ddc..29df37445a 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -1,11 +1,29 @@
Release History
===============
-3.7.1 (2024-06-??)
+3.7.1 (2024-07-07)
------------------
+**Added**
+- Official support for Python 3.13
+ This has been tested outside GitHub CI due to httpbin unready state for 3.13[...]
+- Support for asynchronous auth callables.
+- Support for asynchronous bodies through `AsyncIterable` that yield either bytes or str.
+- Support for purposely excluding a domain/port from connecting to QUIC/HTTP3 via the `quic_cache_layer` property of `Session`.
+ In order to exclude `cloudflare.com` from HTTP3 auto-upgrade:
+ ```python
+ from niquests import Session
+
+ s = Session()
+ s.quic_cache_layer.exclude_domain("cloudflare.com")
+ ```
+
**Fixed**
- auth argument not accepting a function according to static type checkers. (#133)
+- RequestsCookieJar having a lock in `AsyncSession`. Its effect has been nullified to improve performances.
+
+**Changed**
+- urllib3-future lower bound version is raised to 2.8.902
3.7.0 (2024-06-24)
------------------
diff --git a/README.md b/README.md
index be51c5f705..8fc8605626 100644
--- a/README.md
+++ b/README.md
@@ -12,34 +12,34 @@ Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanc
👆 Look at the feature table comparison against requests, httpx and aiohttp!
-| Feature | niquests | requests | httpx | aiohttp |
-|-------------------------------------|:--------:|:---------:|:-------------:|---------------|
-| `HTTP/1.1` | ✅ | ✅ | ✅ | ✅ |
-| `HTTP/2` | ✅ | ❌ | ✅[^7] | ❌ |
-| `HTTP/3 over QUIC` | ✅ | ❌ | ❌ | ❌ |
-| `Synchronous` | ✅ | ✅ | ✅ | ❌ |
-| `Asynchronous` | ✅ | ❌ | ✅ | ✅ |
-| `Thread Safe` | ✅ | ✅ | ❌[^5] | _N/A_[^1] |
-| `Task Safe` | ✅ | _N/A_[^2] | ✅ | ✅ |
-| `OS Trust Store` | ✅ | ❌ | ❌ | ❌ |
-| `Multiplexing` | ✅ | ❌ | _Limited_[^3] | ❌ |
-| `DNSSEC` | ✅[^11] | ❌ | ❌ | ❌ |
-| `Customizable DNS Resolution` | ✅ | ❌ | ❌ | ✅ |
-| `DNS over HTTPS` | ✅ | ❌ | ❌ | ❌ |
-| `DNS over QUIC` | ✅ | ❌ | ❌ | ❌ |
-| `DNS over TLS` | ✅ | ❌ | ❌ | ❌ |
-| `Multiple DNS Resolver` | ✅ | ❌ | ❌ | ❌ |
-| `Network Fine Tuning & Inspect` | ✅ | ❌ | _Limited_[^6] | _Limited_[^6] |
-| `Certificate Revocation Protection` | ✅ | ❌ | ❌ | ❌ |
-| `Session Persistence` | ✅ | ✅ | ✅ | ✅ |
-| `In-memory Certificate CA & mTLS` | ✅ | ❌ | _Limited_[^4] | _Limited_[^4] |
-| `SOCKS 4/5 Proxies` | ✅ | ✅ | ✅ | ❌ |
-| `HTTP/HTTPS Proxies` | ✅ | ✅ | ✅ | ✅ |
-| `TLS-in-TLS Support` | ✅ | ✅ | ✅ | ✅ |
-| `Direct HTTP/3 Negotiation` | ✅[^9] | N/A[^8] | N/A[^8] | N/A[^8] |
-| `Happy Eyeballs` | ✅ | ❌ | ❌ | ✅ |
-| `Package / SLSA Signed` | ✅ | ❌ | ❌ | ✅ |
-| `HTTP/2 with prior knowledge (h2c)` | ✅ | ❌ | ✅ | ❌ |
+| Feature | niquests | requests | httpx | aiohttp |
+|--------------------------------------------------------------------------|:-------------------------:|:----------------------------------:|:-----------------------------:|----------------------|
+| `HTTP/1.1` | ✅ | ✅ | ✅ | ✅ |
+| `HTTP/2` | ✅ | ❌ | ✅[^7] | ❌ |
+| `HTTP/3 over QUIC` | ✅ | ❌ | ❌ | ❌ |
+| `Synchronous` | ✅ | ✅ | ✅ | ❌ |
+| `Asynchronous` | ✅ | ❌ | ✅ | ✅ |
+| `Thread Safe` | ✅ | ✅ | ❌[^5] | _N/A_[^1] |
+| `Task Safe` | ✅ | _N/A_[^2] | ✅ | ✅ |
+| `OS Trust Store` | ✅ | ❌ | ❌ | ❌ |
+| `Multiplexing` | ✅ | ❌ | _Limited_[^3] | ❌ |
+| `DNSSEC` | ✅[^11] | ❌ | ❌ | ❌ |
+| `Customizable DNS Resolution` | ✅ | ❌ | ❌ | ✅ |
+| `DNS over HTTPS` | ✅ | ❌ | ❌ | ❌ |
+| `DNS over QUIC` | ✅ | ❌ | ❌ | ❌ |
+| `DNS over TLS` | ✅ | ❌ | ❌ | ❌ |
+| `Multiple DNS Resolver` | ✅ | ❌ | ❌ | ❌ |
+| `Network Fine Tuning & Inspect` | ✅ | ❌ | _Limited_[^6] | _Limited_[^6] |
+| `Certificate Revocation Protection` | ✅ | ❌ | ❌ | ❌ |
+| `Session Persistence` | ✅ | ✅ | ✅ | ✅ |
+| `In-memory Certificate CA & mTLS` | ✅ | ❌ | _Limited_[^4] | _Limited_[^4] |
+| `SOCKS 4/5 Proxies` | ✅ | ✅ | ✅ | ❌ |
+| `HTTP/HTTPS Proxies` | ✅ | ✅ | ✅ | ✅ |
+| `TLS-in-TLS Support` | ✅ | ✅ | ✅ | ✅ |
+| `Direct HTTP/3 Negotiation` | ✅[^9] | N/A[^8] | N/A[^8] | N/A[^8] |
+| `Happy Eyeballs` | ✅ | ❌ | ❌ | ✅ |
+| `Package / SLSA Signed` | ✅ | ❌ | ❌ | ✅ |
+| `HTTP/2 with prior knowledge (h2c)` | ✅ | ❌ | ✅ | ❌ |
diff --git a/docs/user/advanced.rst b/docs/user/advanced.rst
index 74abe50fd2..af25ef4826 100644
--- a/docs/user/advanced.rst
+++ b/docs/user/advanced.rst
@@ -411,6 +411,7 @@ case you can iterate chunk-by-chunk by calling ``iter_content`` with a ``chunk_s
parameter of ``None``. If you want to set a maximum size of the chunk,
you can set a ``chunk_size`` parameter to any integer.
+.. note:: Since Niquests v3.7.1+ we support having async iterable passed down to ``data=...`` via your ``AsyncSession``.
.. _multipart:
@@ -1280,6 +1281,20 @@ over QUIC.
.. note:: Using a custom DNS resolver can solve the problem as we can probe the HTTPS record for the given hostname and connect directly using HTTP/3 over QUIC.
+Prevent a domain from auto-upgrading to HTTP/3
+----------------------------------------------
+
+In immediate opposition to the previous section::
+
+ from niquests import Session
+
+ s = Session()
+ s.quic_cache_layer.exclude_domain("cloudflare.com")
+
+This will prevent the auto-upgrade to HTTP/3 via the Alt-Svc headers.
+
+.. note:: This is most useful for people that encounter a server that yield its support for HTTP/3 while not able to. This permit to isolate the bad server instead of disabling HTTP/3 session-wide.
+
Increase the default Alt-Svc cache size
---------------------------------------
diff --git a/noxfile.py b/noxfile.py
index 8ce90143b8..18e197c9d2 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -47,7 +47,7 @@ def tests_impl(
)
-@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "pypy"])
+@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "pypy"])
def test(session: nox.Session) -> None:
tests_impl(session)
diff --git a/pyproject.toml b/pyproject.toml
index cafccbe10b..1bc968e675 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -13,7 +13,7 @@ authors = [
{name = "Kenneth Reitz", email = "me@kennethreitz.org"}
]
maintainers = [
- {name = "Ahmed R. TAHRI", email="ahmed.tahri@cloudnursery.dev"},
+ {name = "Ahmed R. TAHRI", email="tahri.ahmed@proton.me"},
]
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -30,6 +30,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy",
diff --git a/src/niquests/__version__.py b/src/niquests/__version__.py
index 7c56e9db38..09bc11c6b5 100644
--- a/src/niquests/__version__.py
+++ b/src/niquests/__version__.py
@@ -9,9 +9,9 @@
__url__: str = "https://niquests.readthedocs.io"
__version__: str
-__version__ = "3.7.0"
+__version__ = "3.7.1"
-__build__: int = 0x030700
+__build__: int = 0x030701
__author__: str = "Kenneth Reitz"
__author_email__: str = "me@kennethreitz.org"
__license__: str = "Apache-2.0"
diff --git a/src/niquests/_async.py b/src/niquests/_async.py
index 1889e51eea..8dfc9b7df8 100644
--- a/src/niquests/_async.py
+++ b/src/niquests/_async.py
@@ -47,6 +47,8 @@
CacheLayerAltSvcType,
RetryType,
AsyncHookType,
+ AsyncBodyType,
+ AsyncHttpAuthenticationType,
)
from .exceptions import (
ChunkedEncodingError,
@@ -218,7 +220,9 @@ def __init__(
#: session. By default it is a
#: :class:`RequestsCookieJar `, but
#: may be any other ``cookielib.CookieJar`` compatible object.
- self.cookies: RequestsCookieJar | CookieJar = cookiejar_from_dict({})
+ self.cookies: RequestsCookieJar | CookieJar = cookiejar_from_dict(
+ {}, thread_free=True
+ )
#: A simple dict that allows us to persist which server support QUIC
#: It is simply forwarded to urllib3.future that handle the caching logic.
@@ -687,11 +691,11 @@ async def request(
method: HttpMethodType,
url: str,
params: QueryParameterType | None = ...,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -708,11 +712,11 @@ async def request(
method: HttpMethodType,
url: str,
params: QueryParameterType | None = ...,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -729,11 +733,11 @@ async def request( # type: ignore[override]
method: HttpMethodType,
url: str,
params: QueryParameterType | None = None,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = WRITE_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -760,10 +764,18 @@ async def request( # type: ignore[override]
hooks=hooks,
)
- prep: PreparedRequest = await async_dispatch_hook(
+ prep: PreparedRequest = self.prepare_request(req)
+
+ if prep._asynchronous_auth:
+ r = await prep._pending_async_auth(prep)
+ prep.__dict__.update(r.__dict__)
+ prep.prepare_content_length(prep.body)
+ del prep._pending_async_auth
+
+ prep = await async_dispatch_hook(
"pre_request",
hooks, # type: ignore[arg-type]
- self.prepare_request(req),
+ prep,
)
assert prep.url is not None
@@ -791,7 +803,7 @@ async def get(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -810,7 +822,7 @@ async def get(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -828,7 +840,7 @@ async def get( # type: ignore[override]
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = READ_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -863,7 +875,7 @@ async def options(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -882,7 +894,7 @@ async def options(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -900,7 +912,7 @@ async def options( # type: ignore[override]
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = READ_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -935,7 +947,7 @@ async def head(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -954,7 +966,7 @@ async def head(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -972,7 +984,7 @@ async def head( # type: ignore[override]
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = READ_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -1003,14 +1015,14 @@ async def head( # type: ignore[override]
async def post(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
json: typing.Any | None = ...,
*,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1024,14 +1036,14 @@ async def post(
async def post(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
json: typing.Any | None = ...,
*,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1044,14 +1056,14 @@ async def post(
async def post( # type: ignore[override]
self,
url: str,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
json: typing.Any | None = None,
*,
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = WRITE_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -1083,14 +1095,14 @@ async def post( # type: ignore[override]
async def put(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
*,
json: typing.Any | None = ...,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1104,14 +1116,14 @@ async def put(
async def put(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
*,
json: typing.Any | None = ...,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1124,14 +1136,14 @@ async def put(
async def put( # type: ignore[override]
self,
url: str,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
*,
json: typing.Any | None = None,
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = WRITE_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -1163,14 +1175,14 @@ async def put( # type: ignore[override]
async def patch(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
*,
json: typing.Any | None = ...,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1184,14 +1196,14 @@ async def patch(
async def patch(
self,
url: str,
- data: BodyType | None = ...,
+ data: BodyType | AsyncBodyType | None = ...,
*,
json: typing.Any | None = ...,
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
files: MultiPartFilesType | MultiPartFilesAltType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1204,14 +1216,14 @@ async def patch(
async def patch( # type: ignore[override]
self,
url: str,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
*,
json: typing.Any | None = None,
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = WRITE_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
@@ -1247,7 +1259,7 @@ async def delete(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1266,7 +1278,7 @@ async def delete(
params: QueryParameterType | None = ...,
headers: HeadersType | None = ...,
cookies: CookiesType | None = ...,
- auth: HttpAuthenticationType | None = ...,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = ...,
timeout: TimeoutType | None = ...,
allow_redirects: bool = ...,
proxies: ProxyType | None = ...,
@@ -1284,7 +1296,7 @@ async def delete( # type: ignore[override]
params: QueryParameterType | None = None,
headers: HeadersType | None = None,
cookies: CookiesType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
timeout: TimeoutType | None = WRITE_DEFAULT_TIMEOUT,
allow_redirects: bool = True,
proxies: ProxyType | None = None,
diff --git a/src/niquests/_typing.py b/src/niquests/_typing.py
index b301e74f82..725d4fca66 100644
--- a/src/niquests/_typing.py
+++ b/src/niquests/_typing.py
@@ -23,7 +23,7 @@
from urllib3_future.contrib.resolver._async import AsyncBaseResolver # type: ignore[assignment]
from urllib3_future.contrib.resolver import BaseResolver # type: ignore[assignment]
-from .auth import AuthBase
+from .auth import AuthBase, AsyncAuthBase
from .structures import CaseInsensitiveDict
if typing.TYPE_CHECKING:
@@ -52,6 +52,10 @@
typing.Iterable[bytes],
typing.Iterable[str],
]
+AsyncBodyType: typing.TypeAlias = typing.Union[
+ typing.AsyncIterable[bytes],
+ typing.AsyncIterable[str],
+]
#: HTTP Headers can be represented through three ways. 1) typical dict, 2) internal insensitive dict, and 3) list of tuple.
HeadersType: typing.TypeAlias = typing.Union[
typing.MutableMapping[typing.Union[str, bytes], typing.Union[str, bytes]],
@@ -92,6 +96,10 @@
AuthBase,
typing.Callable[["PreparedRequest"], "PreparedRequest"],
]
+AsyncHttpAuthenticationType: typing.TypeAlias = typing.Union[
+ AsyncAuthBase,
+ typing.Callable[["PreparedRequest"], typing.Awaitable["PreparedRequest"]],
+]
#: Map for each protocol (http, https) associated proxy to be used.
ProxyType: typing.TypeAlias = typing.Dict[str, str]
diff --git a/src/niquests/auth.py b/src/niquests/auth.py
index def1aa9e57..abb0fcad66 100644
--- a/src/niquests/auth.py
+++ b/src/niquests/auth.py
@@ -19,6 +19,9 @@
from .cookies import extract_cookies_to_jar
from .utils import parse_dict_header
+if typing.TYPE_CHECKING:
+ from .models import PreparedRequest
+
CONTENT_TYPE_FORM_URLENCODED: str = "application/x-www-form-urlencoded"
CONTENT_TYPE_MULTI_PART: str = "multipart/form-data"
@@ -37,10 +40,17 @@ def _basic_auth_str(username: str | bytes, password: str | bytes) -> str:
return authstr
+class AsyncAuthBase:
+ """Base class that all asynchronous auth implementations derive from"""
+
+ async def __call__(self, r: PreparedRequest) -> PreparedRequest:
+ raise NotImplementedError("Auth hooks must be callable.")
+
+
class AuthBase:
- """Base class that all auth implementations derive from"""
+ """Base class that all synchronous auth implementations derive from"""
- def __call__(self, r):
+ def __call__(self, r: PreparedRequest) -> PreparedRequest:
raise NotImplementedError("Auth hooks must be callable.")
diff --git a/src/niquests/cookies.py b/src/niquests/cookies.py
index 9f24195169..8e3ecddf81 100644
--- a/src/niquests/cookies.py
+++ b/src/niquests/cookies.py
@@ -233,8 +233,14 @@ class RequestsCookieJar(cookielib.CookieJar, MutableMapping):
.. warning:: dictionary operations that are normally O(1) may be O(n).
"""
- def __init__(self, policy: cookielib.CookiePolicy | None = None):
+ def __init__(
+ self, policy: cookielib.CookiePolicy | None = None, thread_free: bool = False
+ ):
super().__init__(policy=policy or CookiePolicyLocalhostBypass())
+ if thread_free:
+ from .structures import DummyLock
+
+ self._cookies_lock = DummyLock()
def get(self, name, default=None, domain=None, path=None):
"""Dict-like get() that also supports optional domain and path args in
@@ -467,7 +473,7 @@ def __setstate__(self, state):
"""Unlike a normal CookieJar, this class is pickleable."""
self.__dict__.update(state)
if "_cookies_lock" not in self.__dict__:
- self._cookies_lock = threading.RLock()
+ self._cookies_lock = threading.RLock() # type: ignore[assignment]
def copy(self):
"""Return a copy of this RequestsCookieJar."""
@@ -566,6 +572,7 @@ def cookiejar_from_dict(
cookie_dict: typing.MutableMapping[str, str] | None,
cookiejar: RequestsCookieJar | cookielib.CookieJar | None = None,
overwrite: bool = True,
+ thread_free: bool = False,
) -> RequestsCookieJar | cookielib.CookieJar:
"""Returns a CookieJar from a key/value dictionary.
@@ -575,7 +582,7 @@ def cookiejar_from_dict(
already in the jar with new ones.
"""
if cookiejar is None:
- cookiejar = RequestsCookieJar()
+ cookiejar = RequestsCookieJar(thread_free=thread_free)
if cookie_dict is not None:
names_from_jar = [cookie.name for cookie in cookiejar]
diff --git a/src/niquests/models.py b/src/niquests/models.py
index 9f2b0ff40e..bee9e6cbaf 100644
--- a/src/niquests/models.py
+++ b/src/niquests/models.py
@@ -84,6 +84,8 @@
MultiPartFilesAltType,
MultiPartFilesType,
QueryParameterType,
+ AsyncHttpAuthenticationType,
+ AsyncBodyType,
)
from .auth import BearerTokenAuth, HTTPBasicAuth
from .cookies import (
@@ -196,9 +198,9 @@ def __init__(
url: str | None = None,
headers: HeadersType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
params: QueryParameterType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
cookies: CookiesType | None = None,
hooks: HookType | None = None,
json: typing.Any | None = None,
@@ -301,6 +303,8 @@ class PreparedRequest:
"""
+ _pending_async_auth: AsyncHttpAuthenticationType
+
def __init__(self) -> None:
#: HTTP verb to send to the server.
self.method: HttpMethodType | None = None
@@ -312,7 +316,7 @@ def __init__(self) -> None:
# after prepare_cookies is called
self._cookies: RequestsCookieJar | CookieJar | None = None
#: request body to send to the server.
- self.body: BodyType | None = None
+ self.body: BodyType | AsyncBodyType | None = None
#: dictionary of callback hooks, for internal usage.
self.hooks: HookType[Response | PreparedRequest] = default_hooks()
#: integer denoting starting position of a readable file-like body.
@@ -336,9 +340,9 @@ def prepare(
url: str | None = None,
headers: HeadersType | None = None,
files: MultiPartFilesType | MultiPartFilesAltType | None = None,
- data: BodyType | None = None,
+ data: BodyType | AsyncBodyType | None = None,
params: QueryParameterType | None = None,
- auth: HttpAuthenticationType | None = None,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None = None,
cookies: CookiesType | None = None,
hooks: HookType[Response | PreparedRequest] | None = None,
json: typing.Any | None = None,
@@ -467,7 +471,7 @@ def prepare_headers(self, headers: HeadersType | None) -> None:
def prepare_body(
self,
- data: BodyType | None,
+ data: BodyType | AsyncBodyType | None,
files: MultiPartFilesType | MultiPartFilesAltType | None,
json: typing.Any | None = None,
) -> None:
@@ -479,7 +483,7 @@ def prepare_body(
assert self.headers is not None
# Nottin' on you.
- body: BodyType | None = None
+ body: BodyType | AsyncBodyType | None = None
content_type: str | None = None
enforce_form_data = False
@@ -500,7 +504,7 @@ def prepare_body(
if isinstance(body, str):
body = body.encode("utf-8")
- is_stream = all(
+ is_stream = hasattr(data, "__aiter__") or all(
[
hasattr(data, "__iter__"),
not isinstance(data, (str, list, tuple, Mapping)),
@@ -596,7 +600,7 @@ def prepare_body(
self.body = body
- def prepare_content_length(self, body: BodyType | None) -> None:
+ def prepare_content_length(self, body: BodyType | AsyncBodyType | None) -> None:
"""Prepare Content-Length header based on request method and body"""
assert self.headers is not None
@@ -614,7 +618,11 @@ def prepare_content_length(self, body: BodyType | None) -> None:
# but don't provide one. (i.e. not GET or HEAD)
self.headers["Content-Length"] = "0"
- def prepare_auth(self, auth: HttpAuthenticationType | None, url: str = "") -> None:
+ def prepare_auth(
+ self,
+ auth: HttpAuthenticationType | AsyncHttpAuthenticationType | None,
+ url: str = "",
+ ) -> None:
"""Prepares the given HTTP auth data."""
assert (
@@ -638,9 +646,9 @@ def prepare_auth(self, auth: HttpAuthenticationType | None, url: str = "") -> No
"Unexpected non-callable authentication. Did you pass unsupported tuple to auth argument?"
)
- self._asynchronous_auth = hasattr(
- auth, "__call__"
- ) and asyncio.iscoroutinefunction(auth.__call__)
+ self._asynchronous_auth = (
+ hasattr(auth, "__call__") and asyncio.iscoroutinefunction(auth.__call__)
+ ) or asyncio.iscoroutinefunction(auth)
if not self._asynchronous_auth:
# Allow auth to make its changes.
@@ -651,6 +659,8 @@ def prepare_auth(self, auth: HttpAuthenticationType | None, url: str = "") -> No
# Recompute Content-Length
self.prepare_content_length(self.body)
+ else:
+ self._pending_async_auth = auth # type: ignore[assignment]
def prepare_cookies(self, cookies: CookiesType | None) -> None:
"""Prepares the given HTTP cookie data.
diff --git a/src/niquests/structures.py b/src/niquests/structures.py
index dbd90806b4..8352759425 100644
--- a/src/niquests/structures.py
+++ b/src/niquests/structures.py
@@ -210,6 +210,10 @@ def __getitem__(self, item):
class QuicSharedCache(SharableLimitedDict):
+ def __init__(self, max_size: int | None) -> None:
+ super().__init__(max_size)
+ self._exclusion_store: typing.MutableMapping[typing.Any, typing.Any] = {}
+
def add_domain(
self, host: str, port: int | None = None, alt_port: int | None = None
) -> None:
@@ -219,6 +223,25 @@ def add_domain(
alt_port = port
self[(host, port)] = (host, alt_port)
+ def exclude_domain(
+ self, host: str, port: int | None = None, alt_port: int | None = None
+ ):
+ if port is None:
+ port = 443
+ if alt_port is None:
+ alt_port = port
+ self._exclusion_store[(host, port)] = (host, alt_port)
+
+ def __setitem__(self, key, value):
+ with self._lock:
+ if key in self._exclusion_store:
+ return
+
+ if self._max_size and len(self._store) >= self._max_size:
+ self._store.popitem()
+
+ self._store[key] = value
+
class AsyncQuicSharedCache(QuicSharedCache):
def __init__(self, max_size: int | None) -> None:
diff --git a/tests/test_async.py b/tests/test_async.py
index b3ec3cd016..bf2c396519 100644
--- a/tests/test_async.py
+++ b/tests/test_async.py
@@ -56,6 +56,33 @@ async def emit():
assert all(r.status_code == 200 for r in responses_foo + responses_bar)
+ async def test_with_async_iterable(self):
+ async with AsyncSession() as s:
+
+ async def fake_aiter():
+ await asyncio.sleep(0.01)
+ yield b"foo"
+ await asyncio.sleep(0.01)
+ yield b"bar"
+
+ r = await s.post("https://pie.dev/post", data=fake_aiter())
+
+ assert r.status_code == 200
+ assert r.json()["data"] == "foobar"
+
+ async def test_with_async_auth(self):
+ async with AsyncSession() as s:
+
+ async def fake_aauth(p):
+ await asyncio.sleep(0.01)
+ p.headers["X-Async-Auth"] = "foobar"
+ return p
+
+ r = await s.get("https://pie.dev/get", auth=fake_aauth)
+
+ assert r.status_code == 200
+ assert "X-Async-Auth" in r.json()["headers"]
+
@pytest.mark.usefixtures("requires_wan")
@pytest.mark.asyncio