❇️ Certificate Revocation Protection

.pre-commit-config.yaml

@@ -32,4 +32,4 @@ repos:
   -   id: mypy
       args: [--check-untyped-defs]
       exclude: 'tests/'
-      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.0.934', 'wassima>=1.0.1', 'idna', 'kiss_headers']
+      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.1.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']

HISTORY.md

@@ -1,12 +1,51 @@
 Release History
 ===============
 
-3.0.3 (2023-10-??)
+3.1.0 (2023-10-10)
 ------------------
 
 **Misc**
 - Static typing has been improved to provide a better development experience.
 
+**Added**
+- Certificate revocation verification via the OCSP protocol.
+
+  This feature is broadly available and is enabled by default when `verify=True`.
+  We decided to follow what browsers do by default, so Niquests follows by being non-strict.
+  OCSP responses are expected to arrive in less than 200ms, otherwise ignored (e.g. OCSP is dropped).
+  Niquests keeps in-memory the results until the size exceed 2,048 entries, then an algorithm choose an entry
+  to be deleted (oldest request or the first one that ended in error).
+
+  You can at your own discretion enable strict OCSP checks by passing the environment variable `NIQUESTS_STRICT_OCSP`
+  with anything inside but `0`. In strict mode the maximum delay for response passes from 200ms to 1,000ms and
+  raises an error or explicit warning.
+
+  In non-strict mode, this security measure will be deactivated automatically if your usage is unreasonable.
+  e.g. Making a hundred of requests to a hundred of domains, thus consuming resources that should have been
+  allocated to browser users. This was made available for users with a limited target of domains to get
+  a complementary security measure.
+
+  Unless in strict-mode, the proxy configuration will be respected when given, as long as it specify
+  a plain `http` proxy. This is meant for people who want privacy.
+
+  This feature may not be available if the `cryptography` package is missing from your environment.
+  Verify the availability after Niquests upgrade by running `python -m niquests.help`.
+
+  There is several downside of using OCSP, Niquests knows it. It is not a silver bullet solution. But better than nothing.
+  It does not apply to HTTPS proxies themselves. For now.
+
+- Add property `ocsp_verified` in both `PreparedRequest`, and `Response` to have a clue on the post handshake verification.
+
+  Will be `None` if no verification took place, `True` if the verification leads to a confirmation from the OCSP server
+  that the certificate is valid, `False` otherwise.
+
+**Changed**
+- Bump lower version requirement for `urllib3.future` to 2.1.900 to ensure compatibility with newer features.
+- Internal in-memory QUIC capabilities is now thread safe and limited to 12,288 entries.
+
+**Fixed**
+- `conn_info` was unset if the response came after a redirect.
+
 3.0.2 (2023-10-01)
 ------------------
 

README.md

@@ -2,7 +2,6 @@
 
 **Niquests** is a simple, yet elegant, HTTP library. It is a drop-in replacement for **Requests** that is no longer under
 feature freeze.
-We will support and maintain v2.x.y that only ships with possible minor breaking changes. All breaking changes are issued in the v3.x that should be available as a pre-release on PyPI.
 
 Why did we pursue this? We don't have to reinvent the wheel all over again, HTTP client **Requests** is well established and
 really plaisant in its usage. We believe that **Requests** have the most inclusive, and developer friendly interfaces. We
@@ -48,6 +47,7 @@ Niquests is ready for the demands of building robust and reliable HTTP–speakin
 
 - Automatic Content Decompression and Decoding
 - OS truststore by default, no more certifi!
+- OCSP Certificate Revocation Verification
 - Browser-style TLS/SSL Verification
 - Sessions with Cookie Persistence
 - Keep-Alive & Connection Pooling
@@ -62,12 +62,8 @@ Niquests is ready for the demands of building robust and reliable HTTP–speakin
 - SOCKS Proxy Support
 - Connection Timeouts
 - Streaming Downloads
+- HTTP/2 by default
 - HTTP/3 over QUIC
-- HTTP/2
-
-## API Reference and User Guide available on [Read the Docs](https://niquests.readthedocs.io)
-
-[![Read the Docs](https://raw.githubusercontent.com/jawah/niquests/main/ext/ss.png)](https://niquests.readthedocs.io)
 
 ---
 

docs/index.rst

@@ -59,6 +59,7 @@ Niquests is ready for today's web.
 
 - Automatic Content Decompression and Decoding
 - OS truststore by default, no more certifi!
+- OCSP Certificate Revocation Verification
 - Browser-style TLS/SSL Verification
 - Sessions with Cookie Persistence
 - Keep-Alive & Connection Pooling
@@ -73,8 +74,8 @@ Niquests is ready for today's web.
 - SOCKS Proxy Support
 - Connection Timeouts
 - Streaming Downloads
+- HTTP/2 by default
 - HTTP/3 over QUIC
-- HTTP/2
 
 Niquests officially supports Python 3.7+, and runs great on PyPy.
 

docs/user/advanced.rst

@@ -1139,3 +1139,42 @@ It is also possible to use the ``Timeout`` class from ``urllib3`` directly::
     r = niquests.get('https://github.com', timeout=Timeout(3, 9))
 
 .. _`connect()`: https://linux.die.net/man/2/connect
+
+OCSP or Certificate Revocation
+------------------------------
+
+Difficult subject. Short story, when a HTTP client establish a secure connection,
+it verify that the certificate is valid. The problem is that a certificate
+can be both valid and revoked due its immutability, the revocation status must
+be taken from an outside source, most of the revocation are linked to a hack/security violation.
+
+Niquests try to protect you from the evoked problem by doing a post-handshake verification
+using the OCSP protocols via plain HTTP.
+
+Unfortunately, at this moment, no bullet proof solution has emerged against revoked certificate.
+We are aware of this. But still, it is better than nothing!
+
+By default, Niquests operate a soft-fail verification, or non-strict if you prefer.
+
+This feature is broadly available and is enabled by default when ``verify=True``.
+We decided to follow what browsers do by default, so Niquests follows by being non-strict.
+OCSP responses are expected to arrive in less than 200ms, otherwise ignored (e.g. OCSP is dropped).
+Niquests keeps in-memory the results until the size exceed 2,048 entries, then an algorithm choose an entry
+to be deleted (oldest request or the first one that ended in error).
+
+You can at your own discretion enable strict OCSP checks by passing the environment variable ``NIQUESTS_STRICT_OCSP``
+with anything inside but ``0``. In strict mode the maximum delay for response passes from 200ms to 1,000ms and
+raises an error or explicit warning.
+
+In non-strict mode, this security measure will be deactivated automatically if your usage is unreasonable.
+e.g. Making a hundred of requests to a hundred of domains, thus consuming resources that should have been
+allocated to browser users. This was made available for users with a limited target of domains to get
+a complementary security measure.
+
+Unless in strict-mode, the proxy configuration will be respected when given, as long as it specify
+a plain ``http`` proxy. This is meant for people who want privacy.
+
+This feature may not be available if the ``cryptography`` package is missing from your environment.
+Verify the availability by running ``python -m niquests.help``.
+
+.. note:: Access property ``ocsp_verified`` in both ``PreparedRequest``, and ``Response`` to have information about this post handshake verification.

pyproject.toml

@@ -8,7 +8,7 @@ description = "Python HTTP for Humans."
 readme = "README.md"
 license-files = { paths = ["LICENSE"] }
 license = "Apache-2.0"
-keywords = ["requests", "http/2", "http/3", "QUIC", "http", "https", "http client", "http/1.1"]
+keywords = ["requests", "http/2", "http/3", "QUIC", "http", "https", "http client", "http/1.1", "ocsp", "revocation", "tls"]
 authors = [
   {name = "Kenneth Reitz", email = "[email protected]"}
 ]
@@ -41,7 +41,7 @@ dynamic = ["version"]
 dependencies = [
     "charset_normalizer>=2,<4",
     "idna>=2.5,<4",
-    "urllib3.future>=2.0.936,<3",
+    "urllib3.future>=2.1.900,<3",
     "wassima>=1.0.1,<2",
     "kiss_headers>=2,<4",
 ]

src/niquests/__version__.py

@@ -9,9 +9,9 @@
 __url__: str = "https://niquests.readthedocs.io"
 
 __version__: str
-__version__ = "3.0.2"
+__version__ = "3.1.0"
 
-__build__: int = 0x030002
+__build__: int = 0x030100
 __author__: str = "Kenneth Reitz"
 __author_email__: str = "[email protected]"
 __license__: str = "Apache-2.0"

src/niquests/api.py

@@ -31,9 +31,9 @@
     TLSVerifyType,
 )
 from .models import PreparedRequest, Response
+from .structures import SharableLimitedDict
 
-#: This is a non-thread safe in-memory cache for the AltSvc / h3
-_SHARED_QUIC_CACHE: CacheLayerAltSvcType = {}
+_SHARED_QUIC_CACHE: CacheLayerAltSvcType = SharableLimitedDict(max_size=12_288)
 
 
 def request(

src/niquests/extensions/__init__.py

@@ -0,0 +1,4 @@
+"""
+This subpackage hold anything that is very relevant
+to the HTTP ecosystem but not per-say Niquests core logic.
+"""