Fix pyopenssl adapter under Python 3 #113

webknjaz · 2018-09-03T09:53:59Z

It works under Python 2, but not under Python 3.
It's a follow-up for #6

I'm submitting a ...

bug report
feature request
question about the decisions made in the repository

Describe the bug. What is the current behavior?

It raises exception OpenSSL.SSL.WantReadError.

What is the motivation / use case for changing the behavior?
I want consistency in SSL.

To Reproduce

Steps to reproduce the behavior: tox -- --no-cov -s cheroot/test/test_ssl.py

Expected behavior

That it'd work.

Details

 cheroot/test/test_ssl.py::test_ssl_adapters[builtin] ✓                                                          50% █████     
WantReadError()                                                                                                               
Traceback (most recent call last):        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate                                    
    req.parse_request()            
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request                                   
    success = self.read_request_line()        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line                               
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline                                        
    data = self.rfile.readline(256)         
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline                                          
    b = self.read(nreadahead())     
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)      
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek                                           
    return self._peek_unlocked(size)                                                                                          
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked                                   
    current = self.raw.read(to_read)                                                                                           
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)                                                                                             
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)                       
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()                                                                        
OpenSSL.SSL.WantReadError
Exception in thread CP Server Thread-13:                                                                                      Traceback (most recent call last):
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate
    req.parse_request()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request
    success = self.read_request_line()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline
    data = self.rfile.readline(256)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/workers/threadpool.py", line 112, in run                                     conn.communicate()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1256, in communicate
    self._conditional_error(req, '500 Internal Server Error')
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1282, in _conditional_error                                 req.simple_response(response)                             
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1071, in simple_response
    self.conn.wfile.write(EMPTY.join(buf))
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 31, in write
    self._flush_unlocked()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 40, in _flush_unlocked
    n = self.raw.write(bytes(self._write_buf))
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 607, in write
    return self._sock.send(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1729, in send
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

Environment

Cheroot version: master (6.5.2)
CherryPy version: N/A
Python version: 3.X
OS: XXX
Browser: N/A

Additional context

The text was updated successfully, but these errors were encountered:

webknjaz · 2018-09-03T09:59:53Z

webknjaz · 2018-09-03T10:08:05Z

python-trio/trio#200 (comment)

webknjaz · 2018-09-06T11:07:16Z

https://github.com/Lukasa/hyper/blob/development/hyper/ssl_compat.py#L114-L115

webknjaz · 2018-09-06T11:18:27Z

https://bugs.python.org/issue32219

webknjaz · 2018-09-06T11:18:57Z

https://github.com/python-trio/trio/pull/365/files

webknjaz · 2018-12-30T16:27:51Z

Fixed.

webknjaz · 2018-12-30T19:54:08Z

@mar10 could you please try out PyOpenSSL support now? I think I've finally fixed it.

mar10 · 2018-12-31T11:50:15Z

Hi,

gave it a quick test.
The problem was reproducible here: WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0

(wsgidav-9PbuForg) bash-3.2$ python -m OpenSSL.debug
pyOpenSSL: 18.0.0
cryptography: 2.4.2
cffi: 1.11.5
cryptography's compiled against OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
cryptography's linked OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
Pythons's OpenSSL: OpenSSL 1.0.2p  14 Aug 2018
Python executable: /Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/bin/python
Python version: 3.7.0 (default, Jul 26 2018, 22:23:34) 
[Clang 9.1.0 (clang-902.0.39.2)]
Platform: darwin

2018-12-31 12:09:50.866 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:09:50.866 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0
2018-12-31 12:09:50.866 - INFO    : Serving on https://0.0.0.0:8080 ...
WantReadError()
Traceback (most recent call last):
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 1242, in communicate
    req.parse_request()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 712, in parse_request
    success = self.read_request_line()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 751, in read_request_line
    request_line = self.rfile.readline()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 300, in readline
    data = self.rfile.readline(256)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

(looks like issue #132)
Seems you nailed it with the current master:

(wsgidav-9PbuForg) bash-3.2$ pip install git+git://github.com/cherrypy/cheroot.git@master
[...]
2018-12-31 12:24:28.789 - INFO    : WsgiDAV/3.0.0a8 Python/3.7.0 Darwin-18.2.0-x86_64-i386-64bit
[...]
2018-12-31 12:24:28.850 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:24:28.851 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.4.dev43+gab92fb97 Python/3.7.0
2018-12-31 12:24:28.851 - INFO    : Serving on https://0.0.0.0:8080 ...
2018-12-31 12:24:45.333 - INFO    : 127.0.0.1 - (anonymous) - [2018-12-31 11:24:45] "GET /" elap=0.006sec -> 200 OK

Thanks for keeping up the good work and happy new year 🎉

webknjaz · 2018-12-31T13:56:45Z

Cool thanks :)

webknjaz added bug Something is broken help wanted Somebody help us, please! labels Sep 3, 2018

webknjaz mentioned this issue Sep 3, 2018

Cannot import cheroot.ssl.pyopenssl #6

Closed

This comment has been minimized.

Sign in to view

webknjaz closed this as completed Dec 30, 2018

Rohitbels mentioned this issue Oct 22, 2019

Unhandled OpenSSL.SSL.WantReadError and WantWriteError raised by PyOpenSSL #245

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix pyopenssl adapter under Python 3 #113

Fix pyopenssl adapter under Python 3 #113

webknjaz commented Sep 3, 2018

webknjaz commented Sep 3, 2018

This comment has been minimized.

webknjaz commented Sep 3, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Dec 30, 2018

webknjaz commented Dec 30, 2018

mar10 commented Dec 31, 2018

webknjaz commented Dec 31, 2018

Fix pyopenssl adapter under Python 3 #113

Fix pyopenssl adapter under Python 3 #113

Comments

webknjaz commented Sep 3, 2018

webknjaz commented Sep 3, 2018

This comment has been minimized.

webknjaz commented Sep 3, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Sep 6, 2018

webknjaz commented Dec 30, 2018

webknjaz commented Dec 30, 2018

mar10 commented Dec 31, 2018

webknjaz commented Dec 31, 2018