IOError: unable to complete websocket handshake while running in uwsgi+Nginx #631
Comments
|
Screenshot of my overall project status
|
|
Have you seen the documentation regarding deployment with uwsgi? I suggest you start from the configuration from there. In particular, you have to use one worker. To use multiple workers with SocketIO the set up is more complicated. The workers must all be standalone uwsgi instances with one worker, and nginx should be configured to load balance across all these workers. Enabling multiple workers in uwsgi is not supported because this protocol requires sticky sessions, which nginx provides. |
|
Yes, I configured each and every parameter except worker for uwsgi following the official documentation expecting that multiple worker will be a more performant option for production deployment. So, using And, since I couldn't use multiple worker in the uwsgi part, I could have tried simulating multiple workers in the Nginx configuration. Thanks. |
|
Not sure if it is the only problem, but definitely something you need to fix. Switch to one worker, then test using uwsgi directly, and if that works, then add nginx. |
|
Yes, using single worker solved the problem. No problem in message sending and receiving and everything is working pretty fast. But the errors like unable to complete websocket handshake and IOError: Write Error is still in action. |
|
Do all your clients fail to connect over websocket, or just some of them? You'll have to debug this to identify which clients fail to connect, and why. |
|
Ok. I will check back accordingly and let you know |
|
Hello @miguelgrinberg After analyzing everything as you said and checking some server-client connections :-
Then, I tried to remove nginx out of the picture and tried to run the project directly with the uwsgi server. But still I can reproduce those errors IOError:unable to complete websocket handshake and IOError: Write Error. So, it's obvious that this is not a nginx configuration issue. And, when I run with the default wsgi server Here's the uwsgi status while running uwsgi directly without nginxAnd here's my project status and error logSo, could you please give me some ideas about the above two errors. I don't even know why they are occurring ? what is the root cause of them ? Without properly understanding the errors I think I couldn't be able to solve the errors myself. I have searched for IOError:unable to complete websocket handshake and IOError: Write Error but what I'm getting is out of context of Flask-SocketIO. So, currently I am totally helpless unless I could grasp proper reasoning of those errors. |
|
Socket.IO can work over HTTP as well as WebSocket. The fact that the application works does not mean websocket is working. The websocket error that you are getting likely indicates that the client cannot use websocket. But I wanted to know if the connection fails every time or only occasionally, or maybe for specific clients. The error indicates that the connection handshake between uWSGI and the client could not complete. This error is not a SocketIO error, it comes from uWSGI, so you need to find out what some or all of your clients fail to establish a websocket connection with uWSGI. My guess is that there is a problem with your uWSGI server, specifically in its websocket part. You may want to find a simpler websocket example for uWSGI and see if you can make it work, my guess is that you are still going to see this error. |
|
@miguelgrinberg .. I got the point you described Yes, from the error log I shared it's clearly seen that
HTTP has been used.
So, how can I switch to work SocketIO from HTTP to WebSocket ?? Is it to be handled in coding section or in the uwsgi configuration ?? Here's the header of one the requests from client. But I couldn't figure out from there whether HTTP or WebSocket has been used.
|
|
Your client is using HTTP. For WebSocket there are no requests constantly being issued, it's a long term connection. |
|
@miguelgrinberg Yes, I mentioned it in my comment as well. But, how can I make SocketIO switch from HTTP to WebSocket for all available clients ?? Is it to be handled in coding section or in the uwsgi configuration ?? |
|
It happens automatically. The handshake error prevents the websocket connection from taking over HTTP. As I said above, it could be a uwsgi problem, look for a simpler websocket example and make sure it works before you continue troubleshooting, because my guess is that the problem is at a more basic level than SocketIO. |
|
Yes. It's confirmed that client is connected over HTTP. So, now I have forgot about my project. I just tested the Flask-SocketIO example which You posted in Your blog tutorial and run As same error is appearing regardless of different projects it's clear that problem underlies with my uwsgi configuration or in my ubuntu server . After digging out more what I got is:
So, upgrading my nginx version could help. Anyway, many thanks for your constant support. |
|
Your nginx is fine (1.12 is newer than 1.4). But you may want to try to build uwsgi with ssl and websocket support if you haven't done that already. |
|
Dear @miguelgrinberg, As you already know I have been using uWSGI+ gevent and ssl support for uwsgi seems different and cumbersome for gevent rather than eventlet. So, I have decided to perform the SSL verification part with Nginx for HTTPS support. Here's my Nginx configuration:- It works pretty fine allowing clients to use https with port 443. But in the console I am getting the following error:
It seems that socke.io has been blocked while transferring contents using websockts. Could you please give me any light to resolve this issue ? PS:- I have tried with localhost as I found Firefox allow mixed content for localhost ... but still same error is occurring. So, that might be the socket.io configuration problem. |
|
The JavaScript connection URL for SocketIO should be |
|
I have been used relative protocol caller for the scripts... Hence I will analyze once again. Thanks!! |
|
Many thanks. No errors now in browser console and network tab.
I am sensing the error arises because of Unix Socket. So, is there any hindrance using Unix Socket instead of http-socket in Flask-SocketIO ?? |
|
Does the error go away if you switch to a regular socket? This is saying that the server closed the connection with nginx prematurely, it has nothing to do with SocketIO or WebSocket. |
|
@miguelgrinberg
Clicking on the socket.io link I am seeing
My Nginx config of the ssl part once again |
|
There are a couple of things that are odd in your nginx config:
Hopefully fixing these will address your problem. |
|
Hello @miguelgrinberg, All the errors I addressed regarding this project has been solved. I can now successfully send requests and getting back responses through websokcet. I have used self signed certificate for https support. Now, everything is working just fine. Onemore thing , as far I understand that socket.io is actually a wrapper over original flask applications. So, I could expect similar behaviors regardless of which server I will be using. I have already informed that I have been using uwsgi with gevent as asynchronous. So, when I have been passing certfile and keyfile directly to So, running the server.py directly from virtual environment with But, when I am running the same
then a certification error occurred.
That seems surprising to me as I am passing the same certificated files (foobar.crt, foobar.key), same port, but when I have switched to uwsgi server I am getting the error. |
|
Did you also post this question in Stack Overflow? https://stackoverflow.com/questions/48861768/ssl-sslerror-ssl-certificate-verify-failed-certificate-verify-failed-ssl-c I just answered there, the problem is not with uwsgi, but with the python requests library. |
|
Yes, but as I have been using dialogflow's request mechanism, I have no control over their request mechanism. So, I can't do In another post, I described more details about it https://stackoverflow.com/questions/48878420/how-to-disable-ssl-verification-for-text-request-of-api-ai-dialogflow |
|
And, as you described the problem is not in the chat_message() section because working with HTTPS it only requires the crt and key file. it's working fine. But when I am serving it with uWsgi with the certificate files like that
then control has been shifted to uwsgi and as a result the chat_message() function resides in the server.py file can't find those certificate files and resulting the error. So, is there any way to pass the certificate files concurrently to socketio.run() ? |
|
I'm very confused. The way I understand the flow of requests in your system is as follows:
Why do you keep insisting that this has something to do with your own uWSGI server, given that the request that fails to validate is the HTTP request that is sent in the Socket.IO event handler? Is this request recursively connecting to itself? |
|
I am also confused. I am trying to describe as far I understand.
I think that another server (dialogflow) is using the self-signed certificate which I passed in the socketio.run() method as arguments. Running the application with But, when uwsgi has been used, then socketio.run() has become void (As far my understanding reading the Flask-SocketIO official doc. Pardon me if I'm not correct) and the dialogflow server can't find the self signed certificate files as parameters.
Yes, validation failed only when client start chatting in the browser. I'm just getting user queries from client side and just sending the queries to the server side to get a response from the dialogflow server. That's it. So, when https is used in the browser most probably dialogflow server couldn't find those certificate files. |
|
You are not really explaining how your application works. I have no idea what this other digiflow server is or who runs it. I don't see how it could "steal" your own self-signed certificate and use it for itself. I think you need to provide a better explanation of how your application is designed, I really can't help you with the information you provided. |
|
Okay. I don't know whether the dialogflow is stealing my self-signed certificate or not. I know you're so busy, but still if you really wanna see that magic, I could connect you to my server. Thanks. |
|
I'm really not interested in your server. As far as I can see that works well. The dialogflow server has the problem with an invalid certificate. Who runs that server? Let them know that they need to fix their certificate and that would end your problems. |
|
Dialogflow is a NLP Machine Learning Platform owned by Google. And what I'm using is Dialogflow's Python Client. |
|
As I said above, the problem is not in your code, but in the third-party server you are contacting. You need to let them know that they are using an invalid certificate, there is nothing else you can do. See https://discuss.api.ai/t/api-api-ai-certificate-error/1457, there is a bunch of people complaining about the same thing. Your only solution is to modify their Python client to send the request with verification turned off. |
|
Ok. But my question is why it's working when passing certificate file as a argument in socketio.run() Isn't that surprising and illogical ? ? I already created an issue in their github. :(
Yes, exactly with this thoughts I created the issue and post a question on stack overflow . |
Something else must be different when you run the server in these two modes. Either a different version of Python, any other package, etc. |
|
Hello @miguelgrinberg The problem has been solved. I tried to make it work with HTTP connection class but it doesn't help. Then, I analyzed Dialogflow's root request mechanism and found they're using httplib.HTTPSConnection() as the connection class and
I just extended the And, A great many thanks to You for your constant support and immediate responses all along. So, now the original error including all the sub-errors regarding this issue has been solved. I feel certain that it's time I could close the issue. |
|
Hi, I was following the whole conversation. I am presenting exactly the same inconvenience. |
|
@felocru Oh, I missed that message above. In this context "SSL support" means the SSL libraries, which uWSGI must be using to handle some parts of the websocket handshake. It does not mean that WebSocket only works on secure HTTP, those are two completely different things. But yes, try rebuilding uWSGI after installing SSL libs. |
|
I am getting this same error ( I'm installing from the binary via Is there a way to check if I'm also using |
|
Hello, No, it will not be enough to install uWSGI and SSL separately for running the application perfectly with uWSGI ! This guide will lead you to the right direction all along !! Best of luck !! |
|
I am also getting the same error using the example code of Flask-SocketIO, but only for the first try. This is my ini file: And here is the requirements.txt: It seems that this is on uWSGI side, but is there a way to avoid it? Shall I use an older version of a library? |
|
@evangelos-dimitriadis It's the same error: "you need to build uWSGI with SSL support to use the websocket handshake api function !!!". So do that, and come back if you still have issues with a correctly built uWSGI. |
I have been planning to deploy a flask-SocketIO application into production using uwsgi server behind Nginx
Versions including relevant dependencies of my virtual environment:-
I am intended to run the uwsgi server through systemd and load the uwsgi configurations from a .ini file.
Here's my
wsgi.pyfile:Here's the configuration file
myproject.inifor uwsgiHere's the
myproject.servicefile in/etc/systemd/systemAnd, finally my nginx configuration file:
Errors:-
Actually I'm getting different errors from several times.
1) IOError: unable to complete websocket handshake
Here's my project status while running:_
2) IOError: write error: uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.
3) Nginx error log
Last 20 line of /var/log/nginx/error.log
The text was updated successfully, but these errors were encountered: