Unexpected error "'utf-8' codec codec can't decode byte ... with uwsgi + Flask_socketio

[danlg]

Hi

I have subclassed
MyServer extends flask_socketio.SocketIO
It is working fine with Flask and async_mode='gevent' and serializer ="msgpack" (client is javascript with msgpack too).
But when using uwsgi + async_mode='gevent_uwsgi', I get Unexpected error "'utf-8' codec thrown by engineio on Mac

2022-04-20 15:51:03,250 : WARNING : my.my_server : connect 1 client(s), new 6M7yMZwNqnFx4Yv0AAAD
l_BWAIozh8Sp6DkmAAAC: Sending packet MESSAGE data <binary>
[pid: 66959|app: 0|req: 9/8] 127.0.0.1 () {46 vars in 849 bytes} [Wed Apr 20 15:51:03 2022] POST /socket.io/?EIO=4&transport=polling&t=O17FQ9E&sid=l_BWAIozh8Sp6DkmAAAC => generated 2 bytes in 1 msecs (HTTP/1.1 200) 2 headers in 85 bytes (3 switches on core 8)
[pid: 66959|app: 0|req: 10/9] 127.0.0.1 () {42 vars in 788 bytes} [Wed Apr 20 15:51:03 2022] GET /socket.io/?EIO=4&transport=polling&t=O17FQ9J&sid=l_BWAIozh8Sp6DkmAAAC => generated 1 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 100 bytes (3 switches on core 8)
l_BWAIozh8Sp6DkmAAAC: Upgrade to websocket successful
l_BWAIozh8Sp6DkmAAAC: Sending packet PING data None
l_BWAIozh8Sp6DkmAAAC: Received packet PONG data 
l_BWAIozh8Sp6DkmAAAC: **Unexpected error "'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte", closing connection**
2022-04-20 15:51:46,585 : WARNING : my.my_server : disconnect 0 client(s), removed: 6M7yMZwNqnFx4Yv0AAAD

#MyServer extends flask_socketio.SocketIO

class MyServer(flask_socketio.SocketIO):
    def __init__(self, app=None, **kwargs):
        super().__init__(app, **kwargs)  
 ...

#my/uwsgi.py 
from flask import Flask
from my.my_server import MyServer
import flask_socketio
app = Flask(__name__)
socketio = MyServer(app=app,
    async_mode="gevent_uwsgi",
    serializer ="msgpack",
    logger= True,
    engineio_logger = True,
    cors_allowed_origins= "*"
)

# start_uwsgi.sh
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
echo $OBJC_DISABLE_INITIALIZE_FORK_SAFETY
uwsgi \
    --http :8070 \
    --gevent 10 \
    --http-websockets \
    --master \
    --process 1 \
    --threads 10 \
    --wsgi-file my/uwsgi.py \
    --callable app \
    --logger file:uwsgi.log

@miguelgrinberg any idea ? thanks

[danlg]

Manage to print the stack after modifying 1 line of engineio::socket.py

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask_socketio/__init__.py", line 43, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/middleware.py", line 63, in __call__
    return self.engineio_app.handle_request(environ, start_response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/socketio/server.py", line 597, in handle_request
    return self.eio.handle_request(environ, start_response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/server.py", line 413, in handle_request
    packets = socket.handle_get_request(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/socket.py", line 103, in handle_get_request
    return getattr(self, '_upgrade_' + transport)(environ,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/socket.py", line 158, in _upgrade_websocket
    return ws(environ, start_response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/async_drivers/gevent_uwsgi.py", line 61, in __call__
    self.app(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/socket.py", line 230, in _websocket_handler
    p = websocket_wait()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/socket.py", line 163, in websocket_wait
    data = ws.wait()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/async_drivers/gevent_uwsgi.py", line 142, in wait
    return self._decode_received(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/engineio/async_drivers/gevent_uwsgi.py", line 91, in _decode_received
    return msg.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte

so the issue is that is interpreted as text whereas it is binary:

# in gevent_uwsgi.py", line 91
    def _decode_received(self, msg):
        """Returns either bytes or str, depending on message type."""
        if not isinstance(msg, bytes):
            # already decoded - do nothing
            return msg
        # only decode from utf-8 if message is not binary data
        type = ord(msg[0:1])
        if type >= 48:  # no binary #issue may be here see amended comment below
            return msg.decode('utf-8') # <-- HERE IS THE ISSUE
        # binary message, don't try to decode
        return msg

[danlg]

I have found a workaround by commenting:

#if type >= 48: # no binary
# return msg.decode('utf-8')

it works fine. I wonder if this is not a bug in engineio

[danlg]

small correction
if type >= 48 and type <= 57: # added type <= 57 in order to allow websocket upgrade
return msg.decode('utf-8') # is kept

[miguelgrinberg]

The code that you are looking at tries to compensate for the uWSGI bug, and is what makes text/json encoded packets work. Removing that if statement would make all packets be binary, which is wrong. Changing the logic may give the impression that it works, but as soon as you get a binary packet that starts with 0-9 you'll get errors, or a binary packet might end up being interpreted as text.

Gunicorn with gevent works great. Just saying. Or else, maybe see if you can figure out how to fix uWSGI to return packets as text or binary, instead of always as binary.

[miguelgrinberg]

Can't really tell for sure, but I know that uWSGI has issues with binary WebSocket packets, which are sometimes reported to the application as text packets. See unbit/uwsgi#1167.

[danlg]

Thanks see my reply above. Thinking of creating an issue / bug report in python-engineio as a consequence ?

[danlg]

@miguelgrinberg ?

[miguelgrinberg]

This is an issue with uWSGI. Please talk to them and ask them to fix it.