Can gunicorn (message_queue)` work multiple times? #1940

liuqijie6 · 2023-02-10T10:43:10Z

liuqijie6
Feb 10, 2023

I saw other people's information on the Internet that they can only use uWSGI to perform multiple jobs, but not gunicorn. I want to know which one should be used to perform multiple work.

I useing socketIO(messages_queue='redis')
I tested gunicorn work=10, but some connections have been timed out during the test, I don't know why, if I change to work=1, then it will be connected quickly.

[2023-02-10 18:48:22 +0800] [2641849] [INFO] Starting gunicorn 20.1.0
[2023-02-10 18:48:22 +0800] [2641849] [INFO] Listening at: http://127.0.0.1:5000 (2641849)
[2023-02-10 18:48:22 +0800] [2641849] [INFO] Using worker: eventlet
[2023-02-10 18:48:22 +0800] [2641854] [INFO] Booting worker with pid: 2641854
[2023-02-10 18:48:22 +0800] [2641855] [INFO] Booting worker with pid: 2641855


[2023-02-10 18:48:22 +0800] [2641857] [INFO] Booting worker with pid: 2641857
[2023-02-10 18:48:22 +0800] [2641858] [INFO] Booting worker with pid: 2641858

[2023-02-10 18:48:22 +0800] [2641859] [INFO] Booting worker with pid: 2641859
[2023-02-10 18:48:22 +0800] [2641860] [INFO] Booting worker with pid: 2641860
[2023-02-10 18:48:22 +0800] [2641861] [INFO] Booting worker with pid: 2641861

[2023-02-10 18:48:22 +0800] [2641862] [INFO] Booting worker with pid: 2641862
[2023-02-10 18:48:22 +0800] [2641864] [INFO] Booting worker with pid: 2641864
[2023-02-10 18:48:22 +0800] [2641865] [INFO] Booting worker with pid: 2641865
[2023-02-10 18:48:22 +0800] [2641867] [INFO] Booting worker with pid: 2641867
[2023-02-10 18:48:22 +0800] [2641868] [INFO] Booting worker with pid: 2641868
[2023-02-10 18:48:22 +0800] [2641869] [INFO] Booting worker with pid: 2641869
[2023-02-10 18:48:22 +0800] [2641870] [INFO] Booting worker with pid: 2641870
[2023-02-10 18:48:22 +0800] [2641871] [INFO] Booting worker with pid: 2641871
[2023-02-10 18:48:23 +0800] [2641872] [INFO] Booting worker with pid: 2641872
Server initialized for eventlet.
8uJokejeGKvP4VpiAAAA: Sending packet OPEN data {'sid': '8uJokejeGKvP4VpiAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
-ReZy3Gi96fSxORrAAAB: Sending packet OPEN data {'sid': '-ReZy3Gi96fSxORrAAAB', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
- - [10/Feb/2023:18:48:23 +0800] - <2641855> - "GET /socket.io/?transport=polling&EIO=4&t=1676026102.2354953 HTTP/1.1" - 200 - 97 - "0.001453" - "python-requests/2.27.1"
redis backend initialized.
- - [10/Feb/2023:18:48:23 +0800] - <2641855> - "GET /socket.io/?transport=polling&EIO=4&t=1676026102.6387022 HTTP/1.1" - 200 - 97 - "0.001613" - "python-requests/2.27.1"
8uJokejeGKvP4VpiAAAA: Received request to upgrade to websocket
-ReZy3Gi96fSxORrAAAB: Received request to upgrade to websocket
8uJokejeGKvP4VpiAAAA: Upgrade to websocket successful
8uJokejeGKvP4VpiAAAA: Received packet MESSAGE data 0{"app_id":1,"player_id":1,"uid":1,"permission":1}
8uJokejeGKvP4VpiAAAA: Sending packet MESSAGE data 0{"sid":"cbOvnY-8nwXtph2zAAAC"}
8uJokejeGKvP4VpiAAAA: Received packet MESSAGE data 0{"app_id":1,"player_id":1,"uid":1,"permission":1}
8uJokejeGKvP4VpiAAAA: Sending packet MESSAGE data 4"Unable to connect"
-ReZy3Gi96fSxORrAAAB: Upgrade to websocket successful
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
Server initialized for eventlet.
t9Q0lb45JzejE6MpAAAE: Sending packet OPEN data {'sid': 't9Q0lb45JzejE6MpAAAE', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
- - [10/Feb/2023:18:48:25 +0800] - <2641855> - "GET /socket.io/?transport=polling&EIO=4&t=1676026105.2549372 HTTP/1.1" - 200 - 97 - "0.000643" - "python-requests/2.27.1"
Invalid session t9Q0lb45JzejE6MpAAAE (further occurrences of this error will be logged with level INFO)
- - [10/Feb/2023:18:48:25 +0800] - <2641862> - "GET /socket.io/?transport=websocket&EIO=4&sid=t9Q0lb45JzejE6MpAAAE&t=1676026105.257236 HTTP/1.1" - 400 - 17 - "0.000565" - "-"
t9Q0lb45JzejE6MpAAAE: Received packet MESSAGE data 0{"app_id":1,"player_id":1,"uid":1,"permission":1}
t9Q0lb45JzejE6MpAAAE: Sending packet MESSAGE data 0{"sid":"OpCkTG3pS_Opv1aUAAAF"}
- - [10/Feb/2023:18:48:25 +0800] - <2641855> - "POST /socket.io/?transport=polling&EIO=4&sid=t9Q0lb45JzejE6MpAAAE HTTP/1.1" - 200 - 2 - "0.000866" - "python-requests/2.27.1"
- - [10/Feb/2023:18:48:25 +0800] - <2641855> - "GET /socket.io/?transport=polling&EIO=4&sid=t9Q0lb45JzejE6MpAAAE&t=1676026105.2592976 HTTP/1.1" - 200 - 32 - "0.002000" - "python-requests/2.27.1"
Server initialized for eventlet.

when i connect

S0_Ia0Vzlbsq5ST4AAAA: Sending packet OPEN data {'sid': 'S0_Ia0Vzlbsq5ST4AAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
redis backend initialized.
36.227.235.19 - [10/Feb/2023:18:48:56 +0800] - <2641859> - "GET /socket.io/?transport=polling&EIO=4&t=1676026133.774965 HTTP/1.1" - 200 - 97 - "0.001679" - "python-requests/2.27.1"
Invalid session S0_Ia0Vzlbsq5ST4AAAA (further occurrences of this error will be logged with level INFO)
36.227.235.19 - [10/Feb/2023:18:48:56 +0800] - <2641868> - "GET /socket.io/?transport=websocket&EIO=4&sid=S0_Ia0Vzlbsq5ST4AAAA&t=1676026134.120538 HTTP/1.1" - 400 - 17 - "0.000409" - "-"
S0_Ia0Vzlbsq5ST4AAAA: Received packet MESSAGE data 0{"uid":"123456803","app_id":"2","player_id":"7820345190"}
emitting event "test_message" to all [/]
pubsub message: emit
S0_Ia0Vzlbsq5ST4AAAA: Sending packet MESSAGE data 2["test_message","\u8fde\u63a5\u6210\u529f"]
pubsub message: emit
NjibYLbSKiIKGbOEAAAB is entering room 2_7820345190 [/]
t9Q0lb45JzejE6MpAAAE: Sending packet MESSAGE data 2["test_message","\u8fde\u63a5\u6210\u529f"]
- - [10/Feb/2023:18:48:56 +0800] - <2641855> - "GET /socket.io/?transport=polling&EIO=4&sid=t9Q0lb45JzejE6MpAAAE&t=1676026130.2589822 HTTP/1.1" - 200 - 45 - "6.494045" - "python-requests/2.27.1"
emitting event "test_message" to 2_7820345190 [/]
S0_Ia0Vzlbsq5ST4AAAA: Sending packet MESSAGE data 0{"sid":"NjibYLbSKiIKGbOEAAAB"}
pubsub message: emit
36.227.235.19 - [10/Feb/2023:18:48:56 +0800] - <2641859> - "POST /socket.io/?transport=polling&EIO=4&sid=S0_Ia0Vzlbsq5ST4AAAA HTTP/1.1" - 200 - 2 - "0.032796" - "python-requests/2.27.1"
pubsub message: emit
S0_Ia0Vzlbsq5ST4AAAA: Sending packet MESSAGE data 2["test_message","\u8fde\u63a5\u6210\u529f"]
Invalid session S0_Ia0Vzlbsq5ST4AAAA (further occurrences of this error will be logged with level INFO)
36.227.235.19 - [10/Feb/2023:18:48:56 +0800] - <2641865> - "GET /socket.io/?transport=polling&EIO=4&sid=S0_Ia0Vzlbsq5ST4AAAA&t=1676026134.5209548 HTTP/1.1" - 400 - 17 - "0.000441" - "python-requests/2.27.1"
FJEsB2Yd4KEPD17jAAAA: Sending packet OPEN data {'sid': 'FJEsB2Yd4KEPD17jAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
redis backend initialized.
36.227.235.19 - [10/Feb/2023:18:48:58 +0800] - <2641868> - "GET /socket.io/?transport=polling&EIO=4&t=1676026136.123345 HTTP/1.1" - 200 - 97 - "0.001007" - "python-requests/2.27.1"
Invalid session FJEsB2Yd4KEPD17jAAAA
36.227.235.19 - [10/Feb/2023:18:48:58 +0800] - <2641871> - "GET /socket.io/?transport=websocket&EIO=4&sid=FJEsB2Yd4KEPD17jAAAA&t=1676026136.270974 HTTP/1.1" - 400 - 17 - "0.000369" - "-"
Invalid session FJEsB2Yd4KEPD17jAAAA (further occurrences of this error will be logged with level INFO)
36.227.235.19 - [10/Feb/2023:18:48:58 +0800] - <2641864> - "GET /socket.io/?transport=polling&EIO=4&sid=FJEsB2Yd4KEPD17jAAAA&t=1676026136.6223161 HTTP/1.1" - 400 - 17 - "0.000381" - "python-requests/2.27.1"
Invalid session FJEsB2Yd4KEPD17jAAAA
36.227.235.19 - [10/Feb/2023:18:48:58 +0800] - <2641865> - "POST /socket.io/?transport=polling&EIO=4&sid=FJEsB2Yd4KEPD17jAAAA HTTP/1.1" - 400 - 17 - "0.000233" - "python-requests/2.27.1"
7iaIOFMGIoF_HdroAAAA: Sending packet OPEN data {'sid': '7iaIOFMGIoF_HdroAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
redis backend initialized.
36.227.235.19 - [10/Feb/2023:18:49:01 +0800] - <2641871> - "GET /socket.io/?transport=polling&EIO=4&t=1676026139.6624892 HTTP/1.1" - 200 - 97 - "0.001077" - "python-requests/2.27.1"
Invalid session 7iaIOFMGIoF_HdroAAAA
36.227.235.19 - [10/Feb/2023:18:49:02 +0800] - <2641864> - "GET /socket.io/?transport=websocket&EIO=4&sid=7iaIOFMGIoF_HdroAAAA&t=1676026139.932639 HTTP/1.1" - 400 - 17 - "0.000306" - "-"
Invalid session 7iaIOFMGIoF_HdroAAAA
36.227.235.19 - [10/Feb/2023:18:49:02 +0800] - <2641864> - "GET /socket.io/?transport=polling&EIO=4&sid=7iaIOFMGIoF_HdroAAAA&t=1676026140.2807481 HTTP/1.1" - 400 - 17 - "0.000220" - "python-requests/2.27.1"
Invalid session 7iaIOFMGIoF_HdroAAAA
36.227.235.19 - [10/Feb/2023:18:49:02 +0800] - <2641864> - "POST /socket.io/?transport=polling&EIO=4&sid=7iaIOFMGIoF_HdroAAAA HTTP/1.1" - 400 - 17 - "0.000209" - "python-requests/2.27.1"
8Y6rdBJfLIpVFQAtAAAB: Sending packet OPEN data {'sid': '8Y6rdBJfLIpVFQAtAAAB', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
36.227.235.19 - [10/Feb/2023:18:49:07 +0800] - <2641868> - "GET /socket.io/?transport=polling&EIO=4&t=1676026145.437293 HTTP/1.1" - 200 - 97 - "0.000489" - "python-requests/2.27.1"
Invalid session 8Y6rdBJfLIpVFQAtAAAB (further occurrences of this error will be logged with level INFO)
36.227.235.19 - [10/Feb/2023:18:49:07 +0800] - <2641857> - "GET /socket.io/?transport=websocket&EIO=4&sid=8Y6rdBJfLIpVFQAtAAAB&t=1676026145.588234 HTTP/1.1" - 400 - 17 - "0.000393" - "-"
Invalid session 8Y6rdBJfLIpVFQAtAAAB
36.227.235.19 - [10/Feb/2023:18:49:08 +0800] - <2641871> - "GET /socket.io/?transport=polling&EIO=4&sid=8Y6rdBJfLIpVFQAtAAAB&t=1676026146.0895941 HTTP/1.1" - 400 - 17 - "0.000348" - "python-requests/2.27.1"
Invalid session 8Y6rdBJfLIpVFQAtAAAB (further occurrences of this error will be logged with level INFO)
36.227.235.19 - [10/Feb/2023:18:49:08 +0800] - <2641872> - "POST /socket.io/?transport=polling&EIO=4&sid=8Y6rdBJfLIpVFQAtAAAB HTTP/1.1" - 400 - 17 - "0.000392" - "python-requests/2.27.1"
zMakmlVZ6q8OJzUNAAAB: Sending packet OPEN data {'sid': 'zMakmlVZ6q8OJzUNAAAB', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
36.227.235.19 - [10/Feb/2023:18:49:13 +0800] - <2641871> - "GET /socket.io/?transport=polling&EIO=4&t=1676026151.734952 HTTP/1.1" - 200 - 97 - "0.000484" - "python-requests/2.27.1"
Invalid session zMakmlVZ6q8OJzUNAAAB
36.227.235.19 - [10/Feb/2023:18:49:14 +0800] - <2641872> - "GET /socket.io/?transport=websocket&EIO=4&sid=zMakmlVZ6q8OJzUNAAAB&t=1676026151.913861 HTTP/1.1" - 400 - 17 - "0.000312" - "-"
Invalid session zMakmlVZ6q8OJzUNAAAB
36.227.235.19 - [10/Feb/2023:18:49:14 +0800] - <2641865> - "GET /socket.io/?transport=polling&EIO=4&sid=zMakmlVZ6q8OJzUNAAAB&t=1676026152.26388 HTTP/1.1" - 400 - 17 - "0.000289" - "python-requests/2.27.1"
Invalid session zMakmlVZ6q8OJzUNAAAB
36.227.235.19 - [10/Feb/2023:18:49:14 +0800] - <2641857> - "POST /socket.io/?transport=polling&EIO=4&sid=zMakmlVZ6q8OJzUNAAAB HTTP/1.1" - 400 - 17 - "0.000283" - "python-requests/2.27.1"
S0_Ia0Vzlbsq5ST4AAAA: Sending packet PING data None
FJEsB2Yd4KEPD17jAAAA: Sending packet PING data None
t9Q0lb45JzejE6MpAAAE: Client is gone, closing socket
t9Q0lb45JzejE6MpAAAE: Client is gone, closing socket
7iaIOFMGIoF_HdroAAAA: Sending packet PING data None

clinet

/pkwinners/tests/soketio_client.py
Traceback (most recent call last):
  File "/Users/fanren/pkwinners/tests/soketio_client.py", line 48, in <module>
    sio.connect('http://pkwinners.com/', headers=headers, auth=auth, wait_timeout=10)
  File "/opt/miniconda3/envs/pkwinners/lib/python3.8/site-packages/socketio/client.py", line 347, in connect
    raise exceptions.ConnectionError(
socketio.exceptions.ConnectionError: One or more namespaces failed to connect

if setting work=1 , it is work .

miguelgrinberg · 2023-02-10T11:05:50Z

miguelgrinberg
Feb 10, 2023
Maintainer

There is no need to "look on the internet". This package has official documentation that shows how to run both Gunicorn and uWSGI. For both, you should use a single worker.

Gunicorn: https://flask-socketio.readthedocs.io/en/latest/deployment.html#gunicorn-web-server
uWSGI: https://flask-socketio.readthedocs.io/en/latest/deployment.html#uwsgi-web-server

If you want to run multiple workers, then run multiple Gunicorns or uWSGIs, with nginx as a load balancer in front.

Nginx docs: https://flask-socketio.readthedocs.io/en/latest/deployment.html#using-nginx-as-a-websocket-reverse-proxy

3 replies

liuqijie6 Feb 10, 2023
Author

Currently, is there no way to support it ?

I understand it this way
app1 gunicorn work=1
app2 gunicorn work=1
nginx proxy_pass(ip_hash) -> app1 or app2 ?

The documentation doesn't mention threads, can I use multiple threads?
can threads >=2 ?

miguelgrinberg Feb 10, 2023
Maintainer

You cannot use threads. You are using greenlets already, so threads do not really help you in any way.

liuqijie6 Feb 10, 2023
Author

thanks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Can gunicorn (message_queue)` work multiple times? #1940

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Can gunicorn (message_queue)` work multiple times? #1940

Uh oh!

Uh oh!

liuqijie6 Feb 10, 2023

Replies: 1 comment · 3 replies

Uh oh!

Uh oh!

miguelgrinberg Feb 10, 2023 Maintainer

Uh oh!

liuqijie6 Feb 10, 2023 Author

Uh oh!

miguelgrinberg Feb 10, 2023 Maintainer

Uh oh!

liuqijie6 Feb 10, 2023 Author

liuqijie6
Feb 10, 2023

Replies: 1 comment 3 replies

miguelgrinberg
Feb 10, 2023
Maintainer

liuqijie6 Feb 10, 2023
Author

miguelgrinberg Feb 10, 2023
Maintainer

liuqijie6 Feb 10, 2023
Author