How to use redis combine with flask-socketio

[xiazhibin]

In flask, I can init flask_redis as an extension redis.init_app(app), and use redis.set(key, value) every flask request,
But in flask-socketIO, I have no idea with it.how to init and how to use.
For I want to record some player state in redis

Is this right?

from flask_redis import FlaskRedis

app = Flask(__name__)
app.config['SECRET_KEY'] = 'asdfasdf'
socketio = SocketIO(app)
redis = FlaskRedis(app)

@socketio.on('connect')
def test_connect(auth):
    redis.set(request.sid, "alive")
    emit('message', {'data': 'Connected'})

@socketio.on('message')
def test_connect(data):
    state = redis.get(request.sid)
    emit('message', {'data': state})

socketio.run(app)

[miguelgrinberg]

Are you reading the documentation? https://flask-socketio.readthedocs.io/en/latest/deployment.html#using-multiple-workers

[miguelgrinberg]

@MeshulashAdmin I'm still confused by what you are saying. Are you using Flask-SocketIO with multiple instances and without Redis? As I said, Redis is required when using multiple instances, not optional. If you are not using Redis you are going to have many issues and it is unfair that you are making me waste time when I already told you several times that you are required to use Redis in your setup. So please, add Redis, then report if you have any issues left.

[MeshulashAdmin]

Hi Miguel, thank you for the reply as always your time is precious and your comments are much appreciated!
Im trying to figure out how to work around my problem which is I cannot use sticky sessions.

At the moment 1 instance is running and it works perfect, but I want to add more instances.
As soon as another instance is spawning the connection is lost as we need sticky sessions.

I added on Development Env
socketio = SocketIO(
max_http_buffer_size=MAX_BUFFER_SIZE,
message_queue=REDIS_MESSAGE_QUEUE_URL,
async_mode='gevent',
cors_allowed_origins="*",
logger=True,
engineio_logger=True,
)

When I add message_queue arg then I get this:
web_1 | k394sxBZQ96SbV4mAAAB: Sending packet OPEN data {'sid': 'k394sxBZQ96SbV4mAAAB', 'upgrades': [], 'pingTimeout': 20000, 'pingInterval': 25000, 'maxPayload': 50000000}
web_1 | k394sxBZQ96SbV4mAAAB: Received request to upgrade to websocket
web_1 | k394sxBZQ96SbV4mAAAB: Upgrade to websocket successful
web_1 | k394sxBZQ96SbV4mAAAB: Received packet MESSAGE data 0/chat,
web_1 | k394sxBZQ96SbV4mAAAB: Sending packet MESSAGE data 4/chat,"Unable to connect"

I want to make sure this is the appropriate solution.

My needs are:
Multiple instances of the app on different machines and Redis as a message queue. All messages or emits are sent to redis and the instances will pull the message it self and distribute them to the right rooms.
Is that possible? If not what would be the best practice to achieve that?
We do not run nginx we use Digital Ocean's LB and they do not support sticky sessions. I do want to stay with them that is why Im trying to figure it out.

these are the versions:
python-socketio==5.12.1
python-engineio==4.11.0
Flask-SocketIO==5.5.1

on frontend when the user is in the chat:
const socket = io('/chat', { transports: ['websocket','polling'] });

socket.emit('join', { room: currentCustomerRoom });

socket.on('connect', () => {
console.log('Connected to server');
socket.emit('join', { room: currentCustomerRoom });
});

These logs are when I run it on Development Env and logged in to the chat.

When I remove the message_queue it works fine but then I cant scale up.

Would love to get your insights as I have read almost every post on stack overflow and the documentations of the libraries and the codes of the libraries. I still cant figure it out or its still remain unclear to me. GPT didn't help and I'm reading through most of the discussions to get familiar with problems and solutions.

Your comment is much appreciated,
Thank you.

[miguelgrinberg]

I mentioned this above. The long-polling protocol requires sticky sessions, but the websocket protocol does not. So have your clients connect via websocket directly.

[MeshulashAdmin]

Hi thanks for the reply,

Logs are:
web_1 | kxnHkv_HMdTayi9rAAAA: Sending packet OPEN data {'sid': 'kxnHkv_HMdTayi9rAAAA', 'upgrades': [], 'pingTimeout': 20000, 'pingInterval': 25000, 'maxPayload': 50000000}
web_1 | redis backend initialized.
web_1 | kxnHkv_HMdTayi9rAAAA: Received request to upgrade to websocket
web_1 | kxnHkv_HMdTayi9rAAAA: Upgrade to websocket successful
web_1 | kxnHkv_HMdTayi9rAAAA: Received packet MESSAGE data 0/chat,
web_1 | kxnHkv_HMdTayi9rAAAA: Sending packet MESSAGE data 4/chat,"Unable to connect"

this is on the front end:
// Connect to the /chat namespace
const socket = io('/chat', { transports: ['websocket'] });

When I downgrade the version of the python-socketio to 5.6.0 then it can connect
python-socketio==5.6.0
web_1 | wJY-VBKTJkEoTPuMAAAA: Sending packet OPEN data {'sid': 'wJY-VBKTJkEoTPuMAAAA', 'upgrades': [], 'pingTimeout': 20000, 'pingInterval': 25000}
web_1 | redis backend initialized.
web_1 | wJY-VBKTJkEoTPuMAAAA: Received request to upgrade to websocket
web_1 | wJY-VBKTJkEoTPuMAAAA: Upgrade to websocket successful
web_1 | wJY-VBKTJkEoTPuMAAAA: Received packet MESSAGE data 0/chat,
web_1 | wJY-VBKTJkEoTPuMAAAA: Sending packet MESSAGE data 0/chat,{"sid":"MYTT9opV_r3ojokGAAAB"}
web_1 | wJY-VBKTJkEoTPuMAAAA: Received packet MESSAGE data 2/chat,["join",{"room":"room_2dc7564d-1d6b-4d15-9534-d678c496c8cf_1d404a18-6e59-4e3b-9fe2-73c71d6b3ea6"}]
web_1 | received event "join" from MYTT9opV_r3ojokGAAAB [/chat]
web_1 | wJY-VBKTJkEoTPuMAAAA: Received packet MESSAGE data 2/chat,["join",{"room":"room_2dc7564d-1d6b-4d15-9534-d678c496c8cf_1d404a18-6e59-4e3b-9fe2-73c71d6b3ea6"}]
web_1 | received event "join" from MYTT9opV_r3ojokGAAAB [/chat]
web_1 | wJY-VBKTJkEoTPuMAAAA: Sending packet PING data None
web_1 | wJY-VBKTJkEoTPuMAAAA: Received packet PONG data

version 5.7.0 produces the "unable to connect"
web_1 | V15MxnAk8QE4ToWRAAAA: Sending packet OPEN data {'sid': 'V15MxnAk8QE4ToWRAAAA', 'upgrades': [], 'pingTimeout': 20000, 'pingInterval': 25000}
web_1 | redis backend initialized.
web_1 | V15MxnAk8QE4ToWRAAAA: Received request to upgrade to websocket
web_1 | V15MxnAk8QE4ToWRAAAA: Upgrade to websocket successful
web_1 | V15MxnAk8QE4ToWRAAAA: Received packet MESSAGE data 0/chat,
web_1 | V15MxnAk8QE4ToWRAAAA: Sending packet MESSAGE data 4/chat,"Unable to connect"

and up to version 5.12.1 I've tested each one and
Sending packet MESSAGE data 4/chat,"Unable to connect" is returning on it self

I'm using flask 2.3.2 been upgrading each version and I'm still getting unable to connect.

What am I doing wrong?

the "Unable to connect" appears only when adding the message_queue=f"{REDIS_MESSAGE_QUEUE_URL}",

Another question:
if Im using websocket protocol instead of long polling I will be able to use multiple workers? and the socket will keep working? no problems with the gunicorn and nginx as we dont need sticky sessions?

At the moment this is my command on docker:
CMD ["gunicorn", "--worker-class", "geventwebsocket.gunicorn.workers.GeventWebSocketWorker", "--workers", "1", "--timeout", "120","--keep-alive","10", "--bind", "0.0.0.0:8000", "--access-logfile", "-", "--error-logfile", "-", "hello_app.application:create_app()"]

[MeshulashAdmin]

Found a solution, when i do
from .services.socketio_module import socketio
socketio.init_app(app,
message_queue=f"{REDIS_MESSAGE_QUEUE_URL}",
async_mode='gevent',
cors_allowed_origins="*")

within the create_app function it works excellent. (I use factory pattern)
It connects, everything is working as expected and sending data.
But when on the socketio.py it self from my socketio_module i do
socketio = SocketIO(
max_http_buffer_size=MAX_BUFFER_SIZE,
logger=True,
engineio_logger=True,
message_queue=f"{REDIS_MESSAGE_QUEUE_URL}",
async_mode='gevent',
cors_allowed_origins="*"
)

then i get unable to connect I hope it will help someone out there..

https://www.youtube.com/watch?v=tdIIJuPh3SI

this video helped me to better understand how things are working.