WebSockets

Full duplex real time communication

Important points:

• The connection is created by .accept()
• It's not actually an infinite loop. Under the hood it used non-blocking TCP sockets and performs like any asynchronous code.
• Handling disconnect states is really important.
• A connection manager is needed for appyling connection logic.
• Need to store active connections in a hash map

Use cases:

• chat applications
• any collaborative tasks

Note: Some older browsers might have problems with WS connections and maynot traverse proxies easily.

My test case

Websocket Test Code

1. Whenever client connects they will recieve a greeting like "Hello Client 1"
2. Client 1 and 2 can chat among themselves (private)
3. Client 3 broadcasts anything they send to everyone

Testing done with postman

Server Sent Events (SSE)

Unidirectional communication channel

Important points:

• Only server can send data to the client unlike tarditional request-response architecture
• Every transfered data is in text format ending with blankline or '\n\n'. Also every message can be an event and is applicable to its own logic
• Unlike websockets, sse allows reconnection and works over both HTTP/1.1 and HTTP/2
• Under the hood it is just chunked HTTP streaming with accept: text/event-stream.

Use cases:

• Displaying logs
• streaming llm generations
• geolocations

Note:

1. There can only be atmost 6 simultaneous sessions open on a single browser
2. It is useful for low frequency one way updates
3. SSE has specific parsing format:
   every chunk is: "data: {chunk}\n\n"
   last chunk or event end: "event: done\ndata: [DONE]\n\n"

My test case

Server Sent Events test code:

1. Stream responses back from ollama
2. running ollama on docker:

    docker run -d --gpus=all \
    -v ollamamodels:/root/.ollama \
    -p 11434:11434
    --name ollama
    ollama/ollama

3. download ollama python sdk
4. tested with postman