minimizing GC allocations with Websocket.send_message

[jbrelwof]

Motivation

Every call to Websocket.sendMessage(...) appears to be allocating memory which will hang around until the next GC cycle. I'm checking directly before/after the call, i.e.

            gc_after_message_create = gc.mem_free()
            self.websocket.send_message(message, fail_silently=True)
            gc_after = gc.mem_free()

so it's not including anything due to the creation/formatting of message.

Sending a 29 character JSON packet leaves 112 bytes allocated. A 31 or 32 character packet leaves 128 bytes.

If there's much traffic, these can add up quickly (especially considering that there's also the allocation overhead for the message passed in and artifacts of it's formatting) and force more frequent gc collection cycles.

This isn't a simple thing to "fix", AFAIK there may be significant limitations on what can be done with the current API. However, some/most/maybe all of the leftover allocations could potentially be eliminated by having a separate object managing all the message bits, which could then be passed in to a new method (or "overloaded" on send_message with an internal isinstance check)

Current behavior

gc "leaks" (leftovers per invocation of send_message until next gc.collect()) include

• passing message as str | bytes - both require an immutable per-message allocation before you can even call send_message(...)
• message.encode()* for str/text messages - likely creates a copy
• in def _prepare_frame(opcode: int, message: bytes) -> bytearray:
  • creating new bytearray for frame - another copy
  • payload_length.to_bytes(2, "big") and payload_length.to_bytes(8, "big") both leave allocations (32 bytes)
    • but frame.append( (payload_length>>24) & 0xFF ) ...does not
• _ISocket
  • ```def send(self, data: bytes) -> int: ...`` - if this actually uses ore creates data as bytes internally (not just a memoryview), that's potentially another copy

The three potential internal copies (message encode, frame bytearray, _Isocket.send) plus the leftovers from payload_length.to_bytes(...) appear to match fairly closely with my measured results (on an admittedly small sample set).

Potential solutions

Eliminating this overhead (probably???) requires some form of reusable mutable buffer which ideally can be instantiated and passed in by the caller.
Might be good to wrap this buffer as a member of an object which can handle things like synchronization (don't change it until it's done writing...). Possibilities for the buffer include

• io.StringIO / io.BinIO
  • can be partially reset using instance.seek(0)
  • but unfortunately
    • instance.truncate() is not implemented, and even if it were it only "works" if it holds and reuses allocated memory (like calling clear() on a C++ std::vector but not calling shrink_to_fit() - "resets" the array but retains the underlying memory / capacity )
    • doesn't seem to provide access to the underlying data except through copy (getValue()) i.e. no memoryview
• bytearray
  • not resettable, no way to "rewind" / seek(0)
  • not shrinkable
    • CircuitPython bytearray does not support slice assignment
    • in in regular CPython you can use del byteArrayInstace[:]
    • same caveat as truncate above - only "helps" if it holds memory
• ulab.numpy.ndarray(...,dtype=uint8)
  • also not shrinkable

Whatever is used needs a way to pass the buffer data without reallocation to whatever is actually implementing _ISocket.send. It seems that the minimum requirement would be memorview / readable buffer protocol support. There might also need to be some form of notification of send failed/complete unless _ISocket.send(...) is a fully blocking implementation (in which case, there might be another feature request before long...)

I suppose this (probably more than) enough to get the conversation started...