Interprocess Communication (IPC)

[josephwinston]

I have some numeric kernels that would benefit from codon. What would be the best way to communicate to these processes given the current constraints of Codon?

Normally, for web I'd be using WebSockets <-> FastAPI <-> Numeric code. When I can drop the web, I'd be using Apache Arrow <-> Numeric code.

What is the best pattern here?

[arshajii]

Hi @josephwinston -- there are a couple different options:

1. You can use Codon in JIT mode, which would allow you to use any Python libraries you currently use and pass the data as ndarrays to a Codon-JIT'd function.
2. You can serialize data to a NumPy binary format (e.g. a .npy file or just a raw binary file that can be read with np.fromfile()) and then read it from Codon.

We're planning to add native support for Arrow in the near future as well. Let me know if I can expand on or clarify any of this!