Best practices for sending millions of events to a Pekko actor without running out of memory(actor as Nats Consumer of 10 secnds interval)

[mcpawali]

Hi all,

I have a scenario with Pekko actors where I am implementing a NATS consumer actor that fetches batches of events from a NATS Object Store every 10 seconds. Each batch can contain millions of events, with a total size of around 100 MB.

After fetching, the consumer actor needs to send the events to a processor actor for handling. I see two approaches:

1. Sending each event individually (processorActor ! event)
2. Sending the entire batch as a single message (processorActor ! eventsArray)

I’m concerned about:

• Actor mailbox memory usage and backpressure
• Serialization limits (e.g., maximum-frame-size)
• Processing throughput
• Hot shards / distribution if using cluster sharding

My questions:

1. What is the recommended pattern for safely sending millions of events from a NATS consumer actor to a processor actor in Pekko?
2. Should I batch messages (e.g., 1k–10k events per message), or is sending the whole batch (~100 MB) ever acceptable?
3. Any best practices for cluster sharding or distributing load for high-volume message processing?

Thanks in advance for your guidance!