Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions 20250226-actors-pubsub.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Actor PubSub Implementation Proposal

- Author(s): @joshvanl

## Overview

This proposal introduces PubSub capabilities for Dapr Actors, enabling actors to subscribe to messages for their actor type.
Additionally, actors of a that type have the ability to publish messages to that type, to be delivered to particular actor IDs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we're limiting messaging to actors of the same type? I think we should consider cross-actor and app to actor communication as well.

A single designated PubSub component will be marked as the actor PubSub for that namespace, similar to how an actor state store is defined.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for only allowing one pubsub/statestore per namespace? Could this limit scaling or have more granular storage isolation?

There can be only one actor PubSub Component per namespace.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While having a means to set up a standing subscription like this is useful to subscribe all actors of a given type, I'd also like to see actors able to set up something similar to streaming subscriptions in which specific actor instances of a type can subscribe to a named pubsub component type, a variable topic name and assign their own filter (filtered at the runtime) so the actor is only activated on a match.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opening up the possibility for a streaming connection per actor ID that outlives the actor instance itself makes me a little bit uncomfortable. We should look into how this would scale.

Actors will be spawned, if they are not already, when they receive an actor PubSub message.

## Background

Today, Dapr provides PubSub functionality for applications but lacks native PubSub integration for actors.
This limitation means that actors cannot communicate efficiently via publish-subscribe messaging patterns within their type.
By enabling Actor PubSub, actors can publish messages to specific actor IDs, improving scalability and communication reliability.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circling back to my other comment, having a more dynamic subscription would enable an actor to far more easily subscribe to messages from other actor types altogether (huge for scenarios where an actor "manager" is responsible for cross-domain activities).

In light of line 24, I'll write a separate proposal.


### Considerations

There are a number of scenarios or use cases which could benefit from an "Actor PubSub" feature.
These include implicitly or explicitly subscribing to topics as an actor, whether publishing messages will spawn actors, if the message is broadcast to a single or all active actors, and if multiple topics can exist within a single actor type domain.
This proposal describes a single topic per actor type that is implicitly subscribed to by all actors of that type.
Other scenarios can be considered in future iterations.

## Implementation Details

### Component

A single PubSub component will be designated as the actor PubSub, similar to how an actor state store is configured.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a huge fan of this as:

  1. I feel it really limits potential pubsub applications for Dapr Actors to inter-type reliable messaging instead of of allowing actors to respond to events happening across the rest of the deployed apps
  2. It forces me to use a single pubsub provider for all actors across a solution (requiring that I split up projects to deploy with different components files for each) and potentially running into quota limits (e.g. 5k active connections on Azure Service Bus), which, across multiple actor Apps, might be hit rather quickly.

It is intentional there can only be a single PubSub to a namespace to ensure that infrastructure is completely abstracted from actor developer consumers.
It is also the case having multiple PubSubs would complicate subscribing, forcing actors to watch for PubSub resources and resubscribe as necessary.
It seems logical to keep parity of single Component for both state and PubSub messaging.

The metadata configuration will follow the existing state store pattern:

```yaml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: joshvanl
spec:
type: pubsub.joshvanl
version: v1
metadata:
- name: actorpubsub # Added metadata field for all PubSubs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, Im ok with this proposal as a starting point to be expanded upon. Just a few clarification questions:

Could you provide an example or 2 for this feature from an end user perspective and corresponding diagram depicting it in action?

Does the actorPubSub inherit the pubsub resiliency policies if available or do we need a resiliency policy for this? I see we have a pubsub and actor target.

Similarly, what happens with in-flight messages if an actor instance is deactivated while processing? Should actor deactivation wait for in-flight messages to complete?

value: "true"
```

Only one PubSub component can be marked as the actor PubSub.
Any PubSub type can be used for actors.

### Actor Subscription

On actor type registration, if the Actor PubSub is available, the Dapr runtime will subscribe to the actor PubSub topic for that actor type.
This topic string will take the format of:

```
dapr.actors||$namespace||$actorType
```

Namespace is included to allow for multi-tenancy support of a single PubSub broker across namespaces.
The app ID is not included as actor types transcend app IDs in a namespace.
Upon un-registration of an actor type, the Dapr runtime will unsubscribe from the actor type's PubSub topic.

### API

To facilitate publishing a message to a actor ID, the following client API will be added to the Dapr runtime.
Notice that the message type is similar to the [existing PublishEvent API](https://github.com/dapr/dapr/blob/955436f45f783e52c9af8c2ae32f7f82a287c39c/dapr/proto/runtime/v1/dapr.proto#L381), but with fields dedicated to actor routing and endpoint invocation.
The metadata field will have the same functionality and uses as the [existing PublishEvent API](https://github.com/dapr/dapr/blob/955436f45f783e52c9af8c2ae32f7f82a287c39c/dapr/proto/runtime/v1/dapr.proto#L381).
A client does not need to be an actor or host that actor type to use this API.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how will authz work on this endpoint?

The only requirement is that the actor PubSub is configured.

```proto
service Dapr {
// PublishActorEventAlpha1 publishes an event to an actor ID.
rpc PublishActorEventAlpha1(PublishActorEventRequestAlpha1) returns (google.protobuf.Empty) {}
}

// PublishActorEventRequestAlpha1 is the message to publish event data to an
// actor ID.
message PublishActorEventRequestAlpha1 {
// actor_type is the type of the actor to publish to.
// Required.
string actor_type = 1;

// actor_id is the ID of the actor_type to publish to.
// Required.
string actor_id = 2;

// method is the endpoint of the actor to invoke with the published data.
// Required.
string method = 3;

// The data which will be published to topic.
bytes data = 4;

// The content type for the data (optional).
optional string data_content_type = 5;

// The metadata passing to pub components
//
// metadata property:
// - key : the key of the message.
optional map<string, string> metadata = 6;
}
```

### Event Message Routing

Upon the Daprd runtime receiving an actor PubSub message, the runtime will wrap the message with a CloudEvent envelope as usual.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm entirely in favor of Cloud events being necessary so routing is feasible.

The PubSub used will be the static PubSub Component marked as the actor PubSub, the topic string built using the format above.
If no actor PubSub is defined, an appropriate typed error will be returned to the client.

The Actor PubSub message CloudEvent envelope will have the following fields added to allow for routing of the message on delivery.

```
dapr.actors.id
dapr.actors.method
```

An example of the CloudEvent envelope:

```json
{
"specversion": "1.0",
"type": "com.dapr.event.sent",
"source": "joshvanl",
"id": "5929aaac-a5e2-4ca1-859c-edfe73f11565",
"time": "1970-01-01T00:00:00Z",
"datacontenttype": "application/json",
"data": {
"message": "Hello, World!"
},
"dapr.actors.id": "MyActorID",
"dapr.actors.method": "MyActorMethod"
}
```

Upon the daprd runtime receiving an actor PubSub message, the runtime will unwrap the CloudEvent envelope and route the message to the appropriate actor instance.
The actor type and ID to route is easily extracted from the topic the message was received from, and the ID from the CloudEvent envelope.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to only cater for direct actor id delivery, how can we delver to all actors of a type? omit the ID? THe there are 2 scenarios, as WhitWaldo states, current Actors that are alive, every other actor that did or will exist. Probably not feasible to cater for the latter.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orleans handles the latter (in conjunction with pubsub providers that support token offsets) by making the whole of the stream available from the beginning (with a token) or the last received (without) to any new actor instances that show up. That way, there's no expectation that the messages be kept in perpetuity for instances that don't exist yet, but the subscription is implicitly created when the instance is called into existence.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is indeed possible with some supported pubsub components (eg. stream-log brokers like Redis, Kafka) but messages in other components (like Pulsar or RabbitMQ for example) are removed from the queue when acknowledged. If we go this route docs should make it clear which pubsub components support it.

The data payload will be serialized just as is for normal subscriptions today.
The method field will be used to invoke the actor at that method with the data payload.
The result of that invocation will determine the response code sent back to the PubSub broker component.

All Daprds who share a hosted actor type will subscribe to the same PubSub topic.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still love to see routing here, specifically so some inbound event doesn't activate every virtual actor that's ever been and ever will be when only three of them cared about the content.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an actor controlled the subscription to the actor type topic or any topic for that matter, it might allow for that actor to expire but still maintain the subscription for it to be reinvoked and brought back to life. The actor would have to be responsible for unsubcribing in the future or we could put a TTL on the subscription itself so it can clean up after a period of time if that was desirable.

These Daprds will receive messages from the subscription for that type, destined for IDs which they often will not be currently hosting.
These messages will be routed (proxied) to the correct daprd host, based on the placement table hash.
Proxying requests like this is already implemented and used in the actor runtime engine today, so no work needs to be added to support this.

SDKs will need to be updated to support publishing actor PubSub messages.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No doubt

No changes to SDKs need to be made to support _receiving_ actor PubSub messages.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know about this - at least .NET would need to know about the start/end of the turn for reentrancy and state cache purposes.


## Feature Lifecycle Outline

- Add Subscription on Actor Type Registration & PubSub Component Configuration
- Add PublishActorEventAlpha1 API
- Update SDKs to support `PublishActorEventAlpha1` API