CHA-1452 Event-Platform Update http subscription delivery policy (#1035)

Author: Vlaquit

content/event-platform/concepts.md

@@ -51,8 +51,21 @@ The statuses for a subscription are:
 
 ## Subscription types
 
+::: info
+We will consider adding other subscription destinations based on feedback. Please [fill-in this form](https://forms.gle/XsZ7rovRnqfAn4xF9) to propose & upvote new destination types.
+:::
+
+
 ### Pub/Sub subscription
 
+This is the **preferred and most resilient method** for consuming events. It leverages Google Cloud's managed messaging service.
+
+#### Key Advantages
+
+- **Managed Scalability & Reliability:** Google Pub/Sub inherently handles high throughput and traffic bursts, absorbing the complexities of scaling.
+- **Reduced Infrastructure Burden:** Significantly lowers the operational overhead and risk for the customer.
+- **Simplified Integration:** Easier setup and maintenance compared to managing a custom HTTP endpoint.
+
 #### Configuration
 
 For the `pubsub` subscription type, the `config` property needed when creating the subscription requires both the project ID and the topic ID.
@@ -90,6 +103,25 @@ For comprehensive details on managing subscriptions, consult the complete API re
 
 ### HTTPS subscription
 
+This option pushes events directly to a HTTP endpoint. It offers flexibility but requires careful infrastructure planning.
+
+#### Critical Requirements & Considerations
+
+- **Strong Infrastructure:** The endpoint must be highly available and designed to handle variable event loads.
+    - This also depends on the **source PIM's size and activity**. A small PIM with few SKUs won't generate the same volume as a large instance with many jobs. Thus, the **HTTP endpoint** can be adjusted based on the PIM's intended use.
+- **Mandatory Rate Limiting (HTTP 429):**
+    - Our platform **dynamically adjusts its delivery rate from 1 to 100 events per second.**
+    - This adaptive mechanism **relies entirely on the endpoint correctly returning an `HTTP 429 "Too Many Requests"` status** when its capacity is reached. [ref](https://api.akeneo.com/event-platform/key-platform-behaviors.html#optimized-throughput)
+
+::: warning
+Endpoints without proper HTTP 429 handling are at high risk of being overwhelmed, leading to event loss and subscription suspension.
+:::
+
+- **Fast Acknowledgement:** Endpoints must respond with an `HTTP 200 OK` within **5 seconds**. Delays trigger retries and can lead to suspension. [ref](https://api.akeneo.com/event-platform/key-platform-behaviors.html#delivery-timeout)
+    - **Recommendation:** Implement an asynchronous processing model (e.g., using an internal queue) to acknowledge events quickly and process them separately.
+- **Suspension Policy:** Subscriptions may be suspended due to repeated errors (e.g., `4xx`/`5xx` status codes, timeouts, misconfigured SSL). [ref](https://api.akeneo.com/event-platform/key-platform-behaviors.html#suspension-policy)
+
+
 #### Configuration
 
 For the `https` type, the `config` property requires:
@@ -177,6 +209,8 @@ We currently use a static IP address provided by Google Cloud: `34.140.80.128`
 
 When configuring a subscription, you can optionally define a **filter** to receive **only the events that match specific criteria**.
 
+**Regardless of the destination type, we strongly advise to utilize event filters.** This ensures you only subscribe to and receive events that are relevant to your specific integration needs.
+
 You can find the list of currently available filters and the correct syntax to use [here](/event-platform/available-filters.html).
 
 ### Example

content/event-platform/key-platform-behaviors.md

@@ -1,4 +1,3 @@
-
 # Key platform behaviors
 
 Your subscribing service implementation and architecture must deal with the following capabilities and constraints to consume events from the platform at its best.
@@ -14,15 +13,27 @@ To help identify duplicated events and deal with un-ordered events if it's somet
 
 ## Optimized throughput
 
-Our delivery engine will try to deliver events as fast as possible but will adapt the throughput within the limits described in this section.
+If your subscription has an HTTPS destination, our delivery engine adapts the event delivery rate based on your system's capacity, operating within these limits:
+
+**Maximum rate:** `100` events per second
+
+**Minimum rate:** `1` event per second
+
+
+The throughput automatically adjusts between these limits based on your endpoint's responses:
+- `200 OK`: The delivery rate gradually increases up to the maximum rate
+- `429 Too Many Requests`: The delivery rate decreases to prevent system overload
+
+::: warning
+Events that are throttled and remain undelivered for more than one hour will negatively impact your success rate, as it indicates a potential queuing risk. This may trigger the [suspension policy](/event-platform/key-platform-behaviors.html#suspension-policy).
+:::
 
-If your subscription has an HTTPS destination, please respond with `429 Too Many Requests` when your system is overloaded: this way, the delivery engine will slow down, retry undelivered events, and gradually increase the throughput when your system gets back to recovers (i.e., when it responds `200 OK` again).
+Consider using another [subscription type](/event-platform/concepts.html#subscription-types) to get around these limitations.
 
-Please note that if the throughput drops too significantly, the suspension policy will be triggered ([see bellow](/event-platform/key-platform-behaviors.html#suspension-policy)).
 
 ## Delivery timeout
 
-Specifically for the HTTPS destination type, the delivery timeout ensures that messages are processed within a specified  time frame. Your endpoint is expected to handle requests within **`5 seconds`**.  If processing exceeds this duration, the event will enter the retry process ([see bellow](/event-platform/concepts.html#retry-policy-for-transient-failures.html)).
+Specifically for the HTTPS destination type, the delivery timeout ensures that messages are processed within a specified  time frame. Your endpoint is expected to handle requests within **`5 seconds`**.  If processing exceeds this duration, the event will enter the retry process ([see bellow](/event-platform/concepts.html#retry-policy-for-transient-failures)).
 
 Under normal circumstances, your HTTPS endpoint must handle the event as fast as possible.
 **Our recommendation** is to put the message in a `queuing system` or in a `database` for asynchronous processing.
@@ -53,12 +64,12 @@ Your subscription is immediately suspended if you meet one of these conditions:
 
 ### Threshold-Based Suspension:
 
-This type of suspension is based on the success rate of your HTTPS endpoint. If the success rate drops below 95% within the last rolling hour, your subscription will be suspended.
+This type of suspension is based on the success rate of your HTTPS endpoint. If the success rate drops below 90% within the last rolling hour, your subscription will be suspended.
 
 Here are the errors type that decrease the success rate:
 
 - `5XX Server Error` HTTP statuses
-- `429 Too Many Requests` HTTP statuses (while the minimum rate threshold is exceeded)
+- `429 Too Many Requests` HTTP statuses (while the event remains undelivered for more than one hour)
 - `4xx Client Error Response` HTTP status
 - Delivery timeout