Skip to content

Commit c71c622

Browse files
kzukzu
committed
Add support for interactive flows
Comprehensive support for flow call to action response, data exchange endpoint handling and final flow response processing too. The same webhook for regular messages can now be used for arbitrary flow endpoints too: - Automatic handling of flow ping/health checks - Automatic decryption/encryption of flow request/responses - Seamless integration with existing handler pipeline via FlowDataRequest/Response messages and InteractiveFlowMessage (final interactive response).
1 parent 405c48c commit c71c622

33 files changed

+2025
-81
lines changed

‎docs/whatsapp/flows/media.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
[WhatsApp Flows](https://developers.facebook.com/docs/whatsapp/flows)
2+
3+
# Media upload components
4+
5+
WhatsApp does not guarantee that data (such as images, videos, or documents) shared with you by your customers is non-malicious. Make sure to implement appropriate risk mitigations when processing such data (for example, using well-tested and up-to-date media and document processing libraries).
6+
7+
Media upload components are not supported by the On-Premise API client. Please refer to the [deprecation announcent](https://developers.facebook.com/docs/whatsapp/on-premises/sunset/) and learn how to [migrate to Cloud API.](https://developers.facebook.com/docs/whatsapp/cloud-api/guides/migrating-from-onprem-to-cloud)
8+
9+
# Flow JSON components
10+
11+
Two components can be used to ask users to upload media
12+
13+
* PhotoPicker: allows uploading media from camera or gallery
14+
* DocumentPicker: allows uploading media from files or gallery
15+
16+
## PhotoPicker
17+
18+
_Supported in Flow JSON version 7.2._
19+
20+
| Parámetro | Descripción |
21+
|--------------------------|-------------|
22+
| `type` (required) string | PhotoPicker |
23+
| `name` (required) string | Component's name. Should be distinct among all components on a screen. |
24+
| `label` (required) string | Header text for the component. Dynamic "${data.label}" OR "${screen.<screen_id>.data.label}" Max length: 80 characters |
25+
| `description` string | Body text for the component. Dynamic "${data.description}" OR "${screen.<screen_id>.data.description}" Max length: 300 characters |
26+
| `photo-source` enum | Specifies the source where the image can be selected from. Values: {'camera_gallery', 'camera', 'gallery'} Default: 'camera_gallery' * camera_gallery: user can select from gallery or take a photo * gallery: user can select only from gallery * camera: user can only take a photo |
27+
| `max-file-size-kb` Integer | Specifies the maximum file size (in kibibytes) that can be uploaded. Default value: 25600 (25 MiB) Allowed range: [1, 25600] |
28+
| `min-uploaded-photos` Integer | Specifies the minimum number of photos that are required. This property determines whether the component is optional (set to 0) or required (set above 0). Default value: 0 Allowed range: [0, 30] Note: Above limits apply if media files are sent to the endpoint via "data_exchange" action. For images or documents sent as part of the response message, no more of 10 files can be attached. Additionally, the aggregated size of them cannot exceed 100 MiB. |
29+
| `max-uploaded-photos` Integer | Specifies the maximum number of photos that can be uploaded. Default value: 30 Allowed range: [1, 30] Note: Above limits apply if media files are sent to the endpoint via "data_exchange" action. For images or documents sent as part of the response message, no more of 10 files can be attached. Additionally, the aggregated size of them cannot exceed 100 MiB. |
30+
| `enabled` Boolean \| String | Specifies if user interaction will be enabled on the component(true = enabled, false = disabled). Dynamic "${data.is_enabled}" OR "${screen.<screen_id>.data.is_enabled}" Default: true |
31+
| `visible` Boolean \| String | Specifies if the component will be visible on the screen(true = visible, false = hidden). Dynamic "${data.is_visible}" OR ${screen.<screen_id>.data.visible}" Default: true |
32+
| `error-message` String \| Object | Specifies errors when processing the images. Dynamic "${data.error_message}" * String: specifies a generic error for the whole component. * Object: specifies image specific errors. Only use with dynamic data as media id must be supplied The format for Object is the following: {"media_id_1" : "error_message 1", "media_id_2" : "error_message 2"} Check the [endpoint handling](#endpoint-media-handling) section below to find out more about the media ids. |
33+
34+
### **Example**
35+
36+
Note that the image selection behaviour is mocked in this preview. The actual behaviour on device will be similar to the image selection in WhatsApp chats.
37+
38+
### Limitations and Restrictions
39+
40+
The table below outlines the constraints associated with the PhotoPicker component.
41+
42+
| Constraint | Validation error |
43+
|------------|-----------------|
44+
| `min-uploaded-photos` should not exceed `max-uploaded-photos` | "min-uploaded-photos" cannot be greater than "max-uploaded-photos" for PhotoPicker component ${component_name}. |
45+
| PhotoPicker cannot be initialised using Form `init-values` | Invalid value found for property at ${path}. "init-values" property should not contain a value for PhotoPicker component. |
46+
| Only 1 PhotoPicker is allowed per screen | You can only have a maximum of 1 component of type PhotoPicker per screen. |
47+
| Using both PhotoPicker and DocumentPicker components on a single screen is not allowed. | You can only have a maximum of 1 component of type PhotoPicker or DocumentPicker per screen. |
48+
| The PhotoPicker is not allowed in the `navigate` action payload. To access the component's value from a different screen, you can utilize [Global Dynamic Referencing.](https://developers.facebook.com/docs/whatsapp/flows/reference/flowjson#global-data) | The PhotoPicker component's value is not allowed in the payload of the navigate action. |
49+
| The PhotoPicker component is restricted to top-level usage within the payloads of the `data_exchange` or `complete` action. Valid: <br> ``` "on-click-action": { "name": "data_exchange", "payload": { "media": "${form.photo_picker}" } } ``` <br> Invalid: <br> ``` "on-click-action": { "name": "data_exchange", "payload": { "media": {"photo": "${form.photo_picker}"} } } ``` | The PhotoPicker can only be used as the value of a top-level string property in the action payload. |
50+
| No more than 10 images or documents can be sent as part of the response message. | Additionally, the maximum aggregated size of attached images or documents cannot exceed 100 MiB. |
51+
52+
## DocumentPicker
53+
54+
_Supported in Flow JSON version 7.2._
55+
56+
| Parámetro | Descripción |
57+
|--------------------------|-------------|
58+
| `type` (required) string | DocumentPicker |
59+
| `name` (required) string | Component's name. Should be distinct among all components on a screen. |
60+
| `label` (required) string | Header text for the component. Dynamic "${data.label}" OR "${screen.<screen_id>.data.label}" Max length: 80 characters |
61+
| `description` string | Body text for the component. Dynamic "${data.description}" OR "${screen.<screen_id>.data.description}" Max length: 300 characters |
62+
| `max-file-size-kb` Integer | Specifies the maximum file size (in kibibytes) that can be uploaded. Default value: 25600 (25 MiB) Allowed range: [1, 25600] |
63+
| `min-uploaded-documents` Integer | Specifies the minimum number of documents that are required. This property determines whether the component is optional (set to 0) or required (set above 0). Default value: 0 Allowed range: [0, 30] Note: Above limits apply if media files are sent to the endpoint via "data_exchange" action. For images or documents sent as part of the response message, no more of 10 files can be attached. Additionally, the aggregated size of them cannot exceed 100 MiB. |
64+
| `max-uploaded-documents` Integer | Specifies the maximum number of documents that can be uploaded. Default value: 30 Allowed range: [1, 30] Note: Above limits apply if media files are sent to the endpoint via "data_exchange" action. For images or documents sent as part of the response message, no more of 10 files can be attached. Additionally, the aggregated size of them cannot exceed 100 MiB. |
65+
| `allowed-mime-types` Array <string> | Specifies which document mime types can be selected. If it contains “image/jpeg”, picking photos from the gallery will be available as well. Default: Any document from the supported mime types can be selected Supported values: 1. application/gzip 2. application/msword 3. application/pdf 4. application/vnd.ms-excel 5. application/vnd.ms-powerpoint 6. application/vnd.oasis.opendocument.presentation 7. application/vnd.oasis.opendocument.spreadsheet 8. application/vnd.oasis.opendocument.text 9. application/vnd.openxmlformats-officedocument.presentationml.presentation 10. application/vnd.openxmlformats-officedocument.spreadsheetml.sheet 11. application/vnd.openxmlformats-officedocument.wordprocessingml.document 12. application/x-7z-compressed 13. application/zip 14. image/avif 15. image/gif 16. image/heic 17. image/heif 18. image/jpeg 19. image/png 20. image/tiff 21. image/webp 22. text/plain 23. video/mp4 24. video/mpeg Note: some old Android and iOS OS versions don’t understand all mime types above. As a result, a user might be able to select a file with a different mime type to the ones specified. |
66+
| `enabled` Boolean \| String | Specifies if user interaction will be enabled on the component(true = enabled, false = disabled). Dynamic "${data.is_enabled}" OR "${screen.<screen_id>.data.is_enabled}" Default: true |
67+
| `visible` Boolean \| String | Specifies if the component will be visible on the screen(true = visible, false = hidden). Dynamic "${data.is_visible}" OR ${screen.<screen_id>.data.visible}" Default: true |
68+
| `error-message` String \| Object | Specifies errors when processing the documents. Dynamic "${data.error_message}" * String: specifies a generic error for the whole component. * Object: specifies document specific errors. Only use with dynamic data as media id must be supplied The format for Object is the following: {"media_id_1" : "error_message 2", "media_id_2" : "error_message 2"} Check the [endpoint handling](#endpoint-media-handling) section below to find out more about the media ids. |
69+
70+
### **Example**
71+
72+
Note that the document selection behaviour is mocked in this preview. The actual behaviour on device will be similar to the document selection in WhatsApp chats.
73+
74+
### Limitations and Restrictions
75+
76+
The table below outlines the constraints associated with the DocumentPicker component.
77+
78+
| Constraint | Validation error |
79+
|------------|-----------------|
80+
| `min-uploaded-documents` should not exceed `max-uploaded-documents` | "min-uploaded-documents" cannot be greater than "max-uploaded-documents" for DocumentPicker component ${component_name}. |
81+
| DocumentPicker cannot be initialised using Form `init-values` | Invalid value found for property at ${path}. "init-values" property should not contain a value for DocumentPicker component. |
82+
| Only 1 DocumentPicker is allowed per screen | You can only have a maximum of 1 component of type DocumentPicker per screen. |
83+
| Using both PhotoPicker and DocumentPicker components on a single screen is not allowed. | You can only have a maximum of 1 component of type PhotoPicker or DocumentPicker per screen. |
84+
| The DocumentPicker is not allowed in the `navigate` action payload. To access the component's value from a different screen, you can utilize [Global Dynamic Referencing.](https://developers.facebook.com/docs/whatsapp/flows/reference/flowjson#global-data) | The DocumentPicker component's value is not allowed in the payload of the navigate action. |
85+
| The DocumentPicker component is restricted to top-level usage within the payloads of the `data_exchange` or `complete` action. Valid: <br> ``` "on-click-action": { "name": "data_exchange", "payload": { "media": "${form.document_picker}" } } ``` <br> Invalid: <br> ``` "on-click-action": { "name": "data_exchange", "payload": { "media": {"document": "${form.document_picker}"} } } ``` | The DocumentPicker can only be used as the value of a top-level string property in the action payload. |
86+
| No more than 10 images or documents can be sent as part of the response message. | Additionally, the maximum aggregated size of attached images or documents cannot exceed 100 MiB. |
87+
88+
# Handling media
89+
90+
## Endpoint
91+
92+
Media uploaded by the users are temporarily stored in WhatsApp CDN. Files are encrypted using AES256-CBC+HMAC-SHA256+pkcs7 cryptographic algorithms.
93+
94+
In your endpoint implementation, you must download, decrypt, and validate each media file.
95+
96+
Here’s a payload example for a photo or document.
97+
98+
```
99+
100+
"photo_picker":[{
101+
"media_id": "790aba14-5f4a-4dbd-aa9e-0d75401da14b",
102+
"cdn_url": "https://mmg.whatsapp.net/v/redacted",
103+
"file_name": "IMG_5237.jpg"
104+
"encryption_metadata": {
105+
"encrypted_hash": "/QvkBvpBED2q2AHPIFuhXfLpkn22zj2kO6ggzjvhHv0=",
106+
"iv": "5SHjLrrsfPXTSJTcbrVSkg==",
107+
"encryption_key": "lPa4SXcWbk3sy2so3OxjyXmpV4aE6CcIKd+4byr5hBw=",
108+
"hmac_key": "15l+E9Z5gcL15WH9OQ8GgK7VVCKkfbVigoSiM9djvGU=",
109+
"plaintext_hash": "AOF2dHXVEpm9efk9udNy3R1cUJWnpjFwQKGBEdALqXI="
110+
}]
111+
112+
```
113+
### **Decrypting and validating media**
114+
115+
The files stored in WhatsApp CDN contain the encrypted media and the first 10 bytes of the HMAC-SHA256 (concatenated at the end). For reference, cdn\_file = ciphertext & hmac10
116+
117+
Perform the following steps to decrypt the media:
118+
119+
1. Download cdn\_file file from cdn\_url
120+
2. Make sure SHA256(cdn\_file) == enc\_hash
121+
3. Validate HMAC-SHA256
122+
1. Calculate HMAC with hmac\_key, initialization vector (encryption\_metadata.iv) and ciphertex
123+
2. Make sure first 10 bytes == hmac10
124+
4. Decrypt media content
125+
1. Run AES with CBC mode and initialization vector (encryption\_metadata.iv) on ciphertex
126+
2. Remove padding (AES256 uses blocks of 16 bytes, padding algorithm is pkcs7). We’ll call this decrypted\_media
127+
5. Validate the decrypted media
128+
129+
* Make sure SHA256(decrypted\_media) = plaintext\_hash
130+
## Response message (Cloud API)
131+
132+
Media can be received in the [response message webhook](https://developers.facebook.com/docs/whatsapp/flows/reference/responsemsgwebhook).
133+
134+
Here’s a truncated example using PhotoPicker (same structure for DocumentPicker)
135+
136+
```
137+
138+
{
139+
"nfm_reply": {
140+
// [... redacted ... ]
141+
"response_json": {
142+
"photo_picker": [
143+
{
144+
"file_name": "IMG_5237.jpg",
145+
"mime_type": "image/jpeg",
146+
"sha256": "PqHgadp8cJ/N6mvAYGNMxhs9Ra5hbZFcctCtCClXsMU=",
147+
"id": "3631120727156756"
148+
}
149+
],
150+
"flow_token": "xyz",
151+
"name": "John"
152+
}
153+
}
154+
}
155+
156+
```
157+
158+
The media can be downloaded following the same [steps as for regular image and document messages](https://developers.facebook.com/docs/whatsapp/cloud-api/reference/media/#download-media).
159+
160+
[←AnteriorComponents](/docs/whatsapp/flows/reference/flowjson/components)![](https://www.facebook.com/tr?id=675141479195042&ev=PageView&noscript=1)![](https://www.facebook.com/tr?id=574561515946252&ev=PageView&noscript=1)![](https://www.facebook.com/tr?id=1754628768090156&ev=PageView&noscript=1)

0 commit comments

Comments
 (0)