[Queues] Add rate limit tutorial (#16797)

* add rate limit tutorial

* Update src/content/docs/queues/tutorials/handle-rate-limits/index.mdx

Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com>

* [Hyperlint] Update checks to be more forgiving for Wrangler capitalization + add new check + remove logic around backticks

* Revert "[Hyperlint] Update checks to be more forgiving for Wrangler capitalization + add new check + remove logic around backticks"

This reverts commit 645183c.

* update queues resend tutorial

* update repo link and minor changes

* minor fixes

* update Render componenet params

* update the category

---------

Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com>
Co-authored-by: kodster28 <[email protected]>

Author: 3 people

src/content/docs/queues/tutorials/handle-rate-limits/index.mdx

@@ -0,0 +1,389 @@
+---
+updated: 2024-09-25
+difficulty: Beginner
+title: Handle rate limits of external APIs
+summary: Example of how to use Queues to handle rate limits of external APIs.
+content_type: 📝 Tutorial
+pcx_content_type: tutorial
+products:
+  - Workers
+  - Queues
+languages:
+  - TypeScript
+sidebar:
+  order: 1002
+head:
+  - tag: title
+    content: Cloudflare Queues - Queues & Rate Limits
+description: Example of how to use Queues to handle rate limits of external APIs.
+---
+
+import { Render, PackageManagers } from "~/components";
+
+This tutorial explains how to use Queues to handle rate limits of external APIs by building an application that sends email notifications using [Resend](https://www.resend.com/). However, you can use this pattern to handle rate limits of any external API.
+
+Resend is a service that allows you to send emails from your application via an API. Resend has a default [rate limit](https://resend.com/docs/api-reference/introduction#rate-limit) of two requests per second. You will use Queues to handle the rate limit of Resend.
+
+## Prerequisites
+
+<Render file="prereqs" product="workers" />
+
+4. Sign up for [Resend](https://resend.com/) and generate an API key by following the guide on the [Resend documentation](https://resend.com/docs/dashboard/api-keys/introduction).
+
+5. Additionally, you will need access to Cloudflare Queues.
+
+<Render file="enable-queues" />
+
+## 1. Create a new Workers application
+
+To get started, create a Worker application using the [`create-cloudflare` CLI](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare). Open a terminal window and run the following command:
+
+<PackageManagers
+	type="create"
+	pkg="cloudflare@latest"
+	args={"resend-rate-limit-queue"}
+/>
+
+<Render
+	file="c3-post-run-steps"
+	product="workers"
+	params={{
+		category: "hello-world",
+		type: "Hello World Worker",
+		lang: "TypeScript",
+	}}
+/>
+
+Then, go to your newly created directory:
+
+```sh frame="none"
+cd resend-rate-limit-queue
+```
+
+## 2. Set up a Queue
+
+You need to create a Queue and a binding to your Worker. Run the following command to create a Queue named `rate-limit-queue`:
+
+```sh title="Create a Queue"
+npx wrangler queues create rate-limit-queue
+```
+
+```sh output
+Creating queue rate-limit-queue.
+Created queue rate-limit-queue.
+```
+
+### Add Queue bindings to `wrangler.toml`
+
+In your `wrangler.toml` file, add the following:
+
+```toml
+[[queues.producers]]
+binding = "EMAIL_QUEUE"
+queue = "rate-limit-queue"
+
+[[queues.consumers]]
+queue = "rate-limit-queue"
+max_batch_size = 2
+max_batch_timeout = 10
+max_retries = 3
+```
+
+It is important to include the `max_batch_size` of two to the consumer queue is important because the Resend API has a default rate limit of two requests per second. This batch size allows the queue to process the message in the batch size of two. If the batch size is less than two, the queue will wait for 10 seconds to collect the next message. If no more messages are available, the queue will process the message in the batch. For more information, refer to the [Batching, Retries and Delays documentation](/queues/configuration/batching-retries)
+
+Your final `wrangler.toml` file should look similar to the example below.
+
+```toml title="wrangler.toml"
+#:schema node_modules/wrangler/config-schema.json
+name = "resend-rate-limit-queue"
+main = "src/index.ts"
+compatibility_date = "2024-09-09"
+compatibility_flags = ["nodejs_compat"]
+
+[[queues.producers]]
+binding = "EMAIL_QUEUE"
+queue = "rate-limit-queue"
+
+[[queues.consumers]]
+queue = "rate-limit-queue"
+max_batch_size = 2
+max_batch_timeout = 10
+max_retries = 3
+```
+
+## 3. Add bindings to environment
+
+Add the bindings to the environment interface in `worker-configuration.d.ts`, so TypeScript correctly types the bindings. Type the queue as `Queue<any>`. Refer to the following step for instructions on how to change this type.
+
+```ts title="worker-configuration.d.ts"
+interface Env {
+	EMAIL_QUEUE: Queue<any>;
+}
+```
+
+## 4. Send message to the queue
+
+The application will send a message to the queue when the Worker receives a request. For simplicity, you will send the email address as a message to the queue. A new message will be sent to the queue with a delay of one second.
+
+```ts title="src/index.ts"
+export default {
+	async fetch(req: Request, env: Env): Promise<Response> {
+		try {
+			await env.EMAIL_QUEUE.send(
+				{ email: await req.text() },
+				{ delaySeconds: 1 },
+			);
+			return new Response("Success!");
+		} catch (e) {
+			return new Response("Error!", { status: 500 });
+		}
+	},
+};
+```
+
+This will accept requests to any subpath and forwards the request's body. It expects that the request body to contain only an email. In production, you should check that the request was a `POST` request. You should also avoid sending such sensitive information (email) directly to the queue. Instead, you can send a message to the queue that contains a unique identifier for the user. Then, your consumer queue can use the unique identifier to look up the email address in a database and use that to send the email.
+
+## 5. Process the messages in the queue
+
+After the message is sent to the queue, it will be processed by the consumer Worker. The consumer Worker will process the message and send the email.
+
+Since you have not configured Resend yet, you will log the message to the console. After you configure Resend, you will use it to send the email.
+
+Add the `queue()` handler as shown below:
+
+```ts title="src/index.ts" ins={1-3,17-28}
+interface Message {
+	email: string;
+}
+
+export default {
+	async fetch(req: Request, env: Env): Promise<Response> {
+		try {
+			await env.EMAIL_QUEUE.send(
+				{ email: await req.text() },
+				{ delaySeconds: 1 },
+			);
+			return new Response("Success!");
+		} catch (e) {
+			return new Response("Error!", { status: 500 });
+		}
+	},
+	async queue(batch: MessageBatch<Message>, env: Env): Promise<void> {
+		for (const message of batch.messages) {
+			try {
+				console.log(message.body.email);
+				// After configuring Resend, you can send email
+				message.ack();
+			} catch (e) {
+				console.error(e);
+				message.retry({ delaySeconds: 5 });
+			}
+		}
+	},
+};
+```
+
+The above `queue()` handler will log the email address to the console and send the email. It will also retry the message if sending the email fails. The `delaySeconds` is set to five seconds to avoid sending the email too quickly.
+
+To test the application, run the following command:
+
+```sh title="Start the development server"
+npm run dev
+```
+
+Use the following cURL command to send a request to the application:
+
+```sh title="Test with a cURL request"
+curl -X POST -d "[email protected]" http://localhost:8787/
+```
+
+```sh output
+[wrangler:inf] POST / 200 OK (2ms)
+QueueMessage {
+  attempts: 1,
+  body: { email: '[email protected]' },
+  timestamp: 2024-09-12T13:48:07.236Z,
+  id: '72a25ff18dd441f5acb6086b9ce87c8c'
+}
+```
+
+## 6. Set up Resend
+
+To call the Resend API, you need to configure the Resend API key. Create a `.dev.vars` file in the root of your project and add the following:
+
+```txt title=".dev.vars"
+RESEND_API_KEY='your-resend-api-key'
+```
+
+Replace `your-resend-api-key` with your actual Resend API key.
+
+Next, update the `Env` interface in `worker-configuration.d.ts` to include the `RESEND_API_KEY` variable.
+
+```ts title="worker-configuration.d.ts" ins={3}
+interface Env {
+	EMAIL_QUEUE: Queue<any>;
+	RESEND_API_KEY: string;
+}
+```
+
+Lastly, install the [`resend` package](https://www.npmjs.com/package/resend) using the following command:
+
+```sh title="Install Resend"
+npm install resend
+```
+
+You can now use the `RESEND_API_KEY` variable in your code.
+
+## 7. Send email with Resend
+
+In your `src/index.ts` file, import the Resend package and update the `queue()` handler to send the email.
+
+```ts title="src/index.ts" ins={1,21,26-40} del={24,41}
+import { Resend } from "resend";
+
+interface Message {
+	email: string;
+}
+
+export default {
+	async fetch(req: Request, env: Env): Promise<Response> {
+		try {
+			await env.EMAIL_QUEUE.send(
+				{ email: await req.text() },
+				{ delaySeconds: 1 },
+			);
+			return new Response("Success!");
+		} catch (e) {
+			return new Response("Error!", { status: 500 });
+		}
+	},
+	async queue(batch: MessageBatch<Message>, env: Env): Promise<void> {
+		// Initialize Resend
+		const resend = new Resend(env.RESEND_API_KEY);
+		for (const message of batch.messages) {
+			try {
+				console.log(message.body.email);
+				// send email
+				const sendEmail = await resend.emails.send({
+					from: "[email protected]",
+					to: [message.body.email],
+					subject: "Hello World",
+					html: "<strong>Sending an email from Worker!</strong>",
+				});
+
+				// check if the email failed
+				if (sendEmail.error) {
+					console.error(sendEmail.error);
+					message.retry({ delaySeconds: 5 });
+				} else {
+					// if success, ack the message
+					message.ack();
+				}
+				message.ack();
+			} catch (e) {
+				console.error(e);
+				message.retry({ delaySeconds: 5 });
+			}
+		}
+	},
+};
+```
+
+The `queue()` handler will now send the email using the Resend API. It also checks if sending the email failed and will retry the message.
+
+The final script is included below:
+
+```ts title="src/index.ts"
+import { Resend } from "resend";
+
+interface Message {
+	email: string;
+}
+
+export default {
+	async fetch(req: Request, env: Env): Promise<Response> {
+		try {
+			await env.EMAIL_QUEUE.send(
+				{ email: await req.text() },
+				{ delaySeconds: 1 },
+			);
+			return new Response("Success!");
+		} catch (e) {
+			return new Response("Error!", { status: 500 });
+		}
+	},
+	async queue(batch: MessageBatch<Message>, env: Env): Promise<void> {
+		// Initialize Resend
+		const resend = new Resend(env.RESEND_API_KEY);
+		for (const message of batch.messages) {
+			try {
+				// send email
+				const sendEmail = await resend.emails.send({
+					from: "[email protected]",
+					to: [message.body.email],
+					subject: "Hello World",
+					html: "<strong>Sending an email from Worker!</strong>",
+				});
+
+				// check if the email failed
+				if (sendEmail.error) {
+					console.error(sendEmail.error);
+					message.retry({ delaySeconds: 5 });
+				} else {
+					// if success, ack the message
+					message.ack();
+				}
+			} catch (e) {
+				console.error(e);
+				message.retry({ delaySeconds: 5 });
+			}
+		}
+	},
+};
+```
+
+To test the application, start the development server using the following command:
+
+```sh title="Start the development server"
+npm run dev
+```
+
+Use the following cURL command to send a request to the application:
+
+```sh title="Test with a cURL request"
+curl -X POST -d "[email protected]" http://localhost:8787/
+```
+
+On the Resend dashboard, you should see that the email was sent to the provided email address.
+
+## 8. Deploy your Worker
+
+To deploy your Worker, run the following command:
+
+```sh title="Deploy your Worker"
+npx wrangler deploy
+```
+
+Lastly, add the Resend API key using the following command:
+
+```sh title="Add the Resend API key"
+npx wrangler secret put RESEND_API_KEY
+```
+
+Enter the value of your API key. Your API key will get added to your project. You can now use the `RESEND_API_KEY` variable in your code.
+
+You have successfully created a Worker which can send emails using the Resend API respecting rate limits.
+
+To test your Worker, you could use the following cURL request. Replace `<YOUR_WORKER_URL>` with the URL of your deployed Worker.
+
+```bash title="Test with a cURL request"
+curl -X POST -d "[email protected]" <YOUR_WORKER_URL>
+```
+
+Refer to the [GitHub repository](https://github.com/harshil1712/queues-rate-limit) for the complete code for this tutorial. If you are using [Hono](https://hono.dev/), you can refer to the [Hono example](https://github.com/harshil1712/resend-rate-limit-demo).
+
+## Related resources
+
+- [How Queues works](/queues/reference/how-queues-works/)
+- [Queues Batching and Retries](/queues/configuration/batching-retries/)
+- [Resend](https://resend.com/docs/)