feat: lw13d4 (supabase#30908)

* lw13:d4

* www queues page (supabase#30907)

* new products menu

* set up cron page

* set up cron page

* placeholder sections

* set up queues page

* update meta

* update Supabase Queues landing page content

* Updated the text to exclude queue creation/management from the API management section

* queues www page

---------

Co-authored-by: Wen Bo Xie <[email protected]>
Co-authored-by: Greg Papas <[email protected]>

* feat: add queue module (supabase#30853)

* feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
docs homepage layout

feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
docs homepage layout

feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
feat: postgres integrations

Create a new global navigation and homepage menu section for the Postgres Integrations category.
reorg: move cron docs into postgres integrations
docs homepage layout

* fix: add pg_cron back to extensions sidebar

* update Supabase Cron docs

* feat: add queues module


feat: add queues module

* remove staging url from screenshots

* remove outdated cron files

* typo

* queues description update

---------

Co-authored-by: Francesco Sansalvadore <[email protected]>
Co-authored-by: Wen Bo Xie <[email protected]>
Co-authored-by: Oliver Rice <[email protected]>

* visibility timeout to window

* queues landing page sql example

* capitalize Dashboard

* cron blog post citus data callout update

* added draft of queues feature

* updated features bulleting

* pluralize queues

* fix Queues features page

* Inital bp

* queues docs updates

* format queues docs

* queues blog post updates

* edits

* Update the images for role in the docs quickstart for queues.

* updated images

* update images

* update more images

* Update 2024-12-05-supabase-queues.mdx

* fix breaks

* punchier title

* better ul

* clean up

* Retake screenshots

* grammar

* whitelines

* video + images

* update docs

* blog updates

* blog update

* api snippet

* api snippet fix

* queues features yt video

* api snippet fix

* update docs

* blog update

* api snippet fix and vale

* format

* vale off

* vale off

* blog update

---------

Co-authored-by: Francesco Sansalvadore <[email protected]>
Co-authored-by: Greg Papas <[email protected]>
Co-authored-by: Charis <[email protected]>
Co-authored-by: Oliver Rice <[email protected]>
Co-authored-by: Terry Sutton <[email protected]>
Co-authored-by: Copple <[email protected]>
Co-authored-by: Ivan Vasilov <[email protected]>
Co-authored-by: Jonathan Summers-Muir <[email protected]>

Author: 9 people

apps/docs/app/page.tsx

@@ -79,12 +79,12 @@ const postgresIntegrations = [
     href: '/guides/cron',
     description: 'Schedule and monitor recurring Jobs',
   },
-  // {
-  //   title: 'Queues',
-  //   icon: 'queue',
-  //   href: '/guides/queue',
-  //   description: 'Postgres-native pull queues',
-  // },
+  {
+    title: 'Queues',
+    icon: 'queues',
+    href: '/guides/queues',
+    description: 'Durable Message Queues with guaranteed delivery',
+  },
 ]
 
 const selfHostingOptions = [

apps/docs/components/Navigation/NavigationMenu/MenuIconPicker.tsx

@@ -95,7 +95,7 @@ function getMenuIcon(menuKey: string, width: number = 16, height: number = 16, c
       return <Server width={width} height={height} className={className} />
     case 'cron':
       return <Clock width={width} height={height} className={className} />
-    case 'queue':
+    case 'queues':
       return <SquareStack width={width} height={height} className={className} />
     default:
       return <IconMenuPlatform width={width} height={height} className={className} />

apps/docs/components/Navigation/NavigationMenu/NavigationMenu.constants.ts

@@ -63,12 +63,12 @@ export const GLOBAL_MENU_ITEMS: GlobalMenuItems = [
             href: '/guides/cron',
             level: 'cron',
           },
-          // {
-          //   label: 'Queues',
-          //   icon: 'queue',
-          //   href: '/guides/queues',
-          //   level: 'queue',
-          // },
+          {
+            label: 'Queues',
+            icon: 'queues',
+            href: '/guides/queues',
+            level: 'queues',
+          },
         ],
       ],
     },
@@ -1146,6 +1146,25 @@ export const cron: NavMenuConstant = {
   ],
 }
 
+export const queues: NavMenuConstant = {
+  icon: 'queues',
+  title: 'Queues',
+  url: '/guides/queues',
+  items: [
+    { name: 'Overview', url: '/guides/queues' },
+    {
+      name: 'Getting Started',
+      url: undefined,
+      items: [{ name: 'Quickstart', url: '/guides/queues/quickstart' }],
+    },
+    {
+      name: 'References',
+      url: undefined,
+      items: [{ name: 'API', url: '/guides/queues/api' }],
+    },
+  ],
+}
+
 export const api: NavMenuConstant = {
   icon: 'rest',
   title: 'REST API',

apps/docs/components/Navigation/NavigationMenu/NavigationMenu.tsx

@@ -16,6 +16,7 @@ enum MenuId {
   Storage = 'storage',
   Ai = 'ai',
   Cron = 'cron',
+  Queues = 'queues',
   Platform = 'platform',
   Deployment = 'deployment',
   MonitoringTroubleshooting = 'monitoring_troubleshooting',
@@ -78,6 +79,10 @@ const menus: Menu[] = [
     id: MenuId.Graphql,
     type: 'guide',
   },
+  {
+    id: MenuId.Queues,
+    type: 'guide',
+  },
   {
     id: MenuId.Auth,
     type: 'guide',

apps/docs/components/Navigation/NavigationMenu/NavigationMenu.utils.ts

@@ -129,6 +129,8 @@ export const getMenuId = (pathname: string | null) => {
       return MenuId.MonitoringTroubleshooting
     case pathname.startsWith('platform'):
       return MenuId.Platform
+    case pathname.startsWith('queues'):
+      return MenuId.Queues
     case pathname.startsWith('realtime'):
       return MenuId.Realtime
     case pathname.startsWith('resources'):

apps/docs/content/guides/queues.mdx

@@ -0,0 +1,34 @@
+---
+title: Supabase Queues
+subtitle: 'Durable Message Queues with Guaranteed Delivery in Postgres'
+---
+
+Supabase Queues is a Postgres-native durable Message Queue system with guaranteed delivery built on the [pgmq database extension](https://github.com/tembo-io/pgmq). It offers developers a seamless way to persist and process Messages in the background while improving the resiliency and scalability of their applications and services.
+
+Queues couples the reliability of Postgres with the simplicity Supabase's platform and developer experience, enabling developers to manage Background Tasks with zero configuration.
+
+## Features
+
+- **Postgres Native**
+  <br />
+  Built on top of the `pgmq` database extension, create and manage Queues with any Postgres tooling.
+- **Guaranteed Message Delivery**
+  <br />
+  Messages added to Queues are guaranteed to be delivered to your consumers.
+- **Exactly Once Message Delivery**
+  <br />A Message is delivered exactly once to a consumer within a customizable visibility window.
+- **Message Durability and Archival**
+  <br />
+  Messages are stored in Postgres and you can choose to archive them for analytical or auditing purposes.
+- **Granular Authorization**
+  <br />
+  Control client-side consumer access to Queues with API permissions and Row Level Security (RLS) policies.
+- **Queue Management and Monitoring**
+  <br />
+  Create, manage, and monitor Queues and Messages in the Supabase Dashboard.
+
+## Resources
+
+- [Quickstart](/docs/guides/queues/quickstart)
+- [API Reference](/docs/guides/queues/api)
+- [`pgmq` GitHub Repository](https://github.com/tembo-io/pgmq)

apps/docs/content/guides/queues/api.mdx

@@ -0,0 +1,62 @@
+---
+title: API
+---
+
+{/* <!-- vale off --> */}
+When you create a Queue in Supabase, you can choose to create helper database functions in the `pgmq_public` schema. This schema exposes operations to manage Queue Messages to consumers client-side, but does not expose functions for creating or dropping Queues.
+
+Database functions in `pgmq_public` can be exposed via Supabase Data API so consumers client-side can call them. Visit the [Quickstart](/docs/guides/queues/quickstart) for an example.
+
+## `pgmq_public.pop(queue_name)`
+
+Retrieves the next available message and deletes it from the specified Queue.
+
+- `queue_name` (text): Queue name
+
+---
+
+### `pgmq_public.send(queue_name, message, sleep_seconds)`
+
+Adds a Message to the specified Queue, optionally delaying its visibility to all consumers by a number of seconds.
+
+- `queue_name` (text): Queue name
+- `message` (jsonb): Message payload to send
+- `sleep_seconds` (integer, optional): Delay message visibility by specified seconds. Defaults to 0
+
+---
+
+### `pgmq_public.send_batch(queue_name, messages, sleep_seconds)`
+
+Adds a batch of Messages to the specified Queue, optionally delaying their availability to all consumers by a number of seconds.
+
+- `queue_name` (text): Queue name
+- `messages` (jsonb[]): Array of message payloads to send
+- `sleep_seconds` (integer, optional): Delay messages visibility by specified seconds. Defaults to 0
+
+---
+
+### `pgmq_public.archive(queue_name, message_id)`
+
+Archives a Message by moving it from the Queue table to the Queue's archive table.
+
+- `queue_name` (text): Queue name
+- `message_id` (bigint): ID of the Message to archive
+
+---
+
+### `pgmq_public.delete(queue_name, message_id)`
+
+Permanently deletes a Message from the specified Queue.
+
+- `queue_name` (text): Queue name
+- `message_id` (bigint): ID of the Message to delete
+
+---
+
+### `pgmq_public.read(queue_name, sleep_seconds, n)`
+
+Reads up to "n" Messages from the specified Queue with an optional "sleep_seconds" (visibility timeout).
+
+- `queue_name` (text): Queue name
+- `sleep_seconds` (integer): Visibility timeout in seconds
+- `n` (integer): Maximum number of Messages to read

apps/docs/content/guides/queues/quickstart.mdx

@@ -0,0 +1,195 @@
+---
+title: Quickstart
+subtitle: 'Learn how to use Supabase Queues to add and read messages'
+---
+
+{/* <!-- vale off --> */}
+This guide is an introduction to interacting with Supabase Queues via the Dashboard and official client library. Check out [Queues API Reference](/docs/guides/queues/api) for more details on our API.
+
+## Concepts
+
+Supabase Queues is a pull-based Message Queue consisting of three main components: Queues, Messages, and Queue Types.
+
+### Pull-Based Queue
+
+A pull-based Queue is a Message storage and delivery system where consumers actively fetch Messages when they're ready to process them - similar to constantly refreshing a webpage to display the latest updates. Our pull-based Queues process Messages in a First-In-First-Out (FIFO) manner without priority levels.
+
+### Message
+
+A Message in a Queue is a json object that is stored until a consumer explicitly processes and removes it, like a task waiting in a to-do list until someone checks and completes it.
+
+### Queue Types
+
+Supabase Queues offers three types of Queues:
+
+- **Basic Queue**: A durable Queue that stores Messages in a logged table.
+- **Unlogged Queue**: A transient Queue that stores Messages in an unlogged table for better performance but may result in loss of Queue Messages.
+
+- **Partitioned Queue** (_Coming Soon_): A durable and scalable Queue that stores Messages in multiple table partitions for better performance.
+
+## Create Queues
+
+To get started, navigate to the [Supabase Queues](/dashboard/project/_/integrations/queues/overview) Postgres Module under Integrations in the Dashboard and enable the `pgmq` extension.
+
+<Admonition type="note">
+
+`pgmq` extension is available in Postgres version 15.6.1.143 or later.
+
+</Admonition>
+
+<Image
+  alt="Supabase Dashboard Integrations page, showing the Queues Postgres Module"
+  src={{
+    dark: '/docs/img/queues-quickstart-install.png',
+    light: '/docs/img/queues-quickstart-install.png',
+  }}
+/>
+
+On the [Queues page](/dashboard/project/_/integrations/queues/queues):
+
+- Click **Add a new queue** button
+
+<Admonition type="note">
+
+If you've already created a Queue click the **Create a queue** button instead.
+
+</Admonition>
+
+- Name your queue
+
+<Admonition type="note">
+
+Queue names can only be lowercase and hyphens and underscores are permitted.
+
+</Admonition>
+
+- Select your [Queue Type](#queue-types)
+
+<Image
+  alt="Create a Queue from the Supabase Dashboard"
+  src={{
+    dark: '/docs/img/queues-quickstart-create.png',
+    light: '/docs/img/queues-quickstart-create.png',
+  }}
+  zoomable
+  className="max-w-lg !mx-auto"
+/>
+
+### What happens when you create a queue?
+
+Every new Queue creates two tables in the `pgmq` schema. These tables are `pgmq.q_<queue_name>` to store and process active messages and `pgmq.a_<queue_name>` to store any archived messages.
+
+A "Basic Queue" will create `pgmq.q_<queue_name>` and `pgmq.a_<queue_name>` tables as logged tables.
+
+However, an "Unlogged Queue" will create `pgmq.q_<queue_name>` as an unlogged table for better performance while sacrificing durability. The `pgmq.a_<queue_name>` table will still be created as a logged table so your archived messages remain safe and secure.
+
+## Expose Queues to client-side consumers
+
+Queues, by default, are not exposed over Supabase Data API and are only accessible via Postgres clients.
+
+However, you may grant client-side consumers access to your Queues by enabling the Supabase Data API and granting permissions to the Queues API, which is a collection of database functions in the `pgmq_public` schema that wraps the database functions in the `pgmq` schema.
+
+This is to prevent direct access to the `pgmq` schema and its tables (RLS is not enabled by default on any tables) and database functions.
+
+To get started, navigate to the Queues [Settings page](/dashboard/project/_/integrations/queues/settings) and toggle on “Expose Queues via PostgREST”. Once enabled, Supabase creates and exposes a `pgmq_public` schema containing database function wrappers to a subset of `pgmq`'s database functions.
+
+<Image
+  alt="Screenshot of Queues settings with toggle to expose to PostgREST"
+  src={{
+    dark: '/docs/img/queues-quickstart-settings.png',
+    light: '/docs/img/queues-quickstart-settings.png',
+  }}
+/>
+
+### Enable RLS on your tables in `pgmq` schema
+
+For security purposes, you must enable Row Level Security (RLS) on all Queue tables (all tables in `pgmq` schema that begin with `q_`) if the Data API is enabled.
+
+You’ll want to create RLS policies for any Queues you want your client-side consumers to interact with.
+
+<Image
+  alt="Screenshot of creating an RLS policy from the Queues settings"
+  src={{
+    dark: '/docs/img/queues-quickstart-rls.png',
+    light: '/docs/img/queues-quickstart-rls.png',
+  }}
+/>
+
+### Grant permissions to `pgmq_public` database functions
+
+On top of enabling RLS and writing RLS policies on the underlying Queue tables, you must grant the correct permissions to the `pgmq_public` database functions for each Data API role.
+
+The permissions required for each Queue API database function:
+
+| **Operations**      | **Permissions Required** |
+| ------------------- | ------------------------ |
+| `send` `send_batch` | `Select` `Insert`        |
+| `read` `pop`        | `Select` `Update`        |
+| `archive` `delete`  | `Select` `Delete`        |
+
+<Image
+  alt="Screenshot of configuring API access for roles from the Queues settings"
+  src={{
+    dark: '/docs/img/queues-quickstart-roles.png',
+    light: '/docs/img/queues-quickstart-roles-light.png',
+  }}
+/>
+
+<Admonition type="note">
+
+`postgres` and `service_role` roles should never be exposed client-side.
+
+</Admonition>
+
+### Enqueuing and Dequeuing Messages
+
+Once your Queue has been created, you can begin enqueuing and dequeuing Messages.
+
+Here's a TypeScript example using the official Supabase client library:
+
+```tsx
+import { createClient } from '@supabase/supabase-js'
+
+const supabaseUrl = 'supabaseURL'
+const supabaseKey = 'supabaseKey'
+
+const supabase = createClient(supabaseUrl, supabaseKey)
+
+const QueuesTest: React.FC = () => {
+  //Add a Message
+  const sendToQueue = async () => {
+    const result = await supabase.rpc('send', {
+      queue_name: 'foo',
+      message: { hello: 'world' },
+      sleep_seconds: 30,
+    })
+    console.log(result)
+  }
+
+  //Dequeue Message
+  const popFromQueue = async () => {
+    const result = await supabase.rpc('pop', { queue_name: 'foo' })
+    console.log(result)
+  }
+
+  return (
+    <div className="p-6">
+      <h2 className="text-2xl font-bold mb-4">Queue Test Component</h2>
+      <button
+        onClick={sendToQueue}
+        className="bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600 mr-4"
+      >
+        Add Message
+      </button>
+      <button
+        onClick={popFromQueue}
+        className="bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600"
+      >
+        Pop Message
+      </button>
+    </div>
+  )
+}
+
+export default QueuesTest
+```